[PECO-1263] Add documentation for execute_async #322
Conversation
Signed-off-by: Jesse Whitehouse <[email protected]>
…UID." This reverts commit 1c58313. Signed-off-by: Jesse Whitehouse <[email protected]>
Update changelog. Signed-off-by: Jesse Whitehouse <[email protected]>
Signed-off-by: Jesse Whitehouse <[email protected]>
susodapop
requested review from
arikfr,
yunbodeng-db and
andrefurlan-db
as code owners
Signed-off-by: Jesse Whitehouse <[email protected]>
Signed-off-by: Jesse Whitehouse <[email protected]>
|
Does this method have particular benefits over just running a blocking call in a thread? |
|
Yes, the benefit is to recover a running execution. Imagine you are running a long query (1+ hour) and in the middle, your network cable comes unplugged or your computer runs out of battery. The query continues running on your cluster (incurring cost) but you have no way to recover it or see its results. You can cancel it from the Databricks SQL web UI but apart from that you have no choice but to re-run it and wait another hour for results. In a second usage scenario, if you deploy the connector in a serverless function or lambda. You can wake the lambda to begin executing the query, commit its handle to state, sleep the lambda for some period of time, and wake up to fetch the results. This isn't possible using threads. |
Note for reviewers: Please review this after reviewing #325.
Description
This pull request adds documentation for the forthcoming
execute_async()feature. It describes the interface and gives some usage examples.