Hello,
I have a query has been running in the last 3months without any error (it is always 9min 53s -56s). no changes to the query.
However, today we received the below error message. we did not change the running schedule.
when I hit “re-run” it works perfectly. anyway that I can avoid it happen in the future ?
Error type:
SOURCE_RESOURCE_LIMIT_EXCEEDED Learn more
This query exceeds the resource limit of the data source. Examples of resources involved can include: concurrent query limit, connection limit, and physical server resources. Contact your data source administrator for assistance.
Error details:
Query exhausted resources at this scale factor [Execution ID: 82f36cbd-c981-4080-a04b-dece4b67cff7]
It will be nice to inviestigate if there was something going on in the source at the moment you got the error. Based on the error I think it is Athena right?
I would recommend to open a support case and point the specific query and time you saw the failure happing. Here are the steps to open a support case. If your company has someone who manages your AWS account, you might not have direct access to AWS Support and will need to raise an internal ticket to your IT team or whomever manages your AWS account. They should be able to open an AWS Support case on your behalf.
Depending on the root cause of the issue you might want to take different actions to avoid this in the future. One option to make your refresh more resilient, can be to programatically catch the refresh error or gonig over your specific threshold and restart/cancel it.
You can find a list of the APIs available to manage SPICE ingestions here
thank you! @andres007 do you know what will be the process? we are interested in doing a auto restart refresh. it seems we might need our QS account admin to create “CreateRefreshSchedule” . Then, the auto restart will be activated. or I miss some steps?
I am in the process of writing an article with an approach to the solution to this problem, I will let you know when it is ready. In a nutshell it involves creating a Cloudwatch alarm, which triggers a state machine that starts a new ingestion and checks the status of the ingestion until it finishes or fails again, with some configurable retries and notifications.