Is it possible to run an api or is there an inbuilt report which is all failed datasets? Recently we have had datasets failing to load, so we would like to see as of today, or as of most recent load attempt - how many datasets failed to load, list of these datasets, as well as a possible history?
This would give us a better view opposed to receiving emails, because if we had 10 datasets failed, and these were due to refresh every 30 minutes, after 2 hours of them failing we would receive 40 emails. If a different dataset failed once within that 2 hour period, it would be harder to locate.
To summarise, is there a report or way to see all datasets which have failed as of last load attempt, and is there a way to build a report from this data?
Hi @HarveyB-B
QuickSight API allows you to retrieve information about your datasets, including the status of the last refresh. You can use the following API references.
DescribeDataSet: This API retrieves metadata for a dataset, including the refresh status.DescribeIngestion: This API retrieves details about a specific ingestion attempt, including whether it succeeded or failed.
You can automate this process to check the status of all datasets and filter out those that failed to load. By scripting this with a scheduled job (e.g., using Lambda or a Glue job), you can generate a report of failed datasets.
You can monitor dataset ingestions using CloudWatch by enabling logging for QuickSight. QuickSight logs ingestion events that you can filter by status (e.g., FAILED). You can create CloudWatch alarms or build dashboards based on these logs, giving you a view of all failed ingestions.
Please refer the below documentation this might be helpful for you.
1 Like