I have a dataset which contains up to 98 M rows and is scheduled to refresh every hour, it was refreshing fine until 2nd Aug’24, however, from 2nd Aug’24 it has failed to refresh and the data is not loading to Quicksight. I’m not sure why this is happening. Though it shows refresh completed after taking 5, or 6 hrs the data is not reflected in the dataset or analysts. I’m just wondering why this is happening.
Hello @Bandarusreeni
Welcome to the community!
The issue you’re experiencing with SPICE refresh could be due to several factors, especially considering the large dataset size and the recent change in behavior. Here are a few troubleshooting steps.
SPICE Capacity: Ensure that your dataset isn’t exceeding the SPICE capacity limits for your QuickSight account. If you’re close to or have exceeded the allocated SPICE capacity, the refresh process might show as completed but fail to load data properly. Check your SPICE capacity usage in QuickSight and consider reducing the dataset size, or upgrade your SPICE capacity.
Data Source Availability: The data source may be experiencing issues, leading to partial data loading or long refresh times. Verify the availability and performance of the data source. Ensure that the data source is accessible and that there are no network issues or timeouts during the refresh process.
Complex Transformations: If your dataset involves complex transformations or joins, it could be causing the refresh to take longer, leading to timeouts or failures. Simplify the transformations if possible, or optimize the queries used in the dataset.
Changes in Data Schema: If there were any changes in the schema or data structure in the data source after August 2nd, it might be causing issues with the refresh process. Review any changes in the data source or dataset configuration around that time. Rollback any recent changes to see if they resolve the issue.
Error Logs: Review the error logs for the dataset refresh process. These logs can provide insights into what might be causing the issue.
Please refer the below documentation this might be helpful for you.
If none of these steps resolve the issue, I would recommend filing a case with AWS Support where we can dive into the details so that we can help you further. Here are the steps to open a support case with AWS, providing them with details about the dataset, recent changes, and any logs or error messages you’ve encountered.
1 Like
It’s been a while since we last heard from you, did you have any additional or follow up questions regarding your initial topic? Were you able to try out the approach that was shared?
If we do not hear back within the next 3 business days, I’ll go ahead and mark the solution to close out the topic.
Thank you!
Since we have not heard back from you with any further information or follow-up questions, I will archive this topic. If you still need assistance, please post a new question in the community and link to this topic to provide relevant information. That will ensure you are at the top of the priority list for a response from one of our QuickSight experts.
Thank you!