I have a quicksight dataset and dashboard hooked up to an athena table that gets updated every hour. The quicksight data did an incremental refresh every hour. This worked great for ~3 months.
However I recently ran into this error while one of the incremental refreshes was running:
ROW_SIZE_LIMIT_EXCEEDED Learn more
Error details:
Data set exceeded row limit of 1000000000 rows. (maxRowSize = 1000000000)
What are some suggested next steps for this? Should I start deleting data from my athena table? Is there anyway to enforce sampling at the dataset level i.e. quicksight only loads 5% of the data from my athena table?
Hi @chridoms - Welcome to AWS QuickSight and thanks for posting the question. I am assuming your data set is SPICE based. As per spice quota limit, the maximum row you can store for a data set is 1 Billion rows. See the documentation below. Data source quotas - Amazon QuickSight
The other option is you need to clean up the old data from SPICE and make sure you have not cross the SPICE capacity limit.
so if we want to have more than 1 billion rows in a dataset we can’t use SPICE? If we don’t use SPICE will the dashboard be usable or prohibitively slow?
I can’t be the only person with more than 1 billion rows of data that’s not even that much. Does quicksight just not support users with this much data?
Hello @chridoms, if you are uploading your dataset to SPICE, which I highly recommend for the best dashboard performance and load times, you are limited to 500 million rows/500GB if you are on an Enterprise QuickSight account. I would definitely recommend splitting your data into multiple datasets. It will ensure you can continue using SPICE for your datasets and improve refresh times.
You can split the datasets any way you want as long as you don’t need data fields from multiple datasets on a single visual. You can have up to 50 datasets in a single Analysis/Dashboard and multiple datasets can be used on a single sheet. I hope this helps!