Thanks for your response! I’m not using an incremental refresh; it’s a full refresh scheduled daily. The data in the S3 file is overwritten every day, and I ensure the file is complete before the refresh. However, once the row count reaches around 1M (typically around the 19th of the month), QuickSight consistently reduces the row count by half during the refresh.
The S3 data remains intact, and the refresh settings are configured for a full import. Could this behavior be related to any limitations or issues with QuickSight’s data ingestion process?
The limits of SPICE are 1 billion (1,000,000,000) rows or 1 TB for each dataset
so I do not think the number of rows is the problem here, unless you have columns with a lot of data where you might be hitting the 1TB before the 1B rows, but as you mention this is related to CUR, I do not think this will be the problem.
Can you confirm that when you query the data in Athena you get the 900k rows instead of the 450k you see in QuickSight?