I am struggling to find a definitive answer on this and I was hoping someone could help me.
When we perform an incremental refresh on a SQL table, does Quicksight pass the date of the window start as a parameter to the underlying query, or does it pull all the data from the database and only load the data from the window start time into SPICE?
I am trying to reduce the load on my database, and I don’t quite understand whether incremental refresh is only used when loading data from the database into Spice after the query is run, or if it actually passes the date into the underlying query so that the database returns fewer rows. Can anybody confirm how it works?
Quick Sight performs incremental refresh by passing the window start date as a filter (typically in the form of a WHERE clause on the chosen timestamp column) directly in the SQL query to the underlying database, so only the data changed or added since the last refresh window is retrieved and loaded into SPICE. This means the database itself handles the filtering and returns only relevant records, minimizing database load, reducing row volume, and speeding up SPICE refresh rather than pulling the entire dataset and filtering it after retrieval.
Please refer to the below documentation this might be helpful for you.
Since we have not heard back, I’ll go ahead and close out this topic. However, if you have any additional questions, feel free to create a new post in the community.