Hello team ,
I just wanted to know how my data will incrementally refresh if i do not have date level column,
in my sql query i have week_first_date level data. i want to understand the process of incremental data refresh on week level and do i need to put dynamic dates in query ?
and which window unit should i select to optimize my run time and what filters it will use date column or week_first_date
1 Like
Hi @Zeeshan - Welcome to AWS QuickSight community and thanks for posting the question. Can you please have a look on the below documentation - Refreshing SPICE data - Amazon QuickSight
Tagging @duncan @David_Wong for their advise as well.
Regards - Sanjeeb
Hello @Sanjeeb2022 ,
Thank you for replying but thing is on website it is not mentioned how week level data will work in the refreshing part.
in my query i have week level data of sales but i want to check how week level incremental refresh will work in QS
Thanks
Hi @Zeeshan - From your data set, how do you determine weekly data? Is there any field available which will tell or use to find the incremental. If not, basically you can do a full refresh on weekly basis.
Regards - Sanjeeb
1 Like
when incremental refresh happens, it will get the date column from your data source and use Window size to define how many data need to be updated
In this screenshot, I have defined Windows size(number) to be 3 and Window size (unit) to be Day. SPICE will remove data on or after 3 days ago and ingest again. Assume today is Nov 10, it will remove data on or after Nov7 and ingest again
Same logic applies on week, if you define Window size (unit) to be week and Window size(unit) to be 2, it will remove data on or after 14 days ago and ingest again. Assume today is Nov10, it will remove data on or after Oct28 and ingest again
2 Likes
Hello ErikG
yes it is week first date like first date is monday so date is 20/11/2023,
will it work on week level or daily level because i want to refresh on hour level.
so that is what i want to check if we can update it as week level or day level or daily on week_first_date column.
regards
Hello @royyung
i understand your concept of incremental refresh,
suppose i have weekly data in my data set because i want to save my cost of storage in AWS i will not pull data on date level cause it will increase size of data set and it will cost me more.
I wanted to check if we have option to refresh data on the week_first_date which is a weekly column, so i can run incremental refresh on week_firstdate coumn so my numbers of rows will be reduced and cost will be saved and my ingestion will also saved.
regards