Hello,
Question regarding Data Refresh.
Suppose I own 3 Datasets, used in an Analysis that is published in a Dashboard. Each dataset uses a different ingestion option:
- Dataset A) Manually ingested: To refresh the data in the dataset I need to import it again and the associated analysis/dashboard automatically update after the new ingestion has finished. Correct?
- Dataset B) From a Direct Query: To refresh the data in the dataset/analysis/dashboard is enough to open either the dataset or associated analysis or associated dashboard. Correct?
- Dataset C) From SPICE:
- If there is a scheduled refresh, data are refreshed within 10 mins of scheduled time, and then immediately the associated dataset/analysis/dashboard reflect the refreshed data. Correct?
- If I just refresh the data (not scheduled), does the dataset and associated analysis/dashboard reflect immediately the refresh?
What does impact the refresh duration of a dataset? Why sometimes it’s extremely quick and sometimes it takes ages (same dataset, same rows/columns number)?
Thanks in advance!
Hi @Francesca
Dataset A: Manually Ingested - To refresh the data in the dataset, you need to manually import (re-upload or re-import) the data again. Once the new ingestion completes, the associated analysis and dashboards automatically reflect the refreshed data.
Dataset B: Direct Query Refresh - In the case of a Direct Query dataset, the data is always retrieved “live” from the data source each time the dashboard, analysis, or dataset is accessed. You don’t need to manually trigger any refresh — simply opening the dashboard, analysis, or dataset will automatically retrieve the latest data from the source. This also means, if a user changes a value for example in a filter, the data is queried again in the data source.
Dataset C: SPICE
- Scheduled Refresh: When the scheduled refresh completes, immediately after that, the associated dataset, analysis, and dashboard reflect the refreshed data.
- Manual Refresh: When you manually refresh the dataset in SPICE, the updated data becomes available immediately to the associated analysis and dashboards. Once the refresh is complete, the new data is reflected automatically.
Incremental Refresh vs. Full Refresh:
- Incremental refreshes are usually faster than full refreshes, as they only bring in the delta (i.e., the new/modified records).
- Full refreshes will always take longer since the entire dataset is reloaded.
Underlying Database Performance: For datasets with Direct Query or SPICE datasets that involve transformations during ingestion, the performance of the underlying database can also impact refresh speed. A slow data source will make the refresh longer. Each time a filter is applied on a direct query dashboard/analysis, the data is queried again in the data source.
Please refer to the below documentation this might be helpful for you.
Hope this helps!