How to use external data file (Excel File) in the spice redshift dataset under incremental loading?

Question & Answer
1

Hi, I am trying to convert my existing heavy data set from full refresh to incremental loading. I am using redshift data loading and also using a excel file in joins in the respective dataset. Now, incremental loading is showing when i am removing excel file from dataset. But I want to add that for my further analysis dashboard.

Please reply.

Thanks in advance.

2

Create the redshift dataset with incremental loading, and then, create a dataset on top of this dataset: Creating a dataset using an existing dataset in Amazon QuickSight - Amazon QuickSight. In the child dataset, do the join to excel. Let the excel in right hand side while redshift on left hand side to be the driving.

1 Like
3

Hi Ying,

Thanks for the response.
I am facing a issue regarding updating the child dataset at same frequency with the main dataset.

Like main dataset is getting updated every 15 mins , but child dataset doesn’t operate on incremental loading to get update with same frequency as excel is there in the join.

Moreover for analysis child dataset is the source!

Any solution to this issue?

4

Let the child to be directquery, it is live query from the parent so the performance is still very good. And, you don’t have to worry about the refresh in child.

1 Like
5

Thanks , Will check this for sure:)