Dataset refresh in spice based on file update in S3?

Question & Answer
1

Hi Team,

I dump data into S3 from redshift daily, however there is no fixed time for that refresh (as and when upstream dependencies get resolved data gets dumped). Currently the only way i can achieve fastest dataset is with hourly refreshes, however this causes unnecessary refreshes into spice which at times fails as well. I was wondering is there a way that refresh only occurs when data is refreshed in s3?

Thanks,
Tanish

2

Hi @Tanish.Aggarwal - welcome to the QuickSight Community! Thanks for posting your question.

It seems you want to refersh spice based on the S3 data refresh event. You can leverage the QuickSight CreateIngestion API for the same.

Here is a blog post on event-driven refresh of SPICE datasets in Amazon QuickSight.

Please let us know if this is helpful.

2 Likes
3

Hi @Tanish.Aggarwal - Welcome to AWS QuickSight community and thanks for posting the question. Looking at your problem statement, yes @debapc is correct. Here we have to user event driven approach. The idea is when the file extract is complete → event will be created by S3 → which will call a lambda function ( QuickSight API) ingestion API.

Regards - Sanjeeb

1 Like
4

Thanks @Sanjeeb2022 , @debapc, haven’t worked with quicksight programmatically but surely will give this a shot.