What is best connection to quicksight

Question & Answer
, ,
1

Hi Team,

Currently, I have connected lot of datasets using custom SQL based on redshift connection. However, it is taking almost 10 hours time to refresh the datasets and sometimes it is failed.

To avoid, what could be the best solution. Please advise me.

I thought of below:

→ creating materialized views in redshift and connect them to quicksight. Refresh those materialized views at certain times in a day.

→ Import the data to S3 buckets/folders and connect that data to quicksight.

Any better ideas to fix this? please let me know.

2

Hi @salagalakiran123
“10 hours time to refresh the datasets” means you are creating SPICE datasets?
As you are using custom SQL. Can i assume you only query the columns you need?
Any calculations/filter etc. within the custom SQL?
Are you quering views or tables?
BR

3

Yes, I am using spice datasets and custom sql. I am bringing all the required columns only. I am querying the tables in redshift.

4

Are you using incremental or full refresh?

5

I have used full refresh not incremental. I am not sure exactly how incremental refresh works

6

Incremental refresh will load only the changes happend since the last update. But you need a database column (some date) to check.

7

So, you mean if I set up the window to 1 or 2 by choosing the date column from database, refresh will not take much time?

8

Yes. Should be faster if you only load e.g. one day of data.