Quicksight SPICE refreshes triggering multiple Redshift queries

I am seeing a single SPICE refresh trigger 20-50+ simultaneous queries that appear to be impacting our Redshift cluster performance (e.g. occupying all available WLM queues).

For example, I am used to seeing one big query that looks like this for a given dataset:
unload (
'/* QuickSight 2eb0da7d-1cd2-4e7c-9b60-bd82eba5ffb6 */
[select statement]
to ‘s3://spaceneedle-redshiftunload.prod.us-east-1/484311b1-50bf-4edb-ac70-d2e451306a9b/484311b1-50bf-4edb-ac70-d2e451306a9b_0764fbc7-a420-4bc4-8b6b-378dceee71f9_’ credentials ‘’ manifest verbose escape addquotes parallel on gzip delimiter ‘,’ region ‘us-east-1’ maxfilesize 500 mb

However, recently, SPICE query triggered multiple queries like the following:

/* QuickSight d13209cb-22ea-4ce9-9458-7d414090bd9c {“partner”:“QuickSight”,“entityId”:“5cc30c4f-4b2d-41c0-9279-d70aefdc2c61”,“sheetId”:“c1792917-a1d6-4a9e-a3c0-94dc7d6b8101”,“visualId”:“43e8595e-9ab3-4994-a1d0-dea5499abd2f”} */

Request:

  1. Did something change about the SPICE unload mechanism?
  2. How to mitigate / reduce these seemingly simultaneous queues?

Hi,
there are some possibility as I given below.

I think you are using the Direct query methods which engage DB.
otherwise, your incremental refresh frequency it too low.

if query taking much time. You need to review tables indexes etc.

regards,
Naveed Ali

Hi @Winnie_Kuo ,

As the data is being loaded into SPICE from your Redshift cluster, QuickSight handles this process by unloading data into S3 and then loading into SPICE. Triggering simultaneous queries could mean that data is being extracted to be loaded into SPICE.
Check with your internal teams which is using QuickSight for their dashboards and find out
1/if it was a one time full load
2/or a scheduled run at specific time
3/or automated to run after ETL/ELT process in Redshift.

The 2nd part with ( QuickSight ID / Partner ) refers to the details of an individual visual in an analysis. Each sheet can have multiple visuals ( sheet id , visual id ) . This is not SPICE but direct query where a SQL query is sent to your Redshift cluster.

Example when i run direct query on my Redshift cluster from a QuickSight visual

You have 2 scenarios :
1/ Redshift data load to SPICE
2/ Redshift as direct query for an analysis in QuickSight

Regards,
Koushik