I am seeing a single SPICE refresh trigger 20-50+ simultaneous queries that appear to be impacting our Redshift cluster performance (e.g. occupying all available WLM queues).
For example, I am used to seeing one big query that looks like this for a given dataset:
'/* QuickSight 2eb0da7d-1cd2-4e7c-9b60-bd82eba5ffb6 */
to ‘s3://spaceneedle-redshiftunload.prod.us-east-1/484311b1-50bf-4edb-ac70-d2e451306a9b/484311b1-50bf-4edb-ac70-d2e451306a9b_0764fbc7-a420-4bc4-8b6b-378dceee71f9_’ credentials ‘’ manifest verbose escape addquotes parallel on gzip delimiter ‘,’ region ‘us-east-1’ maxfilesize 500 mb
However, recently, SPICE query triggered multiple queries like the following:
As the data is being loaded into SPICE from your Redshift cluster, QuickSight handles this process by unloading data into S3 and then loading into SPICE. Triggering simultaneous queries could mean that data is being extracted to be loaded into SPICE.
Check with your internal teams which is using QuickSight for their dashboards and find out
1/if it was a one time full load
2/or a scheduled run at specific time
3/or automated to run after ETL/ELT process in Redshift.
The 2nd part with ( QuickSight ID / Partner ) refers to the details of an individual visual in an analysis. Each sheet can have multiple visuals ( sheet id , visual id ) . This is not SPICE but direct query where a SQL query is sent to your Redshift cluster.
Example when i run direct query on my Redshift cluster from a QuickSight visual