Thanks @bstrech for the details. Can you please provide the below details.
- When you performed the joining of large data set with small in Azure data studio and it gave the result, what is the execution time? If it is taking more than 30 Mins, QuickSight spice refresh is timing out ( See the blog - Spice query_timeout - #2 by Jesse).
- What is your underline data source, if it is relational database, can you extract the explain plan for the joining query. The explain plan will provide the details on the cost of the query.
- Is it possible to make the small table as SPICE and then join both tables ( just for POC) and see the result.
Most of the performance tuning we usually do at ETL or database layer or final data set stored in SPICE so that it will NOT hit the DB again. However your usecase is interesting one, and happy to help you as well.
In parallel, please submit a ticket to AWS for see the details as well.
Tagging @Max @Kristin @Tatyana_Yakushev @Bhasi_Mehta @David_Wong @Biswajit_1993 for the expert advise as well.
Regards - San