Hi:
I am trying to load a dataset into SPICE using custom SQL from a MySQL backend data source. I have 8 joins (all 1-1) in the query as well as one table that is a left outer join.
The query works when I run it directly against the data source. The amount of rows I am trying to load is about 65 million.
The refresh into SPICE has not finished after many hours. Is there an issue using so many joins? Is there a way to see how far into the load the query is?
Thanks
Hi. Thanks you for your question! 
Many of our community Experts and folks from our team are on holiday right now. I asked “My Assistant” (universal chat) in Quick Suite, and got this back. @Brett Could you review and validate the answer below on Monday after you return from holiday?
From “My Assistant”:
Dataset refresh time can increase dramatically due to several factors :
Dataset size and complexity - Larger datasets naturally take longer to load into SPICE. One example showed a dataset taking approximately 1 hour to refresh, with the refresh page displaying “successfully refreshed” after ~50 minutes, but the record count continuing to change for another 15 minutes before stabilizing
Data source performance - The underlying data source’s response time and resource availability can significantly impact ingestion speed
SPICE capacity limits - If you’re approaching SPICE capacity limits, performance may degrade
Check for specific error codes - SPICE provides detailed error codes for ingestion issues including:
DATA_SET_SIZE_LIMIT_EXCEEDED - Dataset exceeds maximum allowable SPICE size
ACCOUNT_CAPACITY_LIMIT_EXCEEDED - Data exceeds current SPICE capacity
QUERY_TIMEOUT - Query to data source timed out
SOURCE_RESOURCE_LIMIT_EXCEEDED - SQL query exceeds resource quota
Consider incremental refresh - For large datasets, you can configure incremental refresh to update only changed data rather than performing full refreshes, which can significantly improve performance
Contact support for persistent issues - If dataset refresh times have drastically increased without changes to your data, this may require investigation by the Quick Suite team, as it could indicate backend issues
Hi @Gail_Kiss,
Following up on Kristin’s response above; when you’re dealing with that complex of a setup; long load times on full refreshes may be tough to completely eliminate. The best options in these scenarios would be incremental refreshes as suggested above.
Did the refresh end up finishing? If so, what was the final ingestion time?
Hi @Gail_Kiss,
Since we have not heard back further, I’ll go ahead and mark the solution. However, if you have any additional questions, feel free to let us know.
Thank you