Clarification Required: 1GB Limitation for Secondary Tables in Cross-Source JOIN Datasets

Q&A
, , ,
1

Hi QuickSight Team,
We are facing a recurring dataset refresh failure due to the following QuickSight limitation:

“The combined size of all secondary tables must be less than 1 GB for datasets with cross-source JOIN operations. The primary table in our case is ‘Dataset1’, and all other tables are treated as secondary tables.”

I would like to request clarification and guidance on the following points:

  1. Why does this 1GB limitation exist for cross-source JOIN operations?
    We want to understand the technical or architectural reason behind enforcing this limit.

  2. Is there any recommended approach to avoid or mitigate this limitation?
    For example:

    • Any optimization approach for cross-source datasets?

    • Any alternative configuration or architecture that allows larger secondary tables?

    • Best practices to handle large datasets that need to be joined from different sources?

  3. Is AWS planning to increase this limit in future versions of QuickSight?
    Since our datasets naturally grow over time, this limitation is becoming a blocker for business-critical reporting.

This restriction is currently impacting our monthly finance reporting cycle, so we would appreciate your guidance or any workaround to prevent such failures going forward.

Thank you, and looking forward to your support.

Best regards,

Unni

2

Hi @Unni,

Have you tested out using CustomSQL to handle the join, or Direct Query to see if that avoids this limitation?

3

Hi @Brett ,

I’m not fully sure whether the method I’m using is the best approach.
If I’m doing it incorrectly, please suggest a better or more efficient approach.

Currently:

  • Using Custom SQL makes the dataset refresh take 45–50 minutes.

  • Direct Query makes the analysis slow.

  • So I created multiple smaller datasets and connected them to the main dataset to balance performance and refresh time.

If there is a more optimized or recommended way to handle this, please guide me.

image (78)
image (78)1544×668 46.5 KB

4

Hi @Unni ,

Following up here as it’s been a while since last communication took place on this thread; did you have any additional questions regarding your initial post? Or if you found a solution then please feel free to share!

If we do not hear back within the next 3 business days, I’ll close out this topic.

Thank you

5

Hi @Unni,

Since we haven’t received any further updates from you, I’ll treat this inquiry as complete for now. However, if you have any additional questions, feel free to create a new post in the community and link this discussion for context.

Thanks!