Combine SPICE datasets

[Reena]

Hi,

I am trying to do a full outer join so I can compare the metrics in two SPICE datasets. However, I receive the attached error for DATA_SOURCE_CAPACITY_EXCEPTION. Both my datasets have these capacity values -
Dataset 1 - 5.9 GB (7636285 rows)
Dataset 2 - 2.9 GB (10591510 rows)

Could you please provide suggestions on how I can combine these datasets.
Thanks!

[Sanjeeb2022]

Hi @Reena - Looks like there is a M:M relationship is happening between the 2 data sets and this leads number of rows exceeded SPICE capacity. Please review your data. If you the relation is fine, please check the SPICE capacity for your account as well.

Tagging @duncan @DylanM for their expert advise.

Regards - Sanjeeb

[Reena]

Hi @Sanjeeb2022 ,
Thanks for your response. I have reviewed my query and SPICE capacity as well (screenshot below for your reference). Since there is a huge amount of unused capacity left, the datasets should not exceed the usage limit.
Please let me know if you have any other suggestions to help with this request.

Thanks!

[David_Wong]

Hi @Reena,

You’re hitting the same limitation that was discussed in these 2 topics:

Join tables releases error, due to exceed join memory, need to increase size of join

Hello All, could some one help on this please, this is very weird, how come join limit only 1GB? now we can’t reduce of join, or else if we remove it will breaks whole dashboard, how to increase join size to 10 GB, please error: QuickSight ran out of join memory while processing your request. To join between data sources, the smaller source tables combined can’t exceed the join memory limit of (1GB). The largest one, “accounts_users”, doesn’t count toward the memory limit. To…

Joining dataset is not allowed due to capacity[High Priority]

Both of the dataset are coming from SQL queries, the issue is i dont want to make any changes on the query itself because it will affect other dashboards. I want to create a new dataset which contains both both dataset and use them to create a combined view.

To confirm it, add a filter to your 2nd dataset to reduce it to less than 1 GB and then try your join again.

[Sanjeeb2022]

Hi @David_Wong - Is it possible to have a M:M relation in the data sets which in turn exceeded the limit of Spice space?

Hi @Reena - You can have a look on David’s suggestion or else raise a ticket to AWS Customer support ( Link - Creating support cases and case management - AWS Support) and provide the ingestion id for more detail analysis.

Regards - Sanjeeb

[DeepakS]

Hi @Reena . Checking in. We have not heard back from you regarding your question. We’d still like to help. If we do not hear back in the next 7 days, we will archive the question

[DylanM]

Hello @Reena, I am going to archive this question since follow-up information was not provided. If you still need assistance with this problem or a new issue you are facing in QuickSight, feel free to post a new question in the community and you will be at the top of the priority list for a response from one of our QuickSight experts. Thank you!