Join tables releases error, due to exceed join memory, need to increase size of join

Question and answer
,
#1

Hello All,

could some one help on this please, this is very weird, how come join limit only 1GB? now we can’t reduce of join, or else if we remove it will breaks whole dashboard, how to increase join size to 10 GB, please

error:
QuickSight ran out of join memory while processing your request. To join between data sources, the smaller source tables combined can’t exceed the join memory limit of (1GB). The largest one, “accounts_users”, doesn’t count toward the memory limit. To alleviate this problem, you can start by reducing the size of the initial source tables in the join

#2

This is a SPICE limitation. You can choose direct query and make this join

#3

Thanks Lillie for reply.

I have tried, even direct query will not work, since we have 54 fields on one table, we do required all the fields so i can’t trim the fields. can’t be possible to increase the join limitation to at least 5GB?

#4

What error do you get with direct query?

#5

Screenshot (420)
Screenshot (420)1237×394 27.9 KB

can’t appear direct query, please check screenshot, because it’s (use in a new dataset) from main dataset. BTW, direct queries are very slow for large data,so i usually import into SPICE.

#6

I misread the documentation here: Joining data - Amazon QuickSight. There is just a 1GB data prep limit. I’ll mark this as a feature request. Thank you

#7

Hope increase of join limit release won’t take much time.
thanks for all your support

#8

Hi Lillie,
Any update on increase of Join limit? Please
thank you.

#9

Hi Ashok. No update at this time, but we have marked this as a feature request so that appropriate team can take it under consideration

#10

I have created case/ticket as well from AWS management console, meantime we are looking for other BI tools since we don’t have option.

Thanks Lillie.

#11

Hi,

In your scenario, you can test the following implementation and should not face join limitations :

Note: there will be costs associated if you are testing this solution:

  1. Store files in S3.
  2. Use an interactive query service like Amazon Athena.
  3. ( Workshop Studio )
  4. Within Athena you can create a table that points to location of S3 bucket (S3 , Athena and QuickSight have to be in the same region ).
  5. Amazon QuickSight can connect to Athena as a data source. You can use either use Custom SQL or joins on your different tables and then import the data into SPICE.

Regards,
Koushik

1 Like