hello!
I’m trying to configure the dashboard with SPICE. The dataset is imported from Athena, and once loaded, it works normally, but when loaded into SPICE, only 0 rows are imported. need permission to use SPICE or what settings are required?
Hello @orangchan,
That message means that the dataset is being loaded into SPICE you should see, next to the dataset name 2 arrows going arround shoing it is being refreshed, as you can see here.
When the dataset finishes importing, the dataset should show 100%, and if you click on the 100% it should give you the status of the last refresh as you can see here.
I don’t think this is related to permissions just yet, since I am not sure if the dataset has finished loaded into SPICE.
Can you please confirm what is the status next to the Dataset name? You can also go to dataset on the refresh tab to check the status of the refres.
Hope this helps!
Kind regards,
Andres.
@andres007 thakns for reply.
import is end. but An error has occurred. I didn’t write the SQL syntax when reading the data, could that be the problem?
@orangchan It’s a timeout error.
so most likely an unoptimized query, or too much data getting ported from Athena to QS, or something along those lines… I would start there. Also check how long it took before timing out.
Are you using Athena on top of s3 or something else?
1 Like
Hi @orangchan - Please give more details on the athena query? Since QS is time out, there is an issue in the query performance or the way the table is built in Athena. I agree with @neelay point.
Regards - Sanjeeb
@neelay @Sanjeeb2022 thx reply.
query syntax was not written when import QS.
data is searched in athena by combining parquet files existing in S3 with Glue.
there are approximately 3.7 billion rows of data could this be the reason?
Hello,
There are 2 things here, the query timeout and the amount of data. Importing into SPICE has some limitations, depending on your edition you might be hitting one of these limitations
- For Standard edition, 25 million (25,000,000) rows or 25 GB for each dataset
- For Enterprise edition, 1 billion (1,000,000,000) rows or 1 TB for each dataset
Here you can find some documentation about all the SPICE limits.
The actual timeout you are experiencing is an Athena query timeout, which is 30 minues
The next steps to work in this problem will be to find out if the whole dataset is really nedded or if it is going to be filtered? Can the data be used in an aggregated fomat? Maybe running a Custom SQL on the data source to do some pre filtering or aggregations will be the way to go here, but without knowing the data and the usage of the dashboards you are planning to use I can only give you some general advise.
I hope this helps with understanding the reasons for this specific error and finding a solutions for the underlying problem!
Kind regards,
Andres.
I understand what you said. I will try to find a solution. Thank @andres007