Unable to refresh Datasets

Hi,

We have a few Quicksight dashboard and have been deploying changes and refreshing them regulary for the past 5 months. However in the past 2 weeks we have gotten this error when trying to deploy changes through both cloudformation and boto3. This error occurs when refreshing the data.

Requested tableID: {TABLE_ID} is in state PROVISIONED

Dashboards and datatsets that used to build now no longer do. We have tried multiple dashboard and its the same error on both our dev and prod accounts.

Our datasets use a mix of athena and postgres datasources.

Has anyone experinced this? the support team could not diagnose the issue and have passed it to the Quicksight team but it has been 2 weeks.

Note: datasets/dashboards that are already deployed refresh on schedule and continue to work daily

1 Like

Hello @John-Paul_Kennedy, welcome to the QuickSight community!

I have a few questions. When you go into the QuickSight console directly, and check one of the datasets that are throwing this error, how many rows are being ingested? Also, if you run the refresh manually from the console, is it giving you a similar error? With a little more information, we should be able to figure out why this error is occurring.

Hello @John-Paul_Kennedy, are you able to follow-up with answers to any of the questions in my previous response? That will help me guide you towards your expected solution. If we do not hear back from you in 3 days, I will archive this topic. Thank you!

We’ve been having this issue too, we have datasets created through cloudformation that were not being properly created because of the whole…
Requested tableID: {TABLE_ID} is in state PROVISIONED
… thing.
This only started happening recently, and it seems to happen exclusively to datasets that are the results of data joined from other datasets.

So let’s say we have Dataset A, which is the result of Query A to Athena, and then we create Dataset B, which is the result of Query B to Postgres joined to Dataset A. The error was being thrown during creating Dataset B.

We ended up resolving this issue by adding a DependsOn condition to the dataset object in the CloudFormation template. Previously, we did not need to do this, as simply referencing the arn of another dataset made it work just fine, but now the DependsOn condition is required, and fixes this issue by waiting for the Dataset A to finish fully before attempting to create Dataset B.

Something must have changed somewhat recently to necessitate that change, but I wouldn’t have an idea what that is.

2 Likes

Hey, sorry for the late reply.
@DylanM it was about 200m rows, it threw the same error from the console manually refreshing too.

On 08-03-2024 we tried it via boto3 and it failed then we tried it again on 11-03-2024 with no code changes and it randomly now works fine. I have a feeling what @Lawrence_Kimsey suggested will help prevent the issue.

Also the support teams response to our issue backs up this theory: ‘I have received an update from the Quick sight service team that the SPICE parent dataset is not ready to be used (ingestion still in progress) and thus the child dataset ingestion failed. Internal team has requested you to wait until parent dataset’s ingestion completes before creating a child dataset.’

1 Like

Hello @John-Paul_Kennedy, and thank you @Lawrence_Kimsey for responding with a work-around you found when running into a similar issue. I will mark the above response by Lawrence as the solution, but I appreciate all of the input. It definitely sounds like if the initial dataset takes to long to refresh, it causes the dataset creation after the join to fail. The DependsOn condition seems like the best way to manage this. I’m not sure if any updates were made to cause this issue, but I can also archive this topic and tag it appropriately for our support team. Thank you both!