SPICE_PARENT_DATASET_NOT_READY / Data not available from the dataset used as a data source

Question & Answer
1

We are using S3 + Athena as a datasource, and I am creating a dataset programmatically. The dataset creation is successful, however when trying to use the dataset I get the following error:

region: eu-west-1
timestamp: 1681819959152
requestId: f2b43625-1118-4ecb-8f7a-3516d8554ff3
sourceErrorCode: SPICE_PARENT_DATASET_NOT_READY
sourceErrorMessage: Encountered exception from SPICEClient: table id 7961b689-1538-405a-99a1-b346103a11dd does not exist for customer 33739670****

I have deleted and re-created the dataset a number of times to no avail. I am able to select out the data using the Athena Query Editor.

This is not my first rodeo, we have been using QuickSight for nearly a year now with hundreds of datasets being created in the same manner. Note that we have plenty of SPICE capacity available according to the config page.

A little bit of info about the dataset:

    "LogicalTableMap": {
      "2397667b-b666-44c5-a21a-c289392fa272": {
        "Alias": "client074_fx_algo_post_trade_forecast_v1",
        "DataTransforms": [
          {
            "FilterOperation": {
              "ConditionExpression": "{CreatedFor}='client074'"
            }
          }
        ],
        "Source": {
          "PhysicalTableId": "2397667b-b666-44c5-a21a-c289392fa272"
        }
      }
    },
2

Hi @mkst - Is it possibly to create the data set from UI and see whether you are able to create it or not, this is just to segregate the issue that no issues with SPICE or your data source. If you are able to create the data set from UI, then you need verify your physical and logical map and see whether mapping is correct or not.

Regards - San

3

Hi San,

I /am/ able to create the dataset manually in the UI based on the Athena datasource (and preview the data in the editor).

We use the same function to create our datasets - I successfully created 4 datasets yesterday (same schema, different underlying data) - everything was fine yesterday, and the other 3 are still fine today, only 1 has this issue.

I’m not sure what you mean by verifying the physical and logical maps to check for correctness. The Input/Output columns match when calling the DescribeDataSet boto3 endpoint.

[EDIT]: Whilst I can create the dataset and use it when set to DIRECT access, I am unable to import the dataset into SPICE: “There are too many invalid rows. Amazon QuickSight has reached the limit of rows that it can skip and still continue ingesting. Check your data and try again.”

[EDIT2]: OK looks like QuickSight really doesnt likely float infinity - so when I checked yesterday the data must still have been waiting to load into SPICE.

4

Hi @mkst - Thanks for the details. Looking at your details, you have lots of invalid records in the data due to which you are not able to do the SPICE refresh. When you are refreshing the data via UI, I believe you can download the error data set and validate what happened. This will give a clue whether your data is good or not.

In parallel, you can raise a ticket to AWS customer support team to help you in this regards as well. If you do not have access to raise for the same, request your AWS admin to help you in this process.

Tagging @Bhasi_Mehta and @Karthik_Tharmarajan for their feedback.

Regards - San

5

Yup - exactly this - I was able to download the report when importing manually, and found one of our columns has a large number of Infinity values – the confusion was that the dataset appeared to work yesterday (but I imagine it was doing a DIRECT query rather than SPICE - and then once the dataset was attempted to be loaded into SPICE, this failed, and therefore the dataset was empty).

Thanks for pointing me in the right direction!

1 Like
6

Thanks @mkst . Glad I am able to help you. Have a wonderful day.

Regards - San

1 Like