SPICE dataset refresh error - “The lookback window column is not valid in your dataset.”

Question & Answer
,
1

Hi Team,
I am attempting to run an incremental refresh of a dataset and I’am receiving the error below:

The incremental refresh failed to run due to a problem with the configuration: [The lookback window column is not valid in your dataset.] Please correct this and re-run a full refresh.

This dataset refresh has been running without issue for months and this error is new from last week. The lookback window column is in date format.

I have checked the data preview in the data prep. the column is of date type. but for some reason it is not refreshing. I have checked the publishing history, I have 4 version, all previews are same.

image
image402×859 35.5 KB

I have multiple datasets all are using the same column dt but only this dataset is failing from last week.

image
image199×353 4.35 KB

Any fix for this issue?

2

Hi @Nazhur

Were there any recent changes made to the dataset that might be causing the incremental refresh to fail?

Try these steps to resolve the issue:

  • Validate that the lookback window column’s data type is consistently a proper date/timestamp type.
  • Perform a full refresh before trying the incremental refresh again.
  • Verify that the dataset schema and columns have not changed recently.
  • Check for any recent updates or changes in your data source.
  • In some cases, recreating the incremental refresh configuration from scratch can resolve the issue.
3

Hi @Xclipse,
Thanks for the response.

  • Validate that the lookback window column’s data type is consistently a proper date/timestamp type. – I have cross-checked the type it is date from the start.
  • Perform a full refresh before trying the incremental refresh again. – It says query timed out.
    image
    image782×418 38.4 KB
  • Verify that the dataset schema and columns have not changed recently. – No
  • Check for any recent updates or changes in your data source. – No
4

Hi @Nazhur

Can you please share which database you are using to import data into QuickSight? Also, please verify the SQL query you are trying to refresh. Take the SQL and run it directly in the database to ensure there are no syntax issues. Then, try refreshing it from QuickSight.

Is it a customized dataset or a simple dataset that you are trying to refresh and store in SPICE? Ensure your database server has enough CPU and memory available. Make sure there have been no DDL changes (such as adding or dropping columns, or altering data types) that could impact the query. Also, ensure that the connectivity between QuickSight and your data source is stable.

As suggested earlier, try running a full dataset refresh before switching back to incremental mode. Alternatively, create a new dataset and perform a full load refresh, if it is successful, it indicates that schema changes have occurred at the source, which could be the reason for the failure.

Please refer to the below documentations this might be helpful for you.

5

Hi @Nazhur

It’s been a while since we last heard from you. If you have any further questions, please let us know how we can assist you.

If we don’t hear back within the next 3 business days, we’ll proceed with close/archive this topic.

Thank you!