DATA_TOLERANCE_EXCEPTION Error Cause

Q&A
, ,
1

Hi,
I’m getting DATA_TOLERANCE_EXCEPTION error while trying to ingest a dataset into Spice.

Error threshold exceeded for dataset (10000) (maxErrors = 10000)

10001 Skipped Rows

10001 rows where an undefined exception occurred.

However the error logs are not available. It writes: "This refresh didn’t generate an error report.

So, I created a test dataset where I replicate original dataset. The refresh part works fine until I create one spesific calculated field, and it is:

(Is low ASP)
ifelse({Cost from completed} < 20 or {Cost from target} < 20,‘Yes’,‘No’)

The sytanx is alright, but the refresh fails whenever I add this calculated field and refresh the dataset.

As I know, DATA_TOLERANCE_EXCEPTION error is related to data mismatch. However, {Cost from target} and {Cost from completed} are both decimals:

image

Cost from target: {ISM guidance (PCOGS)} / {ISM guidance (Units)}
Cost from completed: {cumulative_submitted_cost} / {cumulative_submitted_qty}

The fields in the formula (ISM guidance (PCOGS),…, cumulative_submitted_qty) are decimals. Defining these fields are working fine.

I edited the (Is low ASP) calculated field as:

ifelse(nullIf({Cost from completed}, 0) < 20 or nullIf({Cost from target},0) < 20,‘Yes’,‘No’) and it worked. But it’s not what I need, because this gives different results. (in the first scenario the results can be ‘no’ if the both of the fields are null, but this case makes them ‘yes’)

The refresh has been working fine for months until now, I can see the preview results however the refresh fails.

How can i resolve the issue?

Best,

2

Hello @cey , welcome to the Quicksight community!

Has anything about the underlying data set changed in the time since it worked and when your refresh failed?

One work around could be to add those calculations to your dataset in Quicksight before saving it and publishing to your analysis.

1 Like
3

Hi @cey - that is a strange one. Agree that error usually only occurs when we find datatype mismatches but you are showing both fields as decimals, so it looks fine.

@WLS-D 's workaround should work. One small correction, I think he means adding that last calculation to your analysis rather than in the dataset. I understand this may affect performance slightly since the last calc will be computed on the fly rather than materialized in the dataset, but if you dont have a massive amount of rows (tens/hundreds of millions) it will prob be unnoticeable.

2 Likes
4

Hi, here’s the update:

  • Tried to define the calculated field in the dataset query, rather than a calculated field, but the same error occured. So this didn’t work either. There wasn’t a change in the underlying dataset, but I’m not sure why I can’t define it in the query.

  • Solutions: Then, I deleted the calculated field from the dataset and added it in the Analysis part, and it worked. Also modified the other calculated fields that are based on this new “Is Low Asp” field if they were defined beforehand.

Since there’s no error logs avaliable, it’s not easy to detect the underlying problem. Thank you for your time @WLS-D and @Jesse.

2 Likes
5

Sorry we werent able to get to the bottom of it. I would suggest opening a support case and they can help investigate int he actual product logs if you want to find root cause, but Im glad you have an alternative solution in place.

1 Like