How/Why Quicksight excludes records behind the scene?

Quicksight is able to load the full dataset from Athena (number of records matches between Athena and dataset Quicksight). However, when all the records were summed up for one column, the total number in Quicksight is lower than the total number in Athena. No filter has been applied in the Quicksight dataset or in the visual. In this case, it seems that for some reason, Quicksight has excluded some records automatically. Anyone can shed a light on this please?

Hi Wendy , its seem abrupt as there are difference between the number of records.
I would suggest that once check the record at both end using some specific timestamp filters at both side. Maybe some data set loading time difference would creating such difference

Hi Adi,
I appreciate your reply. Since no filters have been applied when I double checked the total sum of a column in Athena and in the visuals, the total sum should have matched.
However, without any filters and with same number of records in both side, the total sum are different. And when I checked the monthly sum in the visuals, some monthly sum are off from the monthly sum in Athena. Any other suggestions please?

Thanks,
Wendy

1 Like

Can you click on the dataset in the “Datasets” view and check if any rows were skipped during ingestion?
dataset

If some rows were skipped, you can see more details by clicking on the “View error summary” link

1 Like

Hi Tatyana,

No records have been skipped. The number of records in Athena are the same as the number of records in the dataset for the visual. Thanks.

Best,
Wendy

Hi,

how do you count the number of records in Quicksight? Do you just sum up a a metric column? Can you count the rows?

Maybe some values in your dataset have a different datatypes. I had a problem in the past, that everything worked, but some fields were empty, because I had integer and doubles in the same column.