How/Why Quicksight excludes records behind the scene?

Question & Answer
,
1

Quicksight is able to load the full dataset from Athena (number of records matches between Athena and dataset Quicksight). However, when all the records were summed up for one column, the total number in Quicksight is lower than the total number in Athena. No filter has been applied in the Quicksight dataset or in the visual. In this case, it seems that for some reason, Quicksight has excluded some records automatically. Anyone can shed a light on this please?

2

Hi Wendy , its seem abrupt as there are difference between the number of records.
I would suggest that once check the record at both end using some specific timestamp filters at both side. Maybe some data set loading time difference would creating such difference

3

Hi Adi,
I appreciate your reply. Since no filters have been applied when I double checked the total sum of a column in Athena and in the visuals, the total sum should have matched.
However, without any filters and with same number of records in both side, the total sum are different. And when I checked the monthly sum in the visuals, some monthly sum are off from the monthly sum in Athena. Any other suggestions please?

Thanks,
Wendy

1 Like
4

Can you click on the dataset in the “Datasets” view and check if any rows were skipped during ingestion?
dataset

If some rows were skipped, you can see more details by clicking on the “View error summary” link

1 Like
5

Hi Tatyana,

No records have been skipped. The number of records in Athena are the same as the number of records in the dataset for the visual. Thanks.

Best,
Wendy

6

Hi,

how do you count the number of records in Quicksight? Do you just sum up a a metric column? Can you count the rows?

Maybe some values in your dataset have a different datatypes. I had a problem in the past, that everything worked, but some fields were empty, because I had integer and doubles in the same column.

7

Hi, @wendy_chen. Did @thomask’s solution work for you? I am marking his reply as, “Solution,” but let us know if this is not resolved. Thanks for posting your questions on the QuickSight Community Q&A Forum!

8

I’m also experiencing the same issue which did not exist before. I have not changed my visualization and now out of 5 invoice_type, the simple visualization table in QS simply not showing 2 out of the 5 invoice_type I had in there before. All 5 invoice_types and all rows are clearly visible on the auto-preview on the datasets page. The total number of rows ingested into Spice is correct with no skipped rows. Anyone else have experienced this issue?