Dataset editor does not show all records when filter is used

Question & Answer
,
1

It appears that when a text filter is applied to a dataset it only shows a partial result. I have tested this on more than one dataset and it appears consistent that not all records is shown, but there does not seem to be consistency of how many of the records is displayed. In one example the dataset returns all 88 valid records to an analysis but only shows 14 records in the dataset preview in the dataset editor. Is there a rule on how the preview results are calculated or is there some other way to ensure that all the records are displayed?

What is the best way to test that there is not an error in the dataset?

Below are the steps that I followed on this specific dataset, but this may be happening on other datasets as well.

Steps:

  1. Create a new dataset based on an existing one.
  2. Once the dataset editor opens, filter the dataset on a field that contains FAMILY, to get all. This does not show all records.
  3. Now save this new Dataset as a new dataset
  4. On an analysis, display all the records of this dataset in a table. Notice that more than 14 records now display in the table, currently 88 records
  5. To confirm the actual number of records that should be in the dataset, run a query on the db. Notice that 88 records is expected.
1 Like
2

Hello @pieter.jacobs, welcome to the QuickSight community!

I believe there is like a 30 second timeout period for the dataset preview that would likely cause the number of rows to cut off at that moment. So depending on the complexity of the query, or the database you are querying data from, it is likely only going to pull a certain number of rows before timing out.

If the query takes longer than the preview limitation, then you won’t know for sure how many records to expect until you refresh the dataset. Let me know if this helps!

1 Like
3

Thanks for the reply. I think that is why we are not seeing all the records.

1 Like
4

Hello @pieter.jacobs, while there is not a solution to this issue at the moment, I can at least tag this topic as a feature request to increase the query times for the dataset preview page.

The only work-around I can really think of at the moment would be to run the SQL queries in a database IDE, like DataGrip, where you can test the SQL before creating the dataset in QuickSight. This is what I have done in the past, especially when I hit the limitation where it won’t preview any of the data because the query is too complex.

Let me know if you have any further questions, otherwise I can archive this question for the Support Team. Thank you!

2 Likes