Hello there!
Has anyone had an issue with data not syncing properly from an S3 bucket? I can pull the file from S3 as a CSV and there are over 400k lines of data but when it gets pulled into QuickSight with an S3 connection I am only seeing 290k lines of data.
Please refer to this article for data source quotas, Data source quotas - Amazon QuickSight
Thank you for your reply! I checked the file size and its 175MB and with only 400k lines of data these are both way below the data limit size.
@BusinessAnalyst - Thank you for posting your query! Did you verify the SPICE Import statistics whether any rows got skipped during the load. If that’s the case, then it should allow you to download the error records to review the issue and fix the same at the source file end. Please let me know if that’s the case. Thank you!
@sagmukhe Thank you for your suggestion! After the refresh this morning I did pull the error file and it skipped 17 rows and only pulled in 264,889. I double checked the csv file in S3 and it has 414,937 rows. I’m just not sure why the other 150,031 rows are not making it into QuickSight.
@BusinessAnalyst - Thanks for your response. Did you double check the manifest file that you are using to fetch data from S3 to ensure everything is looking good over there? Perhaps you can try to create a new manifest file and try again just to see if something is not getting messed up over there.
Hi @sagmukhe I checked the manifest file and I’m not seeing any issues with it.
@BusinessAnalyst - Thanks for the response. In that case, I don’t have any more thoughts unfortunately. Tagging some of the other experts so that they can chime in with their thoughts.
@Max @Sanjeeb2022 @gillepa @Koushik_Muthanna @Kellie_Burton @eperts
@BusinessAnalyst I will state the obvious, please double check paths in the manifest file to make sure you’re loading the correct file. The difference between loaded and expected rows is significant. Since there are not that many rows reported as skipped I’m thinking it’s a misconfiguration issue.
I think at this point the best option is to create a AWS Support ticket so that the service team can take a look.
Here are the steps to open a support case. If your company has someone who manages your AWS account, you might not have direct access to AWS Support and will need to raise an internal ticket to your IT team or whomever manages your AWS account. They should be able to open an AWS Support case on your behalf. Hope this helps!”. Please include sample data file without sensitive data and manifest file in the ticket for replicating the issue.
