Error With Dataset Field When Accessed from S3 but not as upload

Question & Answer
, ,
1

I am getting errors refreshing a query of 5 CSV files that are on S3 which I am accessing with a manifest. I believe QS should be able to handle this data and this is why:

  1. These are number parse failure errors on a field that QS is initially recognizing fine as a decimal float in preview. When I export the errors, they show just fine as decimals in the export CSV
  2. When I upload the individual CSVs, QS is able to load the entire dataset with no issues. Obviously, I need to connect to the S3 folder so that I can merge all of these CSVs so this option isn’t a workaround, just a test.
2

Hi @cactus - Thanks for the question. Is the structure of the csv files are same? Also if you can give sample data and manifest file details, it will help in troubleshoot.

Regards - Sanjeeb

1 Like
3

The CSVs are the same. I just ran a test, each individual CSV uploads fine, and if I do an S3 manifest for a single URI, it imports fine. Only when I do the S3 manifest for multiple files (either listing out the S3 files individually or URI prefix for the folder) does the dataset fail to refresh and give the number parse error.

The CSVs are something like this:
Customer Name|Balance Owed

Where balance owed is registered as a float in SPICE. Again, the refresh failure/errors only appear when I connec to multiple files. Individual files, whether uploaded directly or connected with an S3 manifest targeted at the one file, import with 100% success.

The manifest is like this
{
“fileLocations”: [
{
“URIs”: [
“s3://files/Report 2023.06.01.CSV”,
“s3://files/Report 2023.06.02.CSV”
]
}

],
"globalUploadSettings": {
    "format": "CSV",
    "delimiter": ",",
    "textqualifier": "\"",
    "containsHeader": "true"
}

}

4

Hello @cactus, is this an issue you are still experiencing in QuickSight? I am not seeing any issues with the manifest file you posted above but I have some thoughts on what you could try to fix the error.

First thing, you could try excluding globalUploadSettings altogether since you are only utilizing the default values. Also, are all of the files you are wanting to upload contained within the same Bucket? If so you could use the URIPrefixes identifier rather than URIs to grab each of the files within that bucket. Otherwise, the only thing I can think of would be switching from the s3://examplebucket/data.csv format for your URIs to the https://s3.amazonaws.com/awsexamplebucket/data.csv format and see if that helps.

Let me know if any of those options fix the error you are experiencing. If not I would recommend filing a case with AWS Support where we can dive into the details so that we can help you further. Here are the steps to open a support case. If your company has someone who manages your AWS account, you might not have direct access to AWS Support and will need to raise an internal ticket to your IT team or whomever manages your AWS account. They should be able to open an AWS Support case on your behalf.

Let me know if you submit a ticket, and I can go ahead and archive this topic for our Support Team to review further. Thank you!

5

Hi @cactus - One of the easy solution may be, run the crawler on top of your data so that an athena table will be created and point QuickSight to the the athena table. However , please raise a ticket to AWS support ( as per the suggestion from @DylanM ) so that we can understand the root cause of this problem.

Regards - Sanjeeb

1 Like
6

Hello @cactus, I will go ahead and mark my response as the solution. If you continue to experience issues after trying out the suggestions made and reaching out to AWS Support, please create a new question in the community and link to this topic. That will ensure you are at the top of the priority list for a response from one of our QuickSight experts. Thank you!