Appending dataset using csv files - data source is multiple csv files with same column names

How do I append to the 1st dataset that was created using csv files ? with a 2nd csv file.

You can do it before data load in dataset.
I did not see any option of append multi files in dataset.
There are some other tool like S3 or Athena you can do it.

Naveed Ali

do you mean that the csv files will be updated in a regular time(e.g. daily)?
for instance, day1.csv has Jan1 data. day2.csv has Jan2 data. Then you want to show all data in QuickSight

If it is the case, you may refer to this approach shared by @darcoli using Athena as data source

Thanks Naveed, can u give me some leads / to try other options

Best regards


You can use store your files on S3 using pipelines or Lambda function.
Use Athena to create one table and use this tables in QuickSight.

Naveed Ali

1 Like


This may help you.

1 Like

Thank you. You have nicely described it. I am not familiar using S3 buckets or Athena hence will need some direction with both S3 and Athena

I will like to give that a try

Best regards

I created S3 bucket and loaded files. Used a note pad to create manifest file and added the reference to the S3 bucket. When to quicksight s3 bucket and ran/ uploaded the manifest file.

This works fine. The only issue I have with larger files is the S3 bucket loading did not complete for file size of 32 GB. The loading process times out after 22 percent loading.

Another way to do this is to simply point the quick site to redshift table.


can check if the files have exceeded the limit? Data source quotas - Amazon QuickSight

As the file size are quite big(32GB), suggest to use incremental load if it is possible to reduce data loading time. If the data size is huge or there is already a Redshift in place, suggest to use it.

Hi @rajabhau. Did Roy’s feedback help? Hope so. Let us know if this is now solved. (Or, please mark the Solution.) Thanks! :slight_smile: