Append new .CSV file data?

How do I add additional data to a dataset when using .csv files? I receive a new .csv file periodically and wish to append the data to the existing dataset/spice data. This is not an update to the existing records but addition of new records.

For this type of requirement, I would suggest to make use of S3 buckets and then ingest the data into Quiksight either directly consuming S3 buckets or through Redshift (S3 to Redshift using copy command script).

2 Likes

Hi Rupesh,
Thanks for responding. That is on the roadmap. For now I’m just trying to get the team operational. But they have about 12 months of historical data they wanted to include. For now, we’re going manually combine the data into a single .csv and then replace the existing dataset. But, thanks for confirming the future state approach.

In the meantime, another simple approach might be to store these csv files in an s3 bucket and add an Athena table reading these files. Then, from QuickSight, use that athena table in your dataset rather than uploading the files there. (Assuming you don’t use Athena partitioning, then newly updated files’ data will immediately be visible to QuickSight - but will need a SPICE refresh if you are using that)

2 Likes