Automated Sync of Local Files with S3 Bucket to visualize with Quick Sight

Q&A
,
1

Hey,

is it possible to automatically sync files (eg an Excel File) in a local directory with an S3-Bucket, which is used to visualize the data inside the excel file with Quicksight? So Dashboard Information should change if the information inside the local Excel file gets changed.

Thanks for any help, Fabian

2

Hello @fabian.altendorfer

Yeah you can achieve this with some data engineering set up.

You can drop files into S3 and have an automated pipeline update your dashboards. Set an S3 ObjectCreated event → EventBridge (or S3 → Lambda) to run a job that converts Excel to CSV/Parquet, writes to a partitioned S3 path (e.g., s3://bucket/dt=YYYY‑MM‑DD/), and then updates the Glue Data Catalog (via Glue Crawler or direct API). Athena queries those tables from the Glue Data Catalog. In Quick Sight, connect to Athena:

  • If you use Direct Query, visuals read the latest data on each query (no SPICE refresh needed).
  • If you use SPICE, schedule a refresh (e.g., daily/hourly) to match your file arrival pattern—or use incremental refresh on the partition column.

References -

Let me know if this helps.

Thank you,
Shravya

1 Like
3

Hi @fabian.altendorfer,

It’s been awhile since we last heard from you on this thread, did you have any additional questions regarding your initial post?

If we do not hear back within the next 3 business days, I’ll close out this topic.

Thank you!

4

Hey Brett,

thanks for the reminder. I will check it out!