How can I create Athena tables or views that dynamically/automatically grabs new S3 folders/partitions, like dates where data is partitioned by day and hour. As new data hits S3, to have it in a way that QuickSight can refresh and update visuals with new data.
What options are available to achieve this?
@GeorgeS - I believe this query has nothing to do with QuickSight, rather a generic Athena related question. In that case, I believe your data pipeline should have a piece of code to run glue crawler to add/update the Athena schema to ensure that new partitions are automatically reflected in Athena. You can perhaps go through the below link to get a better understanding.
Add Newly Created Partitions Programmatically into AWS Athena schema