Pull in S3 file name as metadata

Question & Answer
,
1

I have my files on S3, and the file name of the data has the date it was captured. It’s setup to use the S3 dataset connector. The files are in a folder, with some sample file names:

20220816T0600.json
20220816T0700.json

Can I somehow use the capture date (which is also the object name on S3) when I display it in Quicksight? I tried searching around but could not locate a way to do this. Ideas / solutions would be much appreciated.

-Todd

2

In case anyone is hitting the same issue, I ended up solving it by adding a captureTime field to my source data. This calculated the UTC time, then used the jq utility to add it to each json record.

CUR_UTC=$(date --utc +%FT%TZ)
echo “$(jq ‘.[] += {“captureTimeUTC”: "’”$CUR_UTC"‘"}’ data/$THE_FILE)" > data/$THE_FILE

1 Like