Manifest file to fetch the latest file from S3 sub folders

Question & Answer
, ,
1

I want to fetch teh latest file on yearly basis from S3 bucket .
Eg :Need to fetch the daily latest file .
Bucket path structure : S3/{bucket name}/{folder 1}/{folder 2}/{Year}/{month}/{dates }

Currently i am using below manifest file .
{
“fileLocations”: [
{
“URIPrefixes”: [
“s3://approval-reports-us-west-2-188894616023/S3/2023/”
]
}
],
“globalUploadSettings”: {
“textqualifier”: “"”,
“format”: “CSV”,
“delimiter”: “,”

}
}

but it is giving the data which is sum of all the files of 2023

Is there any way to fetch the latest file from sub folders of S3 bucket?

2

Hi @shikhakh
are you able to put just the latest file in a folder?
BR

3

Folder structure is : s3://approval-reports-us-west-2-188894616023/S3/2023/11/7/
And we want to show everyday data on the quicksight dashboard .
Thats the reason i selected the file path till /2023/ in manifest
{
“fileLocations”: [
{
“URIPrefixes”: [
“s3://approval-reports-us-west-2-188894616023/S3/2023/”
]
}
],
“globalUploadSettings”: {
“textqualifier”: “"”,
“format”: “CSV”,
“delimiter”: “,”

}
}

But this is giving latest file .
need some attribute by which we can fetch the latest file form sub folders .

4

Hi @shikhakh
got that. Thats why i ask if it would be possible to move the last file in a different folder.
BR