Please refer to the below link for supported file formats. The documentation also provides samples for the s3 manifest files. supported-manifest-file-format
Alternatively, you can also upload your .XLSX file directly using the “Upload a File” option while creating a new dataset.