Can QuickSight Athena connector produce data in another format?

Question & Answer
, ,
1

QuickSight calls Athena, and Athena produces CSV files as a result. However, this file format is not the most efficient, and our bucket is in the terabytes, even with the lifecycle rules to clean up the transient files.

Is there any way QuickSight could instruct Athena to use parquet format?

2

@m0ltar ,

Can you provide a few details as to what you mean with this file format is not the most efficient ?
What is query mechanism ? SPICE or DQ ? Is Amazon Athena mainly used for data ingestion into SPICE ?

Kind regards,
Koushik

3

Hi @m0ltar

QuickSight does not directly control the format of query results returned by Athena. However, you can optimize storage and query efficiency by configuring Athena to store data in Parquet format instead of CSV.

Example: (Syntax may vary)

CREATE TABLE your_parquet_table
WITH (
   format = 'PARQUET',
   external_location = 's3://your-bucket/your-parquet-folder/'
) AS
SELECT * FROM your_csv_table;
  • Use CTAS (Create Table As Select) to write query results in Parquet.
  • Point QuickSight to the new Parquet-based table.

Please refer to the below documentation this might be helpful for you.

4

Hi @m0ltar

It’s been a while since we last heard from you. If you have any further questions, please let us know how we can assist you.

If we don’t hear back within the next 3 business days, we’ll proceed with close/archive this topic.

Thank you!

5

Hi @m0ltar

Since we have not heard back from you, I’ll go ahead and close/archive this topic. However, if you have any additional questions, feel free to create a new topic in the community and link this discussion for relevant information.

Thank you!