Combien multiple files vertically as partitions of a same data set

Question & Answer
1

How do you combien multiple file uploads representing different partitons of the same data structure ?

2

I believe the way to do it directly in QuickSight is not very straight forward. You can use “Add data” to add all files as different tables and then join them all using a Full Outer Join on a condition that is always false. You would then create calculated fields for all fields you want to have available as follows (example for a field a:

  coalesce(table1.a, table2.a, table3.a...)

This is pretty cumbersome and more of a hack rather than a proper solution - the major limitation here is that QuickSight does not support a UNION ALL operation natively so we need to emulate it.
A better solution is probably to simply upload these files to s3 within a particular directory and set up an AWS Athena table to query these files. (see Creating Tables in Athena - Amazon Athena ). You would then set up the DataSet from QuickSight via the Athena DataSource.

1 Like
3

I was hoping for a proper solution but thank you for laying out the options currently at hand.
Cheers

4

I’ll mark this as a feature request for the team. Thank you

1 Like