Added columns not showing in the joined dataset

Hello, I have an existing dataset joining two Athena tables. both are reading from S3 buckets.

  1. I added additional columns in one of the tables

  2. I can see these additional columns/fields in QS when I add this changed table as a new data source, so Athena side of change is done correctly.

  3. however, I couldn’t see these additional columns/fields in the existing joined dataset after many days and multiple full refresh.

There are quite a few analysis using this dataset, they are outside of my control, so I hope that I dont have to start fresh with a new dataset. Could the experts here lend a hand? thanks in advance.

Hi @Marco1 ,

Welcome to the community!

I have few questions regarding the joining of the tables.
Is the join query written in the custom SQL section for the QS dataset? If so are you using select * in the query and able to see the columns in preview ?

Thanks,
Prantika

1 Like

Thanks Prantika for the reply. it is just a simple left join on a single field. also for future reference, could you please elaborate if anything special needs to be done if it is custom query with select * in the query?

Hi @Marco1,

My understanding was if the column is viewed in dataset preview and not restricted, then it should appear. So if you are already using select * in the query then all the columns are expected to show up. However, if the fields are hard coded in either of the select query, then you may need to manually add the fields up in the select statement.

Hope I am able to explain you the approach I was taking to debug your issue.

Thanks,
Prantika

1 Like