'More than one column has the same display name.' - using s3 bucket as source

Question & Answer
, , , ,
1

Hi everyone,
I create a custom Redshift table and unload the data into S3 which i then connect to a QS dataset using a Manifest file. The custom table evolves over time and sometimes after I have made some changes, I run into this error ‘More than one column has the same display name.’ but there is no actual column duplicacy. In the past I have mitigated it by replacing the final select * statement with select column1, column2.. etc but now even that is not working.
Does anyone have any idea how to fix/mitigate this issue?

All the updates I have made are within the underlying logic, I did not make any changes from a final column populated in the table perspective. That is why I am even more confused why this issue keeps popping up. At most, i would’ve changed the data type of the final columns.

2

I was able to resolve this issue. Redshift was trying to assume a data type (underlying data was float8 type) and was getting confused due to the presence of nulls. So technically there was no duplicate column, it was assuming the same column as 2 columns due to data type assumptions.
I explicitly cast it as text and then in QS dataset layer changed it back to float8, this fixed the error.

1 Like