I have already created a ETL and using unload command to move the output of ETL(using SQL query) to S3. From S3 dataset has been created and dashboard has been published. Now If i need to add few more columns in ETL(using SQL query) will it give any error when I refresh the dataset?
Hi @Minakshi, thanks for posting your question.
Just adding columns to S3 dataset wouldn’t cause any problem. However, if you use “URIPrefixes” parameter in the S3 manifest to implicitly include all the files under a given S3 prefix, remember that all files must have the same number and type of columns (refer to user guide).
Please also note that, in order for the added columns to be included in the existing dataset, you need to open that S3 dataset in dataset editor, confirm the new columns are there, and then hit “save & publish”. This will trigger the dataset refresh process. Once completed, you should be able to use those columns in your analysis.
I would suggest you test this in your lower environment first and familiarize yourself with it.