I need delete de duplicate value when i ncrementally refreshing a dataset and i have duplicate.
I try create a calculated field with this code so that when it detects the same pk, it gives me the duplicate value if so, and then add a filter of only those that are not duplicates. all this at the dataset level.
Pk is the primary key. And date is the date of update, the idea is that when I have the same primary key, I select the most recent date and the other put duplicate and thus put a filter and remove duplicates.
Yes, but this solution is not valid. Since I do incremental loading of data is spice, and my idea is to remove when there is updated and repeated data, hence the calculated field. What other option can you think of?
This filter is going to happen in QuickSight when the visuals load. If you need to remove those duplicate rows before importing into SPICE then you will have to do it in the DB or at least with Custom SQL. SPICE doesnt have any way to delete certain rows from the data, can just filter them out when running queries.