Quick Sight Incremental Dataset Refresh duplicates rows instead of replacing old ones and appending new ones

Hello @vamseep
The issue you’re experiencing occurs because Amazon QuickSight’s incremental refresh doesn’t use a unique key (like ID) to identify and update existing rows.
When incremental refresh runs, it deletes and replaces data within the lookback window timeframe, but if rows were updated outside of the lookback window, they get appended as duplicates rather than replaced.

You can check the reference here:
[+] Duplicates rows are being created via Incremental loading on data set - #6 by Naveed

You can try changing the lookback window to a longer lookback window based on how far back your row updates can occur.

Or, you can set up calculated fields to Filter duplicates.