Does anyone know whether there are any limitations on PRE_FILTER calculated fields at the dataset level?
We have a calculated field to rank the versions of items by most recent to least recent: rnk = rank([{datetime_processed} DESC], [{id}], PRE_FILTER)
Then we add a couple of filters at the dataset level to filter out items that have been archived.
Then at the analysis level, we filter on rnk = 1, so only the most recent version of each active item gets displayed.
This has worked as expected for us in multiple datasets with the same relevant fields and filters. Now we have a new dataset, and the rnk calculation doesn’t seem to be level-aware on this new dataset. I’m seeing archived items (or more specifically, the most recent active version of an archived item) in the analysis for the new dataset. The only difference I see is that this new dataset is somewhat larger, by about 2-3X. The size of the new dataset is 241k rows.
Since you say it has worked on other datasets before and your hunch is that it may be the size of the dataset. Why don’t you try filtering the dataset to a small period say a couple of months and see if the issue goes off.
If Yes, then you may be right about the size.
If Not, then there may be some nuances in your dataset which are different from others that may be causing the problem.