** Urgent ** DistinctCountOver() Function Issue

Hello,

I am building a dynamic QuickSight Visual where users can toggle between different views.

Here’s the setup:

  • ${ShelfChangesValueParam} is a parameter that lets users choose between:

    • Placements
    • Average $ Net Price
    • Average $ Promotion
    • Average % Discount
  • ${ShelfChangesParam} is a parameter that lets users toggle between:

    • Brand (dataset field {brand_name})
    • Retailer (dataset field {merchant_name})
    • Dynamic Price Band (calculated field inside the analysis)
  1. Formula :-
ifelse(
${ShelfChangesValueParam} = “Placements”,
ifelse(
${ShelfChangesParam} = ‘Brands’,
distinctCountOver(concat({part_number},{merchant_name}), [{brand_name}, Year], PRE_AGG),
${ShelfChangesParam} = ‘Retailers’,
distinctCountOver(concat({part_number},{merchant_name}), [{merchant_name}, Year], PRE_AGG),
distinctCountOver(concat({part_number},{merchant_name}), [{Dynamic Price Band}, Year], PRE_AGG)
),

${ShelfChangesValueParam} = "Average $ Net Price", 
ifelse(
    ${ShelfChangesParam} = 'Brands', 
        avgOver({net_price}, [{brand_name}, Year], PRE_AGG),
    ${ShelfChangesParam} = 'Retailers', 
        avgOver({net_price}, [{merchant_name}, Year], PRE_AGG),
    avgOver({net_price}, [{Dynamic Price Band}, Year], PRE_AGG)
),

${ShelfChangesValueParam} = "Average $ Promotion", 
ifelse(
    ${ShelfChangesParam} = 'Brands', 
        avgOver({shelf_price} - {net_price}, [{brand_name}, Year], PRE_AGG),
    ${ShelfChangesParam} = 'Retailers', 
        avgOver({shelf_price} - {net_price}, [{merchant_name}, Year], PRE_AGG),
    avgOver({shelf_price} - {net_price}, [{Dynamic Price Band}, Year], PRE_AGG)
),

ifelse(
    ${ShelfChangesParam} = 'Brands', 
        avgOver(({shelf_price} - {net_price}) / {shelf_price}, [{brand_name}, Year], PRE_AGG) * 100,
    ${ShelfChangesParam} = 'Retailers', 
        avgOver(({shelf_price} - {net_price}) / {shelf_price}, [{merchant_name}, Year], PRE_AGG) * 100,
    avgOver(({shelf_price} - {net_price}) / {shelf_price}, [{Dynamic Price Band}, Year], PRE_AGG) * 100
)
  1. Problem:

Everything works fine when using Brand or Retailer.
But when trying to use Dynamic Price Band inside distinctCountOver(), I get this error:

Field Value
region us-east-1
timestamp 1745835730301
requestId 630a2ef0-a4b1-4702-8068-b721c3e45418
sourceErrorCode DISTINCTCOUNTOVER_INVALID_ARGUMENT
sourceType SPICE

The issue only happens at the placement calculation — when involving {Dynamic Price Band} inside distinctCountOver().
Everything is working fine except the line - distinctCountOver(concat({part_number},{merchant_name}), [{Dynamic Price Band}, Year], PRE_AGG)

  1. Additional Info:
  • {brand_name} and {merchant_name} are dataset fields.
  • {Dynamic Price Band} is a calculated field inside the analysis (not a dataset field).
  • We cannot modify the underlying dataset to add any field.
  1. My Question:
  • Is there a way to allow distinctCountOver to work when grouping by a dynamic calculated field like {Dynamic Price Band}?
  • Or is there a recommended workaround to achieve this ?

We need to keep the calculations inside the Analysis layer only.

  1. More detailed Context of Our goal is :- Urgent_Dynamic Percentage Calculation based on 2 columns - #6 by lary_andr

  2. Formula of Dynamic Price Band -

ifelse(
    ${PriceBandSelection} = 'Ultra Low $10-$40',
    ifelse(
        {net_price} <= 9, '$9 & Below',
        ifelse(
            {net_price} <= 19, '$10 to $19',
            ifelse(
                {net_price} <= 29, '$20 to $29',
                ifelse(
                    {net_price} <= 39, '$30 to $39',
                    '$40 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Very Low $50-$200',
    ifelse(
        {net_price} <= 49, '$49 & Below',
        ifelse(
            {net_price} <= 99, '$50 to $99',
            ifelse(
                {net_price} <= 149, '$100 to $149',
                ifelse(
                    {net_price} <= 199, '$150 to $199',
                    '$200 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Low $100-$400',
    ifelse(
        {net_price} <= 99, '$99 & Below',
        ifelse(
            {net_price} <= 199, '$100 to $199',
            ifelse(
                {net_price} <= 299, '$200 to $299',
                ifelse(
                    {net_price} <= 399, '$300 to $399',
                    '$400 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Medium $200-$800',
    ifelse(
        {net_price} <= 199, '$199 & Below',
        ifelse(
            {net_price} <= 399, '$200 to $399',
            ifelse(
                {net_price} <= 599, '$400 to $599',
                ifelse(
                    {net_price} <= 799, '$600 to $799',
                    '$800 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'High $300-$1200',
    ifelse(
        {net_price} <= 299, '$299 & Below',
        ifelse(
            {net_price} <= 599, '$300 to $599',
            ifelse(
                {net_price} <= 899, '$600 to $899',
                ifelse(
                    {net_price} <= 1199, '$900 to $1199',
                    '$1200 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Very High $500-$2000',
    ifelse(
        {net_price} <= 499, '$499 & Below',
        ifelse(
            {net_price} <= 999, '$500 to $999',
            ifelse(
                {net_price} <= 1499, '$1000 to $1499',
                ifelse(
                    {net_price} <= 1999, '$1500 to $1999',
                    '$2000 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Ultra High $1000-$4000',
    ifelse(
        {net_price} <= 999, '$999 & Below',
        ifelse(
            {net_price} <= 1999, '$1000 to $1999',
            ifelse(
                {net_price} <= 2999, '$2000 to $2999',
                ifelse(
                    {net_price} <= 3999, '$3000 to $3999',
                    '$4000 & Above'
                )
            )
        )
    ),
    ${PriceBandSelection} = 'Extremely High $1499-$6000',
    ifelse(
        {net_price} <= 1499, '$1499 & Below',
        ifelse(
            {net_price} <= 2999, '$1500 to $2999',
            ifelse(
                {net_price} <= 4499, '$3000 to $4499',
                ifelse(
                    {net_price} <= 5999, '$4500 to $5999',
                    '$6000 & Above'
                )
            )
        )
    ),
    'No Price Band'
)

Regards,
Nikhil.

Hi @Nikhilburhade,

Do you mean that the following all work?
avgOver({net_price}, [{Dynamic Price Band}, Year], PRE_AGG)
avgOver({shelf_price} - {net_price}, [{Dynamic Price Band}, Year], PRE_AGG)
avgOver(({shelf_price} - {net_price}) / {shelf_price}, [{Dynamic Price Band}, Year], PRE_AGG) * 100

The only thing that looks different with the distinctCountOver is that it’s using concat. Try creating a separate calculated field to do the concatenation and reference it in your distinctCountOver.

I think I’ve run into issues like that before where QuickSight didn’t like when I used certain functions directly in my LAC-W calculations.

1 Like

Hello @David_Wong,

Thank You for your response.

Yes, formula is working fine except line - distinctCountOver(concat({part_number},{merchant_name}), [{Dynamic Price Band}, Year], PRE_AGG)

We have formula working fine for everything expect combination - ${ShelfChangesValueParam} : Placements, ${ShelfChangesParam} : Dynamic Price Band

I have tried creating Placement field which is - concat({part_number},{merchant_name}) and used that in our main formula like below -

it is showing the below error -

When clicked on show detail -

region: us-east-1
timestamp: 1746085634910
requestId: 5b7428a9-5b6e-4a1d-8d59-c712af486978
sourceErrorCode: DISTINCTCOUNTOVER_INVALID_ARGUMENT
sourceType: SPICE

I have mentioned formula for dynamic price brand in my previous comment.

Regards,
Nikhil.

1 Like

Hello @Nikhilburhade, I am running some tests here, and from what I can tell, the concat should not be causing an issue in this calculation. One thing I have done, is simplify the Dynamic Price Band calculation since there is no need to have so many nested ifelse statements:

ifelse(
    ${PriceBandSelection} = 'Ultra Low $10-$40',
    ifelse(
        {net_price} <= 9, '$9 & Below',
        {net_price} <= 19, '$10 to $19',
        {net_price} <= 29, '$20 to $29',
        {net_price} <= 39, '$30 to $39',
        '$40 & Above'
    ),
    ${PriceBandSelection} = 'Very Low $50-$200',
    ifelse(
        {net_price} <= 49, '$49 & Below',
        {net_price} <= 99, '$50 to $99',
        {net_price} <= 149, '$100 to $149',
        {net_price} <= 199, '$150 to $199',
        '$200 & Above'
    ),
    ${PriceBandSelection} = 'Low $100-$400',
    ifelse(
        {net_price} <= 99, '$99 & Below',
        {net_price} <= 199, '$100 to $199',
        {net_price} <= 299, '$200 to $299',
        {net_price} <= 399, '$300 to $399',
        '$400 & Above'
    ),
    ${PriceBandSelection} = 'Medium $200-$800',
    ifelse(
        {net_price} <= 199, '$199 & Below',
        {net_price} <= 399, '$200 to $399',
        {net_price} <= 599, '$400 to $599',
        {net_price} <= 799, '$600 to $799',
        '$800 & Above'
    ),
    ${PriceBandSelection} = 'High $300-$1200',
    ifelse(
        {net_price} <= 299, '$299 & Below',
        {net_price} <= 599, '$300 to $599',
        {net_price} <= 899, '$600 to $899',
        {net_price} <= 1199, '$900 to $1199',
        '$1200 & Above'
    ),
    ${PriceBandSelection} = 'Very High $500-$2000',
    ifelse(
        {net_price} <= 499, '$499 & Below',
        {net_price} <= 999, '$500 to $999',
        {net_price} <= 1499, '$1000 to $1499',
        {net_price} <= 1999, '$1500 to $1999',
        '$2000 & Above'
    ),
    ${PriceBandSelection} = 'Ultra High $1000-$4000',
    ifelse(
        {net_price} <= 999, '$999 & Below',
        {net_price} <= 1999, '$1000 to $1999',
        {net_price} <= 2999, '$2000 to $2999',
        {net_price} <= 3999, '$3000 to $3999',
        '$4000 & Above'
    ),
    ${PriceBandSelection} = 'Extremely High $1499-$6000',
    ifelse(
        {net_price} <= 1499, '$1499 & Below',
        {net_price} <= 2999, '$1500 to $2999',
        {net_price} <= 4499, '$3000 to $4499',
        {net_price} <= 5999, '$4500 to $5999',
        '$6000 & Above'
    ),
    'No Price Band'
)

I am also just wondering if this is a weird limitation with the distinctCountOver function. It seems to behave slightly different than all of the other LAC-W aggregations, so maybe that is what is causing the issue. This is going to seem a little ridiculous, but I am curious if we switch this to a sum function, if the error will be resolved.

Can you try to create a denseRank calculation for each distinctCountOver option? The first one would look like this:
Placements Brands Distinct Count Values = denseRank([concat({part_number}, {merchant_name} ASC], [{brand_name}, Year], PRE_AGG)

Then, we can try using this value in a sumOver rather than a distinctCountOver to see if that fixes the error you are seeing:
sumOver(ifelse({Placements Brands Distinct Count Values} = 1, 1, 0), [{brand_name}, Year], PRE_AGG)

I know this seems a bit overcomplicated, but I really just want to see if the error is related to the aggregation type rather than the values you are using within it.

1 Like

Hello@DylanM

Thank You so much for your quick revert.

I tried doing the same thing for ${ShelfChangesValueParam} : Placements, ${ShelfChangesParam} : dynamic Price band combination as rest of the combinations are working fine.

This is how the formula looks like :-

I am getting below error by using above formula -

My observation for dynamic price band as this is only causing the issue -

Point A] -

When we are testing - sumOver(ifelse({Placements price band Distinct Count Values } = 1, 1, 0), [{Dynamic Price Band},Year], PRE_AGG) then we are getting the result as below -

Where -

DynamicShelfChange =
ifelse(${ShelfChangesParam}=“Brands”,{brand_name}, ${ShelfChangesParam}=“Retailers”,{merchant_name},{Dynamic Price Band})

Price Band 1=
sumOver(ifelse({Placements price band Distinct Count Values } = 1, 1, 0), [{Dynamic Price Band},Year], PRE_AGG)

Placements price band Distinct Count Values =
denseRank([concat({part_number}, {merchant_name} ASC], [{dynamic price band}, Year], PRE_AGG)

Point B]-

Where -

DynamicShelfChange =
ifelse(${ShelfChangesParam}=“Brands”,{brand_name}, ${ShelfChangesParam}=“Retailers”,{merchant_name},{Dynamic Price Band})

– Now if we are adding

DynamicShelfChangeValue =
ifelse(${ShelfChangesValueParam}=“Placements”,{distinct_count(concat({part_number},{merchant_name}))},${ShelfChangesValueParam}=“Average $ Net Price”,avg({net_price}),${ShelfChangesValueParam}=“Average $ Promotion”,avg({Promotion Value}),(avg(Discount)*100))

to the values in pivot table then we are getting result as below for combination:

${ShelfChangesValueParam} : Placements,
${ShelfChangesParam} : dynamic Price band and
${PriceBandSelection} : ‘Very High $500-$2000’

So results of point A and Point B are matching without adding DynamicShelfChangeValue to the values in pivot table , but we are expecting the numbers as seen in the last image (comes after adding DynamicShelfChangeValue to the values in pivot table)
Problem causing combination is only :-
${ShelfChangesValueParam} : Placements, ${ShelfChangesParam} : dynamic Price band

Additionally Looks like using denserank and sumover combination making the calculations overcomplicated.
Regards,
Nikhil.

  1. Additional Note with regards to above comment :-
    I have also tried using below formula -

Where -
Placements Brands Distinct Count Values = denseRank([concat({part_number}, {merchant_name} ASC], [{brand_name}, Year], PRE_AGG)

likewise for retailers and price bands

We Do not get any visual error by this but values are not at all matching with the expected result as seen in the Point B of above comment.(comes after adding DynamicShelfChangeValue to the values in pivot table)
So looks like denserank and sumover logic is not working fine here.

2) More explanation about point A mentioned in above comment (May 1, 1:21 PM)

We are also able to get up to the highlighted numbers mentioned in the point A without mentioning price band 1 in the values of pivot table keeping the rows and column fields as it is.

Hello @Nikhilburhade, when you add the new calculated field into the pivot table, how is it being aggregated? From the image, it looks like it is aggregating either as Custom or a count. When you are using the LAC-W aggregations, you want to make sure you are returning the value as either Max or Min. That is because the partitions cause the value to be displayed on many rows, so it can easily make the number appear to be much larger than it is supposed to be.

Hello @DylanM

I Have also tried aggregating it as a Min / Max but still we are not able to get upto the expected results as i mentioned in previous comment ( result comes after adding DynamicShelfChangeValue to the values in pivot table)

For combination -
${ShelfChangesValueParam} : Placements,
${ShelfChangesParam} : dynamic Price band and
${PriceBandSelection} : ‘Very High $500-$2000’

Expected Result -

Where -

DynamicShelfChange=ifelse(${ShelfChangesParam}="Brands",{brand_name}, ${ShelfChangesParam}="Retailers",{merchant_name},{Dynamic Price Band})

DynamicShelfChangeValue = ifelse(${ShelfChangesValueParam}="Placements",{Placements (Unique)},${ShelfChangesValueParam}="Average $ Net Price",avg({net_price}),${ShelfChangesValueParam}="Average $ Promotion",avg({Promotion Value}),(avg(Discount)*100))

Result we are getting by Denserank, Sumover method by aggregating it as a min/max

Where -

price band 1 = sumOver(ifelse({Placements Price band Distinct Count Values } = 1, 1, 0), [{Dynamic Price Band}, Year], PRE_AGG)

Placements Price band Distinct Count Values =denseRank([concat({part_number}, {merchant_name}) ASC], [{Dynamic Price Band}, Year], PRE_AGG)

So what might be happening here is -
The issue with using denseRank here is that it assigns a rank of 1 to only one row per group (e.g., per price brand and year), so when summed, it always returns 1 instead of the true distinct count. This leads to significant undercounting compared to distinctCountOver, which evaluates all unique combinations correctly.

Regards,
Nikhil.

Hello @Nikhilburhade, if it is grouping too many values in the denseRank, maybe we just need to update the partition values on that calculation to ensure it is accounting for all individual groupings. I suggested only using brand_name and year because I thought that was the expected level of granularity. Is there another field besides those that could hide an instance that you would want to be included?

Hey @DylanM

Thank You for the response.

We are only interested in the specific granularity defined in our formula — for example: brand_name + year for Brands, merchant_name + year for Retailers, and dynamic price band + year for Price Band calculations. Nothing beyond these groupings is required for our use case.
We also tried the approach you mentioned (denserank + sumover), but unfortunately we’re still not able to achieve the expected results, as highlighted above(as in the comment from May 5, 9:01 PM).

Here is the detailed formula we’re using — everything works as expected except for the Placements + Price Band combination, where we currently have a NULL placeholder. We’re looking to replace that part with a working solution that matches the logic used for Brands and Retailers.

You can also understand the intended granularity directly from the formula structure:

  • brand_name + year for Brands,
  • merchant_name + year for Retailers,
  • and dynamic price band + year for Price Band calculations.

Let me know if you see a clean way to plug this into the formula without disrupting existing behavior.

Alternatively, would it be better to create a static version of the dynamic price band directly at the dataset level and use that in distinctCountOver to ensure consistent results?
(The challenge is that our current Dynamic Price Band field is parameter-driven, making it difficult to replicate exactly at the dataset level.)

Also, I came across a related issue that seems quite similar:
:link: DistinctCountOver not taking calculated field as partition by field

Regards,
Nikhil.

Hello @Nikhilburhade, maybe we try breaking this out a little more to see if we can resolve the error. It might be worth trying to separate each response to the SelfChangesValueParam into individual calculated fields. Then, we can see if they run as expected with the update of the SelfChangesParam value update without an issue. Afterwards, you can try plugging each individual calculated field into a grouped calculation and try running it again.

I haven’t seen that SPICE Limit exceeded error on a calculated field before, so I am hoping we can get some more information to try and debug the issue.