Calculations introducing extra decimal places

I noticed that sometimes my calculations add extra decimal places to my numbers. For example, the numbers in the screenshot below are calculated but they should be exactly 45, 25, 45, 50, 45, 60 and 45 respectively. When I export this visual to CSV, I don’t see all the extra decimal places.

Normally this would be fine because I can easily set the number of decimal places to 0 at the visual level. The problem is when I add a top/bottom filter, QS doesn’t correctly identify ties because of all the extra decimal places in the calculated numbers.

Is there a reason why I’m getting these extra decimal places in my calculated numbers?

Decimal Places

Hi @David_Wong -

I’ve run into this issue and believe it’s a bug with QuickSight.

You can use this workaround.

  1. round your score to 2 decimal places
  2. use tostring, split, substring, and locate to truncate the extra decimal places
  3. convert back to decimal with parseDecimal

Decimal

parseDecimal(concat(split(toString(round({score},2)),".",1),substring(toString(round({score},2)),locate(toString(round({score},2)),".",1),3)))

Int

parseInt(split(toString(round({score})),".",1))

Hi, Thanks for bringing this issue to us for attention. We will dive deep and work on a fix for it.

1 Like

@David_Wong, What is the type of “Score”? Is it Decimal or Float?
And what aggregation are you using for Score is it “SUM”, “MIN”, “AVG” etc?

QuickSight on final result rendering uses Float type which can sometimes be in-precise (language agnostic - Why are floating point numbers inaccurate? - Stack Overflow).

That said I am not sure if the TopBottom issue with tie-breakers is related to residual values that you see in the visual.

1 Like