Hello,
The data-set is loading the backend database table directly (select *)… and it has same # of rows as the backend database table. However, the SPICE data-set size is 3 times the actual table size.
Size in database:
Size in QS Spice:
Please advise.
Thank you
Hi @harpar1808 – The larger SPICE data set size compared to the backend database table size is expected behavior based on how SPICE calculates and stores data. This user guide page talks about how to estimate SPICE dataset size … SPICE calculates the logical size of a data set based on the data types and number of rows, rather than just the raw file size of the source data. The formula below helps in calculating the SPICE dataset size.
Logical dataset size in bytes = (Number of Numeric cells * (12 bytes per cell)) + (Number of Date cells * (12 bytes per cell)) + SUM ((24 bytes + UTF-8 encoded length) per Text cell)
So even if the source database table file is relatively small, once the data is loaded into SPICE and the logical sizes are calculated per the data types, the total logical size in SPICE can be significantly larger than the original file size, especially if there are many text or string columns in the data set.
Hi
From my experience QuickSight converts your data into a tabular dataset so if you have joins etc it can increase the number of rows.
Trevor
