I’m trying to optimize cost, and I came across someone stating that SPICE is built on top of S3. I haven’t found any other resources that actually confirm that anywhere, so the assertion seems shaky to me. Does SPICE storage have anything to do with S3?
I’m asking because with other AWS services, the costs are allocated specifically by service which adds some complexity to attributing costs to business operations. For example, running an Athena query means that I’m charged for the query allocated to Athena, for the S3 get operation to retrieve the data allocated to S3, and for the storage of the results. This would compound potentially with QuickSight. Ingesting data into SPICE from an Athena query might create a billable event for the query itself, storing the query results in S3, collecting the results from S3 into SPICE, and whatever SPICE is doing. I’m mainly trying to understand the “whatever SPICE is doing” part. If SPICE is built on S3, then dashboard operations would trigger many S3 get operations. If SPICE has nothing to do with S3 billing, then it might be cheaper to enable spice for most direct query datasets. A good chunk of our monthly costs are for S3 get operations specifically, so I’m trying make sure there aren’t any sources that we’re not accounting for.
SPICE stands for Super-fast, Parallel, In-memory Calculation Engine, this is separate from S3. If you use SPICE for your dataset, the underlying data source is only queried when SPICE refreshes (which you can trigger manually, on a schedule, or via the API).
If you do not use SPICE then the queries from the dashboard will be sent to the underlying data source every time the dashboard is loaded by a user.
If using Athena as a datasource then I would recommend using SPICE where possible. The will reduce the number of Athena queries made.
Thank you for your replies, they are helpful but I’m looking for something slightly different. I’m fairly familiar with the QuickSight SPICE and cost documentation. Some of the other AWS product docs don’t explicitly mention what downstream costs will occur, but say that depending on usage other costs may occur. A great example of this is with the Athena docs, AWS covers the query cost itself in great detail, then states that other costs may occur (referring to S3 Get/Put operations to actually store and do something with the output). My question is mostly targeted at understanding if there are any downstream/tangential costs from utilizing SPICE storage. I didn’t see any mention of additional charges resulting from SPICE usage in the docs, so am I safe to assume then there are none? I suppose I might be able to review the Cloud Watch logs after a SPICE refresh or something to be sure, but I’d rather not have to dig that deep.
Thanks @meow . This is a very good question and good to know you are looking for the actual cost of SPICE ( in case of refresh). I believe it depends upon the source. For example, if your data source is S3 or Athena, then it depends upon the amount of data it scan in athena and number of gets request from S3 as well. From relational database, it also depends upon which type of database and the cost engaged with respect to database. if your QuickSight is in VPC or not in SPICE, data transfer charges are there or not, i am not sure but usually ingress cost is always free.
For SPICE the only cost is the per GB charge. SPICE can be queried as much as you want without any additional charges.
If the data in SPICE needs to be refreshed with new data, then you are interacting with the underlying data source to do the refresh e.g. Athena + S3… then you will be charged for that query. However this will be a single query to load all the data in to SPICE on each refresh (incremental refresh is also an option), and not multiple queries like you have if using it as a Direct Query dataset.
For example you can see here I have a SPICE dataset in QuickSight using an Athena Data Source, it is set to refresh every hour, so there is one query an hour, that incurs a small amount of cost.
So there are no downstream data-usage related billable events that occur as a result of using SPICE storage? Effectively, once the data is stored in SPICE, I’m not going to be charged for usage from any other AWS services for using that SPICE data in QuickSight.
Only the green highlighted boxes are billable events related to data usage.
