Use Case
Customers at times have challenges in understanding and obtaining information about the SPICE consumption details for a QuickSight account. The maximum SPICE capacity and SPICE capacity consumed in a QuickSight region can be monitored using the Amazon CloudWatch automatic dashboards for QuickSight. As QuickSight adoption grows, customers have the need to analyze how their SPICE is consumed and to secure insights that can help them proactively monitor SPICE datasets.
In this article, we will consider the following 3 use cases:
-
ISV customers with multiple namespaces who would like to know the SPICE usage for each dataset across customer namespaces/groups for all regions.
-
Customers who would like to determine datasets which are created via file upload or via other data sources and analyze SPICE consumption for each of them.
-
Scenarios where customers would like to identify all duplicate datasets or any orphaned datasets not being utilized in an account for housekeeping purposes.
Solution
Using Amazon QuickSight API’s, we can build a solution that can retrieve all the dataset details along with their permissions and ownership and store them in a database or csv files in an Amazon S3 bucket. The data can then be used to build a QuickSight dashboard. Below are the steps to generate the data:
- List all datasets along with SPICE capacity using ListDataSets and check for Import mode as SPICE.
- Using the list above, now describe the datasets using DescribeDataSet and get creation date, last updated date, and their permissions using DescribeDataSetPermissions.
- Retrieve information about File based datasets and their details using ListIngestions.
- Track SPICE usage by users, groups or namespaces ,get list of all users across all namespaces using ListUsers.
- List all dashboards using ListDashboards and identify dependent datasets with DescribeDashboard.
The below flowchart outlines the steps required to build the solution:
Shown below is a sample dashboard built in QuickSight from the data extracted to an S3 bucket.
The process to extract all dataset information using API operations can be scheduled to run frequently, and results can be pushed to an S3 bucket in your account. Furthermore, we can use new suites of data ingestion APIs from this link to refresh data in SPICE providing updated view for all insights in dashboard.
In this article, we demonstrated how you can get SPICE consumption details for all datasets for your QuickSight account. We also showed how you can easily determine SPICE usage as well as which datasets are created with the upload file option and which ones are not file-based. Once you obtain all of the above information, you can easily visualize and derive insights and create alerts for dataset usage.