Lambda timing out when calling QuickSight APIs for dashboards, datasets, datasources, analyses, and theme

I am following this blog post on building an admin console on QuickSight to analyze and visualize QuickSight metrics. Building an administrative console in Amazon QuickSight to analyze usage metrics | AWS Big Data Blog

The AWS Lambda function “Data_Prepare” is scheduled to run hourly. This function calls QuickSight APIs to get QuickSight namespace, group, user, and assets access permissions information and saves the results to an Amazon S3 bucket. The Lambda stores this information into two different csv files that will then be stored in S3.

In order to access the data for dashboards, the function calls the API list_dashboards and goes into a for loop where for each dashboard, the API describe_dashboard_permissions is called and the necessary information is appended to a list. The same process is done for datasets, datasources, analyses, and themes, by calling list_datasets, list_datasources, list_analyses, list_themes, describe_data_set_permissions, describe_data_source_permissions, describe_analysis_permissions, and describe_theme_permissions.

Given the growth of the data that the Lambda is handling, and the long process of looping through each of these objects, the Lambda times out at 15 minutes, the maximum timeout for a Lambda function.

I am wondering if anyone else has ran into an issue of a Lambda timing out when doing QuickSight API calls or if there are other APIs that can give me the necessary information without having to loop and call for each object’s information.

Hi @cecardev,
Welcome to the community!

To get all the necessary information, the combination of listing the various asset types and getting details for each asset via the respective describe API is required. To avoid running into the timeout, you could either split the Lambda function up into multiple Lambda functions and chain them (e.g. using Workflow Orchestration - AWS Step Functions - AWS) or, if the runtime would still meet your own requirements, run the code as is within a container on a compute option of your choice (e.g. AWS Fargate, EC2 instances, etc.).

Did this answer your question? If so, please help the community out by marking this answer as "Solution!