Build embedded analytics architectures using Amazon QuickSight

amazonquicksight · October 5, 2023, 4:13pm

Business applications generate a lot of data, such as clickstream data, sales orders, operational metrics, and more. Organizations are trying to make sense of this data, and also want to share insights with the users of their applications. Application owners are constantly looking for new ways to differentiate their applications to enable their users to derive valuable data insights; therefore, embedding analytics within their applications is emerging as a key trend. They are modernizing their business applications on the cloud with embedded analytics to improve their decision-making and build better customer experiences.

This post helps cloud architects, cloud engineers, and developers build embedded analytics architectures on the AWS Cloud using AWS purpose-built data analytics services and Amazon QuickSight by providing various architectural patterns for building personalized dashboards based on user role or job function. We also discuss best practices and key considerations that showcase the power of embedded analytics, and share additional resources for getting started with building embedded analytics on AWS.

Benefits of building cloud embedded analytics

Building embedded analytics with on-premises tools and data stores can be challenging and expensive. To build a compelling embedded analytics experience, you require front-end engineers, UX designers, and other resources who are focused on the analytics experience. This means hiring these specialized resources or diverting those that you already have allocated to the core application effort. These analytics experiences also often involve the use of charting libraries, which need to be updated and patched, adding maintenance overhead. Depending on the complexity of the data, you might also need a back end analytics infrastructure such as a data warehouse or data lake to store, process, and derive insights.

Additionally, this can be time-consuming. To build a compelling experience, you often have to build in filtering, drill down, and interactivity in your visuals—features that mirror typical business intelligence (BI) functionality. Replicating this feature set can be a long process. As you dive deeper into the experience and want to offer more personalized and differentiated experiences, project cycles can stretch from initially planned months to years.

Lastly, after building a great experience, you want it to scale with your application’s success. This means that because you have both planned and unplanned bursts in growth, the analytics experience must remain just as snappy as it was on day one. To ensure this, you need to build extensive monitoring and management, and also manage infrastructure growth. This brings us to embedded third-party BI solutions, which make it faster to get to market, but have various challenges with scalability, continuous upgrades, and integrating with other machine learning (ML) frameworks, leading to higher costs and time to market. Cloud embedded analytics enables you to quickly build and manage persona-centric dashboards to improve user engagement.

What is embedded analytics ?

Embedded analytics enables you to integrate business analytics capabilities into your own applications (such as software as a service) and web portals. This integration empowers users to gain deeper and faster insights through the embedded nature of interactive visualizations and dashboards, allowing for easier in-depth analysis of data.

Without embedding capabilities, standalone BI tools force users to switch back and forth between their preferred application and the BI tool in order to access the proper capabilities needed to analyze data and derive insights. This continuous need to sync data from your application to a standalone BI solution reduces productivity and the quality and pace of decision-making for your organization. Embedded analytics integrates BI capabilities directly into your application, thereby empowering users to discover actionable insights without dealing with the complexities of a separate BI tool.

Embedded analytics allows for a seamless blend of analytics into your own web portal or application, making it easier for your users to adopt new analytics capabilities and improve their user experience. By differentiating product offerings, businesses can create additional revenue streams through upselling embedded analytics capabilities to their customers. For example, an independent software vendor (ISV) that sells manufacturing automation software can embed analytics within their software, allowing its end-customers to analyze data and gain insights in a secure way on their own data, delivering additional value to customers. Through ML and ad hoc capabilities (such as building custom dashboards and reports), embedded analytics allows businesses to self-serve and generate deeper insights for their users.

Embedded analytics with QuickSight

QuickSight lets you quickly embed interactive dashboards and visualizations into your applications without needing to build your own analytics capabilities. QuickSight provides comprehensive embedding capabilities to meet all your needs, be it visual-level embedding for high-traffic applications, dashboard embedding for in-app analytics, embedded authoring for power users, Q embedding for self-service insights, or anonymous public embedding on websites.

QuickSight has a serverless architecture that automatically scales from tens to hundreds of thousands of users, eliminating the need to set up, configure, or manage servers. QuickSight is fully managed, so you never have to worry about manually upgrading to newer versions.

QuickSight offers rich APIs and SDKs that let you integrate QuickSight dashboards, set defaults, connect controls, and handle errors to match your applications user experience. QuickSight APIs allow you to programmatically manage your analytics workflows, such as moving dashboards across accounts, creating tenant-specific datasets, automating deployments, and managing access for users with single sign-on (SSO).

With QuickSight ML capabilities such as anomaly detection, forecasting, and natural language queries, you can easily gain deeper insights into your data. These capabilities provide you with new ways of monetizing and differentiating your applications. QuickSight has high availability built in, native connectors to a variety of data sources including AWS services, on-premises databases and business applications, and granular access control such as row-level and column-level security to securely share your data only with the intended audience. QuickSight offers pay-as-you-go pricing models with no upfront investments so you can take your solutions to market and scale your costs as adoption scales.

Embedded analytics architecture patterns

When you’re building an embedded analytics architecture on AWS, you must first understand your user personas and your specific use case so that you can choose the right design pattern for building embedded analytics with your business applications.

These architectures can be divided into two main categories:

No-code architectures
Code-based architectures

No-code architectures

Analytics and especially BI users are often not technically equipped with programming languages and not as fluent in writing code. This becomes a bottleneck for scaling the adoption and integration of BI dashboards into business applications. This is when no-code architectures are really powerful because business analysts can copy the embed code for a dashboard with a click of a button, paste it in the respective application, and make it available to end-users.

There are two main types of architectures in no-code embedding:

Embed dashboards for the public
Embed dashboards for enterprises

Embed dashboards for the public

There could be scenarios where you want to show the same dashboard for tens of thousands of users without any user management, and the dashboard contains the same data for all users. For this design pattern, you can embed visuals and dashboards on a public website that contains the data statistics that update every hour and needs to be shared with tens of thousands of users. QuickSight supports such use cases and allows you to stand up these dashboards and visuals in minutes, without any user or server provisioning.

In this architecture, the concept of no-code public embedding is applied, where you can embed a visual or dashboard in your application using an embed code. The following diagram illustrates the solution workflow.

The workflow steps are as follows:

Copy the embed code from QuickSight to the public website.
Allow list the domain for the public internet website in QuickSight and make dashboards publicly accessible.
The user accesses the portal.
The dashboard or the visual is rendered at runtime.

The dashboard or visual is publicly accessible in this case and must be used only on portals or websites where the data is intended to be publicly accessible. Examples include school district information, stock tickers, or population metrics of a city or town.

Embed dashboards for enterprises

In other cases, there could be dashboards or visuals that you would like to share across your company with key KPIs that are intended for all employees on an internal portal. In these cases, you want to keep the dashboard secured and accessible only by employees of your organization, for this QuickSight offers enterprise no-code embedding. The only difference between public embedding and enterprise embedding is that there are no users to manage in public embedding and the dashboard is publicly accessible.

In this architecture, the concept of no-code enterprise embedding is applied, where you can embed a visual or dashboard in your application using an embed code. The following diagram illustrates the workflow.

The workflow steps are as follows:

Copy the embed code from QuickSight to the public website.
Allow list the domain for the intranet portal in QuickSight and make dashboards publicly accessible.
Users access the portal.
If they are not authenticated with QuickSight, they are prompted to authenticate.
Upon successful authentication, the dashboard or visual is rendered at runtime.

Code-based architectures

QuickSight offers capabilities to embed dashboards and visuals without needing to write code, but that comes with its own limitation of lack of seamless interactivity between the application and the dashboard or visual itself. In cases where you want to have such interactivity and dynamic data filtering capabilities driven by your application, it’s recommended to implement code-based architectures. In a nutshell, QuickSight offers embedding APIs and an embedding SDK that can be used for seamless integration. Let’s dive into some of the diverse use cases in the following sections.

Build embedded dashboards for QuickSight users

There are scenarios where you want to show persona-centric or user role-based dashboards for your users based on user-level permissions and data restrictions. In this case, when the user accesses the webpage or application and requests the page with embedded dashboards, the app server requests QuickSight for the embedded dashboard with user details, and QuickSight checks to ensure the user has access to the dashboard and makes sure the row- and column-level permissions are applied. QuickSight enables you to manage and maintain named users, along with the ability to maintain row- and column-level permissions on data. This allows you to scale to tens of thousands of users in your apps, with the right level of permissions.

The following diagram illustrates the workflow for building embedded visuals and dashboards into business applications with QuickSight users based on user role or job function.

The steps are as follows:

The end-user accesses the application on a web or mobile client.
The application client requests a page with an embedded dashboard.
The application server confirms the end-user identity from the identity store. The identity store can be AWS Identity and Access Management (IAM) or an external identity provider like Okta or Ping Identity.
When using the QuickSight embedding API, the application server invokes a GenerateEmbedUrlForRegisteredUser call to obtain the embedded dashboard URL for the specific user. Refer to Embedding QuickSight data dashboards for registered users and Embedding QuickSight visuals for registered users for additional details on how to embed dashboards and visuals.
The application server embeds the signed dashboard URL in the application using the QuickSight JavaScript SDK.
The end-user sees the application page loaded with the embedded QuickSight dashboard.

Build embedded dashboards without user provisioning

In a larger enterprise, there are scenarios where a large number of users (thousands) access business applications and aren’t known users. In these cases, user management becomes a bottleneck for scalability. To support such use cases, QuickSight offers a programmatic approach of using APIs to request dashboards for users that don’t exist in an QuickSight account. QuickSight enables users to request dashboards or visuals while restricting data by row-level permissions. This allows you to scale to hundreds of thousands of users in your apps, with the right level of permissions.

User authentication happens at the application level. If there is no user authentication as part of the application, the dashboard or visual loaded using this approach will be publicly accessible through the business application.

The following diagram illustrates the workflow for building embedded visuals and dashboards into business applications without user provisioning or building a common corporate embedded dashboard.

The steps are as follows:

The end-user accesses the application on a web or mobile client, where they are prompted to authenticate.
Upon successful authentication, the application client requests the page with the embedded dashboard.
When using the QuickSight embedding API, the application server invokes a GenerateEmbedUrlForAnonymousUser call to obtain the embedded dashboard URL. Refer to Embedding QuickSight data dashboards for anonymous users and Embedding QuickSight visuals for anonymous users for more details on embedding dashboards or visuals for non-QuickSight users.
The application server embeds a signed dashboard URL in the application using the QuickSight JavaScript SDK.
The end-user sees the application page load with the embedded QuickSight dashboard.

Build custom dashboard authoring in applications

There are many cases where your users want more than just out-of-the-box dashboards and reports. QuickSight authoring can be embedded in applications, allowing you to provide a curated data exploration and dashboard creation experience within your business applications. This provides a scalable experience that can provide new revenue streams and strong differentiation for your applications. The difference here from the embedded dashboard flow with QuickSight users is that the app server requests for an embedded session instead of a specific dashboard, and the URL returned then opens the session for the power user.

The following diagram illustrates the workflow to build custom embedded analytics into business applications.

The steps are as follows:

The power user accesses the application.
The application client requests the page with the embedded authoring functionality.
The application server confirms the power user identity from the identity store.
The application server invokes a GenerateEmbedUrlForRegisteredUser call with QuickSightConsole for session configuration to obtain an embedded authoring session for the specific user.
The application server embeds the signed embedded authoring URL in the webpage using the QuickSight JavaScript SDK.
The end-user sees the application page load with the embedded QuickSight authoring functionality.

Build a multi-tenant architecture with namespaces

In many ISV use cases, it’s required to separate end-customers with flexible settings in order to provide embedded visual and dashboards in a single application. You can build multi-tenant architectures with QuickSight namespaces, which provide logical grouping and isolation of users within an QuickSight account.

The following diagram illustrates the workflow to build custom embedded dashboards and visuals into business applications with multi-tenancy.

The steps are as follows:

You must first create datasets with customer-specific filters.
You can create a dashboard by using the template API to create dashboards.
Namespaces allow authors and readers of one organization to be isolated from one another.
From QuickSight, the application server can retrieve customized dashboards for each client.
User credentials are checked against the identity store by the application server.
Customer-specific dashboards from QuickSight are rendered by an application server.

Refer to Embed multi-tenant analytics in applications with Amazon QuickSight for more details on this design pattern.

Build natural language querying capabilities in embedded analytics

With today’s developments in generative AI, every use case and architecture calls for a way to integrate natural language capabilities into business applications. This brings us to the QuickSight natural language querying option for enterprise applications called Amazon QuickSight Q. Q uses ML and built-in large language models (LLMs) to automatically understand the intent of the question being asked and map it to the underlying business data to provide instant and accurate answers with relevant visualizations. Q can be also be embedded into business applications so that end-users can ask questions in natural language within the application and get instant answers.

The following diagram illustrates the workflow for embedding the natural language querying capability into business applications.

The steps are as follows:

The end-user accesses the application on a web or mobile client.
The application client requests a page with an embedded Q search bar.
The application server confirms the power user identity from the identity store. (This is optional for non-QuickSight users.)
When using the QuickSight embedding API option, the application server invokes a GenerateEmbedUrlForRegisteredUser or GenerateEmbedUrlForAnonymousUser call to obtain the embedded Q search bar URL.
The application server embeds the signed Q search bar URL using the QuickSight JavaScript SDK.
The end-user sees the application page load with the embedded Q search bar.
The user can ask questions from the Q search bar and get answers instantly.

Next steps

Consider the following next steps to expand your embedded analytics:

Start by setting the business objectives for your application if you add an embedded analytics component.
Identify your customer user personas, who will be consuming the insights, and identify how you can help them be better at their job with embedded analytics. This could be translating KPIs and metrics into embedded dashboards or visuals within business applications.
After you identify your business objectives and associated embedding dashboards and visuals, choose an appropriate embedding design pattern.
Ensure your customers only see the dashboards and data they are allowed to see by setting up correct access controls. You can use row-level and column-level restrictions as needed. You can also use SAML 2.0 to federate with your external identity provider for your identify management.
After your embedding analytics has successfully been launched, use customer feedback to fine-tune your product as needed.

Conclusion

This post demonstrated various design patterns for embedding BI visuals and dashboards into your enterprise applications. You can build your own embedded analytics with QuickSight using the information in this post.

For detailed architectural patterns, refer to the following resources:

About the Authors

Raghavarao Sodabathina is a Principal Solutions Architect at AWS, focusing on Data Analytics, AI/ML, and cloud security. He engages with customers to create innovative solutions that address customer business problems and to accelerate the adoption of AWS services. In his spare time, Raghavarao enjoys spending time with his family, reading books, and watching movies.

Srikanth Baheti is a Specialized World Wide Principal Solution Architect for Amazon QuickSight. He started his career as a consultant and worked for multiple private and government organizations. Later he worked for PerkinElmer Health and Sciences & eResearch Technology Inc, where he was responsible for designing and developing high traffic web applications, highly scalable and maintainable data pipelines for reporting platforms using AWS services and Serverless computing.

Raji Sivasubramaniam is a Sr. Solutions Architect at AWS, focusing on Analytics. Raji is specialized in architecting end-to-end Enterprise Data Management, Business Intelligence and Analytics solutions for Fortune 500 and Fortune 100 companies across the globe. She has in-depth experience in integrated healthcare data and analytics with wide variety of healthcare datasets including managed market, physician targeting and patient analytics.

This is a companion discussion topic for the original entry at https://aws.amazon.com/blogs/business-intelligence/build-embedded-analytics-architectures-using-amazon-quicksight/