How to visualize Cross-Region Amazon DynamoDB insights in Amazon QuickSight using the Amazon Athena DynamoDB connector and AWS Glue

Question & Answer
1

Existing Setup:

1 DDB table per region. Same schema. Each table in each region contains data for clients in that region.

What I am trying to achieve:

In 1 QuickSight analysis in the us-east-1 region, I want to load & refresh data from these multiple DDB tables (1 per region) once a day.

I do understand that there would be data transfer charges when the cross-region DDB tables are refreshed (once-a-day) and I am ok with that.

What worked:

To achieve the above goal, I tried the steps mentioned in this AWS Blog -

I was able to follow these steps and load the data from us-east-1 DDB table into the Quicksight analysis in us-east-1 region.

What didnt work:

Attempt 1:

Now, for my us-west-2 DynamoDB table, I followed all the steps mentioned in the above blog and created the following resources in us-west-2 region:

  • Athena DynamoDB Connector

  • AWS Glue Database

  • AWS Glue Crawler (which in turn created a new Glue Table in the same region)

Using these resources, I tested the connector with the Athena SQL editor (us-west-2) and I was able to read the data from DDB.

Now, when I came to the step of allowing QuickSight to access the above resources, the Athena DynamoDB connector Lambda Function (us-west-2) was not appearing in QuickSight (us-east-1). A quick Google Search said that " You cannot connect QuickSight to Athena in another region"

Ref: Athena queries for different region

Attempt 2:

Hence, to move past this constraint, I made my Athena DynamoDB Connector in us-east-1 while the Glue Database and Glue Table were still in us-west-2 and then tested the connector with the Athena SQL editor which gave the following error-
"SCHEMA_NOT_FOUND: line 1:15: Schema ‘<database_name>’ does not exist "

Attempt 3:
And if I try to create a Glue Database and a Glue Crawler (to create a Glue Table) in us-east-1 and give the DynamoDB table as the datasource which is in us-west-2, then I get the following error-
“DynamoDB target <table_name> does not exist.”

Hence in summary, I am not able to run the Athena SQL (us-east-1) as I am not able to create a Glue Crawler (us-east-1) that can Crawl on a DynamoDB (us-west-2) table that is present in another region and neither can I create an Athena query (us-east-1) on a Glue Table (us-west-2) present in another region (us-west-2) (same region as the DynamoDB table).

I am not able to find a solution for this problem on any AWS Documentation or related Blogs or Quicksight community.

2

Hello @rajmishra !

I would recommend creating a support ticket. Here are the steps to open a support case. If your company has someone who manages your AWS account, you might not have direct access to AWS Support and will need to raise an internal ticket to your IT team or whomever manages your AWS account. They should be able to open an AWS Support case on your behalf.

1 Like
3

Hello @rajmishra, since we have not heard back from you, I will archive this topic. If you need further assistance on this issue after discussing it with Support, please post a new topic in the community and link to this topic. That will ensure you are at the top of the priority list for a response from one of our QuickSight experts. Thank you!