Dataset Import failed with ""Table Not found" when dataset is created using API in AWS IDC user QuickSight Account

Question & Answer
,
1

I have created an AWS account and created AWS Identity Center User. Then created QuickSight Account using AuthenticationMethod as AWS Identity center user. Added the user to Admin and Author groups.

Then I have run list-users API and fetched the arn of the user.
In the Lake Formation, I have provided access to Glue catalog Tables and Databases for that QuickSight user arn .

Logged into the QuickSight Console using AWS IDC user authentication and was able to create Athena Datasource and was able to see all tables and databases in GlueCatalog. Was able to successfully created Dashboards, Analysis and Datasets.

Now I am trying to Create same Datasource, Datasets , Analysis and Dashbaords in AWS IDC user account using Quick Sight API as an Account Admin who is not QuickSight user.
When I logged into QuickSight console as AWS IDC user - I do see all assets are created.

But in the Datasource, I do not see any tables.
In the Datasets, Import failed with “TABLE_NOT_FOUND”.

Question is same AWS IDC user is able to see the tables and datasets successfuly when logged into console and create but when created using APIs, facing issues ?

I have run similar test on another QuickSight account for an IAM federated user and I do not see any issues. Is it something to do with APIs and AWS IDC user?

2

Hi @thishali, it appears that you are encountering a common issue related to permissions and the scope of API access in AWS QuickSight when using AWS Identity Center (IDC) for authentication and authorization. Here’s a detailed breakdown and some potential solutions to troubleshoot and resolve the issue:

Possible Causes

  1. API Permission Scoping: When you create resources in QuickSight using the AWS console as an IDC-authenticated user, you are directly authenticated and authorized via your IDC permissions. However, when using the API, the permissions might not be scoped or propagated in the same way, especially if the API calls are being made under different credentials (like an Account Admin who is not a QuickSight user).
  2. Role-based Access Control: If the admin account making the API calls doesn’t explicitly have permissions to view or interact with the specific datasets or data sources, this could lead to the issues you are experiencing. This is different from logging in through the console where your IDC permissions are directly applied.
  3. API Credential Configuration: Ensure that the credentials used in API calls are correctly configured. AWS Identity Center integrates with QuickSight through roles and policies that need to be explicitly allowed to interact with QuickSight APIs.

Troubleshooting Steps

  1. Review IAM Policies: Check the IAM policies attached to the role or user making the API calls. Ensure that they have permissions not just for QuickSight API actions but also for accessing the necessary resources in AWS Glue and Athena.Example permissions to include in IAM policy:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "quicksight:*",
        "glue:GetTable",
        "glue:GetDatabase",
        "athena:*"
      ],
      "Resource": "*"
    }
  ]
}
  1. Use the Correct QuickSight Role: When making API calls, ensure that the credentials being used are associated with a role or user that has been assigned the appropriate QuickSight roles (such as QuickSight Admin or Author). This can be configured in the QuickSight admin panel under Manage QuickSight → Manage Users.
  2. Debug API Calls: Use AWS CloudTrail and the QuickSight console logs to track API requests and errors. This can provide insights into any permission or configuration issues.
  3. Consistency of User ARNs: Double-check that the ARN used in API calls matches the ARN that has permissions in Lake Formation. Sometimes, subtle discrepancies in ARN formatting or outdated ARN references can cause access issues.
  4. Simulate API Permissions: Use the IAM policy simulator to test the policies attached to the credentials making the API calls. This can help identify any missing permissions or configurations.

Final Recommendations

If after reviewing and adjusting the above settings the issue persists, consider involving AWS Support. There could be additional configurations or hidden issues related to AWS Identity Center integration with QuickSight that are not immediately apparent. AWS Support can provide more detailed logs and guidance specific to your account setup.

Did this solution work for you? I am marking this reply as, “Solution,” but let us know if this is not resolved. Thanks for posting your questions on the QuickSight Community!

4

From the cloud trial events , I see that Athena: StartQueryExecution request failed during creation of QuickSight dataset/datsource - error is “TABLE_NOT_FOUND: line 3:101: Table ‘XXXXXXXX’ does not exist”

I have compared the Cloud Trail events for Athena: StartQueryExecution API during createDatasource
between Datasource created by API call assuming Admin and Datasource created from QuickSight Console.

Only difference I see is QuickSight Console request has : “workGroup”: “primary” set in the StartQueryExecution request and where as request that failed had no work group set.

I think that should not impact though.

Also, I have verified all the permissions mentioned in the post and everything looks good but issue still persist

5

Hi @thishali, thank you for letting us know. I would recommend filing a case with AWS Support where we can dive into the details so that we can help you further. Here are the steps to open a support case. If your company has someone who manages your AWS account, you might not have direct access to AWS Support and will need to raise an internal ticket to your IT team or whomever manages your AWS account. They should be able to open an AWS Support case on your behalf. Hope this helps!