Permissions for dataset ingestions

Question & Answer
, , ,
1

Hey all,

I am not able to understand the required permissions for a group to enable dataset ingestions.
The aim is to have limited and scoped permissions for different user groups.
I have a dataset in Athena with a related s3 bucket as data source.
On my group I have granted permissions to the bucket via an assigned IAM policy and in the main account I have Athena as overall resources established but when I run the dataset refresh on the dataset I get an error that I don’t have permissions to access the S3 bucket once I add the bucket to the main account permissions it works.
Any idea what is missing on my IAM group policy?

2

Thanks @tb102122 for posting your query. What we typically do while configuring accesses for Athena based data sources is below. Please check and let me know if this resembles with what you are doing or if there are any differences.

  1. We have a QuickSight group consisting all the users who are supposed to access ingest the tables.
  2. Add the S3 Bucket via Quicksight Management Console i.e. “Manage QuickSight” > “Security and Permissions” > “Manage” > “Select the relevant S3 Buckets”. We should not be manually adding any bucket access to the Service Role linked with QuickSight, that typically corrupts the policy. It is recommended to do this via the “Manage QuickSight” console.
  3. Since we govern access to glue databases via Lake Formation, you need to provide access to the Glue database/Tables via Lake Formation by providing the QuickSight Group ARN and Describe/Select accesses.
  4. In addition to we need to ensure that the underlying S3 Bucket is registered and access is granted on the bucket (from Lake Formation) to the same QuickSight group

image
image276×522 27.8 KB

Hope this helps!

3

Hey @sagmukhe thanks for your reply. I am considering to create IAM policies managed via terraform with the required permission per user group. So I thought adding the relevant bucket to the IAM policy used by the relevant group. But in that case I am getting the error that the S3 permissions are not correct. I am not using Lake Formation. So it is needed to add the bucket to the Group policy and the QuickSight Account policy?

4

@tb102122 - Thank you for responding back promptly. I believe your understanding is correct - we need to provide access to the bucket in the group Policy as that is what is governing access instead of Lake formation (in your scenario). As far as QuickSight account access to S3 Bucket is concerned, that needs to be done using the “Manage QuickSight” console as per the recommendation.

Adding few experts in case they have additional insights.

@ErikG, @David_Wong, @Sanjeeb2022, @DylanM, @duncan
@Deep @Koushik_Muthanna @Neeraj @gillepa

2 Likes
5

Hello @tb102122, I believe @sagmukhe is correct here! I will mark his response as the solution, but if you have follow-up questions or issues, please let us know.

1 Like
6

@DylanM @sagmukhe thanks for your help on the topic. I have one more questions regarding permissions. When would I need to setup permissions per user group to e.g. S3? Only in case of direct queries?