Automating asset-bundle export

Question & Answer
,
1

Hi,

Our team is trying to automate asset-bundle migration using boto3. We have been able to export the asset bundle, post this we are able to download the bundle from the presigned url.

However, we are unable to upload it to s3. I need help with 2 queries here:

  1. Is there a way we directly put the bundle object in s3 from the presigned url instead of downloading to local machine?
  2. What could be the possible issue with put object in s3?
    Error message : upload failed: .\quicksight_bundle.qs to s3:////quicksight_bundle.qs An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.
    Error copying the asset bundle to S3: Command ‘aws s3 cp quicksight_bundle.qs s3:////quicksight_bundle.qs’ returned non-zero exit status 1.

Thanks,
Prantika

2

Hello @prantika_sinha, here is some documentation that includes an example about downloading the asset-bundle directly to S3.

Are your S3 bucket and the QS account you are migrating from in the same AWS account. You may be using the wrong Access Key for your destination bucket.

3

Hi Dylan,

Appreciate the response. Yes, the S3 bucket and QS account is in same AWS account for the code trial I am running. However, in the ideal scenario we need to upload it in the s3 bucket of the target account.

Also the link you shared take me to the import job, which is next step to the bundle upload in S3. Let me know if I am missing something here.

Thanks.

4

Hello @prantika_sinha, I am thinking the issue is with the presigned url and S3 bucket access. The bucket policy needs to be updated to make sure it has access to receive information from the presigned url. Plus you need to make sure your account has access to put objects in the S3 bucket. One more thing to note, the presigned url may require an expire date to be passed in with the AccessKeyIds for the QuickSight account and S3 bucket. Here is a link to some bucket policy documentation.

5

No, you need to download the asset bundle file first using the pre-signed URL.

Then, you can call import API with the asset bundle file that you downloaded, without having to upload to another S3 bucket first.

If you are using awscli, you can use:
--asset-bundle-import-source-bytes fileb://file_name.qs

You can do the same using the AWS SDKs.

1 Like
6

Thanks @zhaopan.
We are trying to put a terraform script for this implementation and want to skip the download to local step. My question is more around if there is any way we can bypass this and directly put to S3.

Adding to your response @DylanM , the presigned url is auto generated based on the export command and the account has access to put object in S3. Both these checks are done.

7

At the moment, no. There are a number of ways you can download file in Terraform. remote-exec privisioner + curl seems to be the recommended method.