Asset Bundle Import Job for Snowflake Datasource not working

Hi everyone,

the boto3 API has had the “asset_bundle_export/import_job” functions for several months. I am currently trying to migrate a Snowflake asset including an analysis, dataset and datasource to another account. The export works without any problems. When importing, I noticed that I have to adjust the VPN configuration in the AssetBundle so that the datasource works in the target account, otherwise I get errors that the subnets are not recognised. I have solved this problem by overriding the VPNConnectionJson with configuration from the target account. So if I import the analysis alone, it works without any problems. However unfortunately not if I try to import all other assets as well. I get the error message:

Response

{
“statusCode”: 500,
“body”: ""An error occurred: Import job failed: [{‘Arn’: ‘arn:aws:quicksight:eu-west-1:211125443023:datasource/ca62a712-3cc1-48e8-b4cb-da46f2ad28ba’, ‘Type’: ‘com.amazonaws.services.quicksight.model.InvalidParameterValueException’, ‘Message’: ‘Feature is not supported for this data source type’}]"
}

This is the datasource JSON as exported with “start_asset_bundle_export_job”:

{
“resourceType”: “datasource”,
“dataSourceId”: “ca62a712-3cc1-48e8-b4cb-da46f2ad28ba”,
“name”: “Snowflake Int XS”,
“type”: “SNOWFLAKE”,
“dataSourceParameters”: {
“snowflakeParameters”: {
“host”: “xxx",
“database”: "xxx”,
“warehouse”: “xxx",
“authenticationType”: “password”
}
},
“vpcConnectionProperties”: {
“vpcConnectionArn”: "arn:aws:quicksight:eu-west-1:217169904035:vpcConnection/xxx”
},
“sslProperties”: {
“disableSsl”: false
},
“permissions”: [
{
“principal”: “arn:aws:quicksight:eu-west-1:217169904035:user/default/xxx”,
“actions”: [
“quicksight:PassDataSource”,
“quicksight:DescribeDataSourcePermissions”,
“quicksight:UpdateDataSource”,
“quicksight:UpdateDataSourcePermissions”,
“quicksight:DescribeDataSource”,
“quicksight:DeleteDataSource”
]
}
]
}

xxx = filler
I modified already the account number everywhere and the vpcConnectionArn
What do I have to modify here also?

1 Like

Hello @kosta, upon first glance, I think the parameter that it is throwing an error on is the authenticationType param within the snowflakeParameters section. When looking through the boto3 documentation, it only shows 3 possible parameter options for snowflake:

'SnowflakeParameters': {
                        'Host': 'string',
                        'Database': 'string',
                        'Warehouse': 'string'
                    },

Then, you might want to utilize the Credentials parameter to handle the connection to Snowflake if it isn’t managed by the VPC:

 'Credentials': {
                    'CredentialPair': {
                        'Username': 'string',
                        'Password': 'string'
                    },
                    'SecretArn': 'string'
                }

That is my initial thought on the issue, let me know if that helps! I will also long some boto3 documentation below. Thank you!

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/quicksight/client/start_asset_bundle_import_job.html

It seems to work by deleting the authentication type. However I receive now some positional errors where I dont know how to solve them:

Response

{
  "statusCode": 500,
  "body": "\"An error occurred: Import job failed: [{'Arn': 'arn:aws:quicksight:eu-west-1:211125443023:datasource/39e9e5ac-47c1-4d42-b70c-33d467192cd6', 'Type': 'com.amazonaws.services.quicksight.model.InvalidParameterValueException', 'Message': 'DataSourceCredentials field is incorrectly set'}]\""
}

This is the datasource JSON now:

{
  "resourceType": "datasource",
  "dataSourceId": "39e9e5ac-47c1-4d42-b70c-33d467192cd6",
  "name": "Snowflake Int L",
  "type": "SNOWFLAKE",
  "dataSourceParameters": {
    "snowflakeParameters": {
      "host": "xxx",
      "database": "xxx",
      "warehouse": "xxx"
    }
  },
  "vpcConnectionProperties": {
    "vpcConnectionArn": "arn:aws:quicksight:eu-west-1:211125443023:vpcConnection/xxx"
  },
  "sslProperties": {
    "disableSsl": false
  },
  "Credentials": {
    "DataSourceCredentials": {
      "CredentialPair": {
        "Username": "xxx",
        "Password": "xxx"
      }
    }
  }
}

I tried it also with only

"DataSourceCredentials": {
      "CredentialPair": {
        "Username": "xxx",
        "Password": "xxx"
      }
  }

and

'Credentials': {
    'CredentialPair': {
        'Username': 'xxx',
        'Password': 'xxx'
    },
2 Likes

Hello @kosta, okay, it at least seems like we are on the right track here. One thing I am curious about, are you using any kind of VPC to connect Snowflake to QuickSight? Maybe that is what we should be utilizing instead of the Credentials. Also, you may just need to add the Snowflake Parameters portion again within the Credentials parameter. This is how it looks in the documentation:

Credentials={
        'CredentialPair': {
            'Username': 'string',
            'Password': 'string',
            'AlternateDataSourceParameters': [
				'SnowflakeParameters': {
						'Host': 'string',
                        'Database': 'string',
                        'Warehouse': 'string'
                    },
				]
			}
		}

This might be how it references the credentials for that database connection. Let me know if this helps!

1 Like

Hello @kosta, since we have not heard back from you, I will mark my previous response as the solution. Please let me know if you still need assistance with this issue and I can help guide you further. Thank you!

@kosta @DylanM My customer ran into the same “DataSourceCredentials field is incorrectly set” error, but with Redshift as the data source.

Kosta, are you still experiencing this issue? Thankfully we were able to identify the root cause and solve it. You may want to verify whether our error and solution are applicable to you.

First of all, we should’ve read the StartAssetBundleImportJob documentation carefully.

The documentation clearly states, the parameter override is intended for the data source included in the asset bundle :sweat_smile:

DataSources → (list)
A list of overrides for any DataSource resources that are present in the asset bundle that is imported.

Initially we used the target data source Id in our parameter override, which was incorrect. That was how we ran into the error. So the way we solved it by using the data source Id in the asset bundle (GUID) in our parameter override instead.

Good luck!