How can i create a dataset using boto3 python

Q&A
, , ,
1

I am trying to create a dataset with boto3/python and i am having some issues.

that’s my code:

import boto3
import uuid

quicksight_client = boto3.client(‘quicksight’, region_name=‘your-region’)

aws_account_id = ‘your-aws-account-id’
dataset_name = str(uuid.uuid4())
athena_data_source_arn = ‘arn:aws:quicksight:your-region:your-aws-account-id:datasource/your-datasource-id’

athena_database = “your-athena-database”
athena_table_name = “your-athena-table”

df = wr.catalog.table(database=athena_database, table=athena_table_name)

input_columns = [
{‘Name’: row[‘Column Name’], ‘Type’: row[‘Type’].upper()} for index, row in df.iterrows()
]

type_mapping = {
‘BIGINT’: ‘INTEGER’,
‘DOUBLE’: ‘DECIMAL’,
‘STRING’: ‘STRING’,
‘TIMESTAMP’: ‘DATETIME’,
‘DATE’: ‘DATETIME’,
‘INT’: ‘INTEGER’
}

for column in input_columns:
column[‘Type’] = type_mapping.get(column[‘Type’], column[‘Type’])

physical_table_id = str(uuid.uuid4())
logical_table_id = str(uuid.uuid4())

try:
response = quicksight_client.create_data_set(
AwsAccountId=aws_account_id,
DataSetId=str(uuid.uuid4()),
Name=dataset_name,
PhysicalTableMap={
physical_table_id: {
‘CustomSql’: {
‘DataSourceArn’: athena_data_source_arn,
‘Name’: dataset_name,
‘SqlQuery’: f’select * from {athena_database}.{athena_table_name}‘,
‘Columns’: input_columns
}
}
},
LogicalTableMap={
logical_table_id: {
‘Alias’: dataset_name,
‘DataTransforms’: [
{
‘ProjectOperation’: {
‘ProjectedColumns’: [column[‘Name’] for column in input_columns]
}
}
],
‘Source’: {‘PhysicalTableId’: physical_table_id},
},
},
Permissions=[
{
‘Principal’: f’arn:aws:quicksight:{aws_region}:{aws_account_id}:user/default/YourUsername’,
‘Actions’: [‘quicksight:DeleteDataSet’,
‘quicksight:UpdateDataSetPermissions’,
‘quicksight:PutDataSetRefreshProperties’,
‘quicksight:CreateRefreshSchedule’,
‘quicksight:CancelIngestion’,
‘quicksight:PassDataSet’,
‘quicksight:DeleteRefreshSchedule’,
‘quicksight:ListRefreshSchedules’,
‘quicksight:UpdateRefreshSchedule’,
‘quicksight:DescribeDataSetRefreshProperties’,
‘quicksight:DescribeDataSet’,
‘quicksight:CreateIngestion’,
‘quicksight:DescribeRefreshSchedule’,
‘quicksight:ListIngestions’,
‘quicksight:DescribeDataSetPermissions’,
‘quicksight:UpdateDataSet’,
‘quicksight:DeleteDataSetRefreshProperties’,
‘quicksight:DescribeIngestion’
]
},
],
ImportMode=‘DIRECT_QUERY’
)

print("Dataset creado correctamente:", response['Arn'])

except quicksight_client.exceptions.ResourceExistsException as e:
print(“El dataset ya existe. Error:”, e)
except Exception as e:
print(“Error al crear el dataset:”, e)

The creation is succesfull but after when i go to the quicksight IU and trying to see any data on an analysis i have the following error.

image
image1016×625 30.7 KB

I have checked the following documentation: create_data_set - Boto3 1.35.41 documentation

and I have compare with other dataset created manually as well with the method describe_data_set.

Kind regards.

Pablo

2

Hello @pablo,

Welcome to the Quick Sight Community, it is great having you here!

Have you tried creating the same dataset with the same configuration from the console? Did it work correctly?

3

Hi @pablo,
It’s been awhile since we last heard from you, did you have any additional questions regarding your post?

If we do not hear back within the next 3 business days, I’ll go ahead and close out this topic.

Thank you!

4

Hi @pablo,
Since we haven’t heard back, I’ll close out this topic. However, if you have any additional questions, feel free to create a new post in the community and link this discussion for relevant information if needed.

Thank you!