I am trying to create a dataset with boto3/python and i am having some issues.
that’s my code:
import boto3
import uuid
quicksight_client = boto3.client(‘quicksight’, region_name=‘your-region’)
aws_account_id = ‘your-aws-account-id’
dataset_name = str(uuid.uuid4())
athena_data_source_arn = ‘arn:aws:quicksight:your-region:your-aws-account-id:datasource/your-datasource-id’
athena_database = “your-athena-database”
athena_table_name = “your-athena-table”
df = wr.catalog.table(database=athena_database, table=athena_table_name)
input_columns = [
{‘Name’: row[‘Column Name’], ‘Type’: row[‘Type’].upper()} for index, row in df.iterrows()
]
type_mapping = {
‘BIGINT’: ‘INTEGER’,
‘DOUBLE’: ‘DECIMAL’,
‘STRING’: ‘STRING’,
‘TIMESTAMP’: ‘DATETIME’,
‘DATE’: ‘DATETIME’,
‘INT’: ‘INTEGER’
}
for column in input_columns:
column[‘Type’] = type_mapping.get(column[‘Type’], column[‘Type’])
physical_table_id = str(uuid.uuid4())
logical_table_id = str(uuid.uuid4())
try:
response = quicksight_client.create_data_set(
AwsAccountId=aws_account_id,
DataSetId=str(uuid.uuid4()),
Name=dataset_name,
PhysicalTableMap={
physical_table_id: {
‘CustomSql’: {
‘DataSourceArn’: athena_data_source_arn,
‘Name’: dataset_name,
‘SqlQuery’: f’select * from {athena_database}.{athena_table_name}‘,
‘Columns’: input_columns
}
}
},
LogicalTableMap={
logical_table_id: {
‘Alias’: dataset_name,
‘DataTransforms’: [
{
‘ProjectOperation’: {
‘ProjectedColumns’: [column[‘Name’] for column in input_columns]
}
}
],
‘Source’: {‘PhysicalTableId’: physical_table_id},
},
},
Permissions=[
{
‘Principal’: f’arn:aws:quicksight:{aws_region}:{aws_account_id}:user/default/YourUsername’,
‘Actions’: [‘quicksight:DeleteDataSet’,
‘quicksight:UpdateDataSetPermissions’,
‘quicksight:PutDataSetRefreshProperties’,
‘quicksight:CreateRefreshSchedule’,
‘quicksight:CancelIngestion’,
‘quicksight:PassDataSet’,
‘quicksight:DeleteRefreshSchedule’,
‘quicksight:ListRefreshSchedules’,
‘quicksight:UpdateRefreshSchedule’,
‘quicksight:DescribeDataSetRefreshProperties’,
‘quicksight:DescribeDataSet’,
‘quicksight:CreateIngestion’,
‘quicksight:DescribeRefreshSchedule’,
‘quicksight:ListIngestions’,
‘quicksight:DescribeDataSetPermissions’,
‘quicksight:UpdateDataSet’,
‘quicksight:DeleteDataSetRefreshProperties’,
‘quicksight:DescribeIngestion’
]
},
],
ImportMode=‘DIRECT_QUERY’
)
print("Dataset creado correctamente:", response['Arn'])
except quicksight_client.exceptions.ResourceExistsException as e:
print(“El dataset ya existe. Error:”, e)
except Exception as e:
print(“Error al crear el dataset:”, e)
The creation is succesfull but after when i go to the quicksight IU and trying to see any data on an analysis i have the following error.
I have checked the following documentation: create_data_set - Boto3 1.35.41 documentation
and I have compare with other dataset created manually as well with the method describe_data_set.
Kind regards.
Pablo