Getting CONTEXTUAL_SYNTAX_ERROR

Hi Experts,

We are getting CONTEXTUAL_SYNTAX_ERROR in two calculated fields while migrating the dashboard. Have cross checked and the same fields do not throw any error at analyses/dashboard level and also giving correct data. The definition of both fields are as below :-

  1. Avg Entity True Positive Indicator - distinct_count(ifelse({Entity True Positive Indicator_Alias}=‘True Positive’,{entity_id},NULL))/sumOver(distinct_count({entity_id}),[{Entity Number Of Alerts_Dim}])

  2. True Positive Alert Ids - ifelse({Alert True Positive Indicator}=1,{Original Alert Id},NULL)

Here {Entity True Positive Indicator_Alias} = ifelse({Entity True Positive Indicator}=0,‘False Positive’,‘True Positive’)

AND

{Entity True Positive Indicator} = “ifelse(
${TruePositiveMeasure} = ‘At least one alert is “Issue”’,
ifelse(
{Entity Number of Issue Alerts} > 0,
1,
0
),
${TruePositiveMeasure} = ‘At least one alert added to Case’,
ifelse(
{Entity Number of Alerts in Case} > 0,
1,
0
),
${TruePositiveMeasure} = ‘At least one form filed’,
ifelse(
{Entity Number of Alerts with Form Filed} > 0,
1,
0
),
${TruePositiveMeasure} = ‘4’,
ifelse(
{Entity Number of Alerts} = 0,
0,
ifelse(
{Entity Number of Issue Alerts} / {Entity Number of Alerts} >= ${MinimumAlertPerc},
1,
0
)
),
${TruePositiveMeasure} = ‘5’,
ifelse(
{Entity Number of Alerts} = 0,
0,
ifelse(
{Entity Number of Alerts in Case} / {Entity Number of Alerts} >= ${MinimumAlertPerc},
1,
0
)
),
${TruePositiveMeasure} = ‘6’,
ifelse(
{Entity Number of Alerts} = 0,
0,
ifelse(
{Entity Number of Alerts with Form Filed} / {Entity Number of Alerts} >= ${MinimumAlertPerc},
1,
0
)
),
0
)”

where ${TruePositiveMeasure} is a static defined parameter with values At least one alert is “Issue”, At least one alert added to Case & At least one form filed, {Entity Number of Issue Alerts} , {Entity Number of Alerts in Case}, {Entity Number of Alerts with Form Filed} and {Entity Number Of Alerts} where {Entity Number of Issue Alerts} has formula sumOver({alert_is_issue}, [{entity_id}],PRE_FILTER). Here ${MinimumAlertPerc} is a integer parameter with static value 60.

For True Positive Alert Ids field {Alert True Positive Indicator} = "ifelse(
${TruePositiveMeasure} = ‘At least one alert is “Issue”’, ifelse(
${ExcludeNonAMLReasonCodesfromTPNo}=‘Yes’ AND coalesce({reason_code_is_aml}, ‘No’) = ‘No’, 0, {alert_is_issue}),

${TruePositiveMeasure} = 'At least one alert added to Case', ifelse(
    ${ExcludeNonAMLReasonCodesfromTPNo}='Yes' AND coalesce({reason_code_is_aml}, 'No') = 'No', 0, {alert_is_in_case}),

${TruePositiveMeasure} = 'At least one form filed', ifelse(
    ${ExcludeNonAMLReasonCodesfromTPNo}='Yes' AND coalesce({reason_code_is_aml}, 'No') = 'No', 0, {form_filed_for_alert}),
NULL

)"

where ${ExcludeNonAMLReasonCodesfromTPNo} is a static string parameter with values Yes & No and {reason_code_is_aml}, {alert_is_issue}, {alert_is_in_case} & {form_filed_for_alert} are string fields coming from source.

{Original Alert Id} = ifelse({number_of_alerts}=0,NULL,{alert_id}) where {number_of_alerts} is a direct measure field and {alert_id} is a direct dimension field.

Hello @arpitarorait3016, are the calculated fields causing the analysis to fail when you migrate? Or is it migrating the analysis but then giving you errors when you try to utilize those 2 calculated fields. That is interesting that is it throwing an error, but some of the calculations can be error prone during migrations.

Also, how are you migrating? Are you using the analysis template or are you moving the analysis JSON definition? With a little more information, I can help resolve the issue.

1 Like

Hi Dylan,

Below is the latest error we are getting when migrating the dashboard post we made some changes in analyses :-

Type: COLUMN_NOT_FOUND,Message: Field id 'c55deabe-cb59-431a-a2f3-f756d20277da.alert_finding.1.1730283709071' in DataBarsOptions not found in field wells,ViolatedEntities: [{Path: sheet/d22b553f-dc87-4263-82bb-2f8941ee419c_b2e742e9-818d-4940-807d-4be71d54e030/visual/d22b553f-dc87-4263-82bb-2f8941ee419c_b95b7fe1-e60b-4dc1-867f-fe3ede80ea53/field/c55deabe-cb59-431a-a2f3-f756d20277da.alert_finding.1.1730283709071}]}, {Type: COLUMN_NOT_FOUND,Message: Field id 'c55deabe-cb59-431a-a2f3-f756d20277da.alert_finding.1.1730283709071' in DataBarsOptions not found in field wells,ViolatedEntities: [{Path: sheet/d22b553f-dc87-4263-82bb-2f8941ee419c_1958c122-ceb7-47ad-8f27-61df583ccaa1/visual/d22b553f-dc87-4263-82bb-2f8941ee419c_9f451ed2-2453-4d2b-bc00-f6cd127b1bd2/field/c55deabe-cb59-431a-a2f3-f756d20277da.alert_finding.1.1730283709071

Here “alert_finding” is a direct field coming from the database and is present in the analyses/dashboard and also giving correct results but when migrating then this filed is throwing error.

For your earlier question we are migrating the dashboard through analysis JSON definition.

1 Like

Hello @arpitarorait3016, in the past, I have found that migrating the analysis definition directly can be somewhat error prone. It can be difficult to find and debug the errors as well, since the source analysis is working as expected.

I would highly recommend trying to utilize the analysis template and migrating that to see if it resolves the errors you are seeing. Also, in regards to the dataset issue, did you make sure that you included the PhysicalTableMap and the LogicalTableMap sections from the source dataset? Without the LogicalTableMap some of the datatypes linked to the fields in the source will be ignored.

Below, I will link some documentation on the CreateTemplate API:

Let me know if this helps!

1 Like

Hi Dylan,

I checked the json files of both datasets extracted through describe-data-set and can see the details of LogicalTableMap for both but in both the files i am not able to see details of calculated fields and i am not sure if this is correct behavior because in one of my dashboard which we are able to migrate without any issue there also the calculated fields are not present under LogicalTableMap for one of the dataset.
I will pass on your suggestions to our migration team and will update you further, thank you!

1 Like

Hi Dylan,

Below is the latest error message we are getting when trying to migrate the dashboard :-

Traceback (most recent call last): File “opr_qs_get_publish_requestid.py”, line 53, in raise Exception(‘Error occurred while publishing dashboard.’, response.content)Exception: (‘Error occurred while publishing dashboard.’, b’{“statusCode”: 500, “status”: “FAILED”, “status_details”: [{“tenantId”: “11ef5617-b255-35a0-93d4-0242ac110003”, “JobStatus”: “FAILED_ROLLBACK_COMPLETED”, “Errors”: [{“Arn”: “arn:aws:quicksight:us-east-1:694164520908:dashboard/11ef5617-b255-35a0-93d4-0242ac110003_AMLAlertPerformanceReport”, “Message”: “[{Type: INVALID_CALCULATED_COLUMN_EXPRESSION,Message: Calculated field Entity True Positive Indicator_Alias failed with error code CONTEXTUAL_SYNTAX_ERROR during validation,ViolatedEntities: [{Path: calculated-field/Entity True Positive Indicator_Alias}]}, {Type: INVALID_CALCULATED_COLUMN_EXPRESSION,Message: Calculated field False Positive Alert Id failed with error code CONTEXTUAL_SYNTAX_ERROR during validation,ViolatedEntities: [{Path: calculated-field/False Positive Alert Id}]}]”, “Type”: “com.amazonaws.services.quicksight.model.DashboardError”}]}], “request_id”: “f41fcc04-6665-4cb2-9c75-6a9557088114”, “dashboardName”: “AMLAlertPerformanceReport”, “solution”: “AW_AML”, “version”: “1.0.0rc8”}')

We have cross checked and there are no issues with the calculated fields which are mentioned in the error message, thank you!

1 Like

Hello @arpitarorait3016, I would check the LogicalTableMap and PhysicalTableMap to make sure that the fields you are utilizing in the calculated fields in the analysis are set to the correct datatypes. If something doesn’t match between the source and the destination JSONs, that could cause the calculated fields to error.

Is there a reason you do not want to use the analysis template rather than the definition JSON? Through my experience, the template is significantly easier to migrate. There is very little useful information in the error codes that QuickSight provides when the analysis creation fails.

Hi Dylan,

We have cross checked and the fields type are correct under PhysicalTableMap, the migration team is only editing the dataset id and Account id in ARN and then trying to migrate the dashboard. The framework is set to migrate through JSON only and hence they cannot migrate through analysis template, thank you!

1 Like

Hi Dylan,

We made some changes as per suggestion from AWS and post that we are not getting CONTEXTUAL_SYNTAX_ERROR but now while migrating the dashboard we are getting COLUMN_NOT_FOUND error although the mentioned column is present in the analysis as we have checked through analysis definition. Below is the exact error message we are getting :-

Exception: (‘Error occurred while publishing dashboard.’, b’{“statusCode”: 500, “status”: “FAILED”, “status_details”: [{“tenantId”: “xyz”, “JobStatus”: “FAILED_ROLLBACK_COMPLETED”, “Errors”: [{“Arn”: “abc”, “Message”: “[{Type: COLUMN_NOT_FOUND,Message: Field id 'def' in DataBarsOptions not found in field wells,ViolatedEntities: [{Path: sheet/ghi}]}, {Type: COLUMN_NOT_FOUND,Message: Field id 'def' in DataBarsOptions not found in field wells,ViolatedEntities: [{Path: sheet/ghi}]}]”, “Type”: “com.amazonaws.services.quicksight.model.DashboardError”}]}], “request_id”: “jkl”, “dashboardName”: “mno”, “solution”: “AW_AML”, “version”: “1.0.0rc12”}')

Hello @arpitarorait3016, is it possible that either the name of the column has changed (case sensitivity matters here) or that the datatype of the field is different? Some datatypes can only be managed from the LogicalTable portion of the dataset, so it is possible that the field is not exactly what is expected from the analysis.

I remember migrating a dataset in the past this way, and the LogicalTable wasn’t included when I ran the describe-data-set command. I had geospatial datatype fields that were being imported as a string due to this issue. That caused visuals to break in my analysis since those visuals required a geospatial field. This could be causing the error you are experiencing as well. Let me know if that helps!

Hi Dylan,

Thank you for all your help but the issue was in two tables we were using as legends in the dashboard, there we included some settings for Data bars which was throwing error during migration, post removing these setting we were able to successfully migrate the dashboard.

2 Likes