Hello I am working with data from 2 different AWS cluster , could anyone suggest the best way to merge the data so that we have a global point of reference?
I am still new to QS and AWS ecosystem but I thought to achieve this via AWS Glue DataBrew and then plug the data into QS , what do you think about this approach?
Thank you in advance for you help
I am working on AWS QS, Glue and Redshift(Data-warehouse) since last couple of years.
This is best model I am suggesting you two options
One with low data size and other with bit data.
Approach # 1
Model one is cost effective with small data size.
Write Lambda function in python to migrate data from two different account datasource to one place(S3).
You can use Lambda + S3 + Athena and QS
Note:- A finned tuned code required for data synch. A little bit challenge here
Approach # 2 with big data.
AWS Glue + Athena/Redshift and QS
This required VPC between your two region to transfer data from one region to other.
Little bit configuration but not complex.
You can contact if you need my support on it.