Time series correlation (pearson) heatmap

Question & Answer
1

Dear Community,

I am loading a Glue Table into a QuickSight dashboard with data from production, so the table grows all the time. I have columns in the Glue table like “feature” and “feature_value”, whereas “feature” holds different names of signal names and “feature_value” the corresponding signal values. In addition, there is of course something like a “time” column as well.

I designed my dashboard in a way, that I can easily adjust the time range of interest for the signals (line chart) to show. Via filtering (multi-select), I can also show multiple signals next to each other.

How can I easily add a dynamic (according to the time range and number of features selected) correlation matrix/heatmap (like the pearson correlation coefficient; analogous to pandas.DataFrame.corr) to the dashboard?

Thanks for your help!

Regards,
Johannes

2

@Johannes_Munk did you try the heatmap visual available in QuickSight, does that help you meet your requirement.

3

Of course I tried that initially. However, did not manage by myself to meet my requirements therewith.

4

can share screenshots showing why it cannot meet requirement?

5

also, sharing a blog post on May2023 with updates of Scatter Plot features to show correlation between 2 numeric values New scatter plot options in Amazon QuickSight to visualize your data | AWS Big Data Blog

6

Is a screen-sharing session also possible? I am struggling actually already with the starting point. How to compute the perason correlation coefficient in the first place in QuickSight, secondly dynamically (due to time range set and number of paramters chosen) and putting this together to display as heatmap. To my opinion, this should come out-of-the box in QuickSight. If you think you might help me I’d highly appreciate having a 30min session together. Thanks for your reply.

7

Hi @Johannes_Munk ,

From QuickSight, there is no function directly available to calculate pearson correlation coefficient . I will mark this a feature request.

Possibly if you can explain with the sample data and the expected output, we can check if there is a way to use available functions in QuickSight to build this. We will reach out separately to get some understanding of your requirement.

Kind regards,
Koushik

8