You can use the following syntax to calculate the correlation between two columns in a pandas DataFrame:
df['column1'].corr(df['column2'])
The following examples show how to use this syntax in practice.
Example 1: Calculate Correlation Between Two Columns
The following code shows how to calculate the correlation between columns in a pandas DataFrame:
import pandas as pd #create DataFrame df = pd.DataFrame({'points': [25, 12, 15, 14, 19, 23, 25, 29], 'assists': [5, 7, 7, 9, 12, 9, 9, 4], 'rebounds': [11, 8, 10, 6, 6, 5, 9, 12]}) #view first five rows of DataFrame df.head() points assists rebounds 0 25 5 11 1 12 7 8 2 15 7 10 3 14 9 6 4 19 12 6 #calculate correlation between points and assists df['points'].corr(df['assists']) -0.359384
The correlation coefficient is -0.359. Since this correlation is negative, it tells us that points and assists are negatively correlated.
In other words, as values in the points column increase, the values in the assists column tend to decrease.
Example 2: Calculate Significance of Correlation
To determine whether or not a correlation coefficient is statistically significant, you can use the pearsonr(x, y) function from the SciPy library.
The following code shows how to use this function in practice:
import pandas as pd from scipy.stats import pearsonr #create DataFrame df = pd.DataFrame({'points': [25, 12, 15, 14, 19, 23, 25, 29], 'assists': [5, 7, 7, 9, 12, 9, 9, 4], 'rebounds': [11, 8, 10, 6, 6, 5, 9, 12]}) #calculate p-value of correlation coefficient between points and assists pearsonr(df['points'], df['assists']) (-0.359384, 0.38192)
The first value in the output displays the correlation coefficient (-0.359384) and the second value displays the p-value (0.38192) associated with this correlation coefficient.
Since the p-value is not less than α = 0.05, we would conclude that the correlation between points and assists is not statistically significant.
Additional Resources
How to Calculate Spearman Rank Correlation in Python
How to Calculate Partial Correlation in Python
How to Calculate Cross Correlation in Python