Correlation Analysis

April 2011

This month’s newsletter takes a look at correlation analysis. Correlational studies are done to look at the linear relationships between pairs of variables. There are basically three possible results from a correlation study: a positive correlation, a negative correlation or no correlation. These are described below. A scatter diagram is often used to visually see the relationship between the two variables. A scatter diagram matrix can be used to see the relationship between multiple pairs of variables. This newsletter will demonstrate how to calculate the correlation coefficient and determine the probability that it is statistically significant.

In this issue:

• Positive Correlation
• Negative Correlation
• No Correlation
• Correlational Relationships and Causal Relationships
• Example: Smoking and Lung Cancer
• Scatter Diagrams
• Pearson Correlation Coefficient
• When is R Significant?
• Impact of Outliers
• R Squared and R
• Final Cautions on Correlation  Analysis
• Quick Links

A word of caution to start with: a significant correlation between two variables does not mean that one is the cause of the other. This is the difference between a “correlational” relationship and a “causal” relationship.

Please feel free to leave comments at the end of the newsletter.

Positive Correlation

A positive correlation exists between variable X and variable Y if an increase in X results in an increase in Y (and vice-versa).

• The more cigarettes you smoke, the greater the chance of lung cancer.
• If you are paid by the hour, the more hours you work, the more pay you receive.
• The more time you spend studying, the better grades you make in school.

Negative Correlation

A negative correlation exists between variable X and variable Y if a decrease in X results in an increase in Y (and vice-versa).

• The heavier your car is, the lower your gas mileage is.
• The colder it is outside, the higher your heating bill.
• The more time you spend watching TV, the lower your grades are in school.

No Correlation

In this case, a change in X has no impact on Y (and vice-versa). There is no relationship between the two variables. For example, the amount of time I spend watching TV has no impact on your heating bill.

Correlational Relationships and Causal Relationships

It is important to remember that simply because there is a significant correlation between two variables, it does not mean that one is the cause of the other. Suppose we find a significant correlation between X and Y. We know the association exists, but not necessarily the direction of that association:

• Does a change in X cause a change in Y?
• Does a change in Y cause a change in X?

But, to complicate things even more, there may be a third, unmeasured variable (Z) that is actually the cause of the change.

• Does a change in Z cause a change in both X and Y?

Jeffery Ricker has a good discussion of this. I will summarize what he says here. A study was done in 2005 to determine the impact of where students sit in a classroom to the grade they received. The study indicated that students who sat up front received better grades than those who sat in the back of the room. The seats were assigned at random. A follow-up study in 2007 by others offered a different way to interpret the results. This is how Dr. Ricker explains it:

“In the correlation between classroom seat location and course grades, it could be that

• sitting in front (Variable A) causes students to get better grades (Variable B),
• getting better grades (Variable B) causes students to sit in front (Variable A), or
• being more motivated (an unmeasured third variable, C) causes students both to sit in front (Variable A) and get better grades (Variable B).”

The key point is that is impossible just from a correlation analysis to determine what causes what. You don’t know the cause and effect relationship between two variables simply because a correlation exists between them. You will need to do more analysis (such as designed experiments) to define the cause and effect relationship.

Example: Smoking and Lung Cancer

The example involves a study done in England to look for a correlation between smoking and lung cancer. Twenty-five occupational groups were included in the study. The X variable is the number of cigarettes smoked per day relative to the number smoked by all men of the same age. If X = 100, then the men smoked an average amount; if above 100, the men smoked more than average and if below 100, they smoked less than average.

The Y variable is the mortality ratio for deaths from lung cancer. Again, if Y = 100, then the morality ratio for deaths from lung cancer for that group was average; above 100 it was greater than average and below 100 it was less than average. The data are shown below. We will use this data to do a correlation analysis of smoking and mortality.

Scatter Diagram

It is always best to begin with a scatter diagram of the data to get a visual picture of the relationship. For more information on scatter diagrams, please see our February 2005 newsletter. A scatter diagram simply plots the Y variable versus the X variable. The scatter diagram for the smoking data is shown below.

The scatter diagram does appear to show a positive correlation. The mortality rate appears to increase as the smoking ratio increases. In addition to this visual picture, it would be nice to have a numerical measurement of this correlation. This is done by calculating the correlation coefficient.

Pearson Correlation Coefficient

The Pearson correlation coefficient (or just correlation coefficient) provides a measure of the correlation between two variables. The equation for the correlation coefficient is given below.

where X_i and Y_i are the individual paired data points, n is the total number of paired data points, Xbar is the average X value, Ybar is the average Y value, and s_x and s_y are the standard deviations of the X and Y values respectively.

If R is positive, then there is a positive correlation between X and Y. What makes R positive? Look at the numerator in the R equation:

If an X value is above average and the paired Y value is above average, then the numerator is positive. If an X value is below average and the paired Y value is below average, you would be multiplying two negative numbers – which again gives a positive number. So, in a positive correlation, large value of X tend to generate large values of Y and small values of X tend to generate small values of Y.

If R is negative, then there is a negative correlation between X and Y. What makes R negative? Again, look at the numerator in the R equation above. If an X value is below average and the paired Y value is above average, you will be multiplying a negative number by a positive number – which generates a negative number. If an X value is above average and the paired Y value is below average, you will be multiplying a positive number by a negative number – again generating a negative number. So, in a negative correlation, large values of X tend to generate small values of Y and vice-versa.

The calculation of the correlation coefficient is straightforward based on the equation above. The calculations are shown in the table below.

R can now be calculated:

The R value is positive indicating a positive correlation. This confirms what the scatter diagram shows.

When is R Significant?

You have your scatter diagram and the value of R. Is the correlation significant? In other words, does it mean anything to you in the real world? Some people give guidelines for interpreting R. For example:

The author does note that others may question his boundaries. They are somewhat arbitrary. It is much better to determine the “p value” to help you determine if the correlation is significant. A p value is the probability of getting something more extreme than your result, when there is no correlation between the two variables. The p value is calculated using the t distribution where:

You can then use the t distribution to find the probability associated this value of t for 23 degrees of freedom. In Excel, use the TDIST function. The value of p is then:

p = 0.000057

This is the probability that you would obtain a result more extreme than the 0.716 if there was no correlation between the two variables. Since this value is so small, you assume that there is evidence of a correlation between the two variables.

Many researches assume there is a statistically significant correlation if p <= 0.05. If the value of p>=0.20, there is no correlation. If the value falls between 0.05 and 0.20, more data are needed.

Impact of Outliers

Outliers in the data (special causes of variation) can sometimes mask the results. That is why it is important to construct a scatter diagram to look for outliers. You can also use a Box-Whisker chart to look for outliers (see our March 2007 newsletter). Suppose you have the data set below and you want to see if there is correlation between X and Y.

You could calculate the correlation coefficient and the p value. If you do that, you get the following results:

R = 0.116
p = 0.826

Since p is so large, you conclude there is no correlation between X and Y and go happily on your way. The scatter diagram for the two variables is shown below. Note the outlier in the upper right corner.

Since this is so far removed from the rest of the data, it may be an outlier. So, do you just toss the point out and redo the calculations? No, you should not just assume that is an outlier. You need to find out if there was a special cause of variation. If you find the reason for the outlier, you may toss the point out. If not, you probably need to consider taking more data. If you deleted the point from the calculations, the results are:

R = -0.962
p = 0.009

So, there is a strong, negative correlation. These data were just to demonstrate the impact of outliers. You normally would not do a correlation analysis with these few data points. You should always strive to have as much data as possible to do a correlation analysis.

R-Squared and R

You can multiply R by itself and get R-Squared. This is the term that is often used in regression analysis. It gives the % of variation in Y that is explained by the variation in X. In the lung cancer example, R-squared is 0.513. This means that 51.3% of the variation in lung cancer mortality is explained by the variation in the smoking ratio.

Final Cautions on Correlation Analysis

A couple of quick cautions on correlation analysis. A significant correlation does not prove causation. You will need to do more work to prove that in most cases. Correlation is looking only at linear relationship between the variables. And last, your analysis is valid only for the range of data you used. There may be a different result if you use a different range of data.