Bland-Altman Plot

September 2024

The Bland-Altman plot is one of the new techniques in the newly released SPC for Excel Version 7!  For more information, please select this link.

Sometimes, you need to compare two measurement systems.  You might be seeing if a new measurement system gets the same results as the existing measurement system.  Or maybe you are comparing  your measurement system to a supplier’s measurement system.  The Bland-Altman plot is one way of doing this.

The Bland-Altman plot is used to visualize differences between two measurement systems.  It is one chart that allows you see the average difference between the two measurement systems and the amount of variation in the differences.  It is introduced in this publication.

In this issue:

• Introduction
• Example Data
• Bland-Altman Calculations
• Bland-Altman Plot
• Bland-Altman Plot Interpretation
• Summary
• Quick Links

Please feel free to leave a comment below.  You can download a pdf copy of this publication at this link.

Introduction

When comparing two measurement systems, you want to know if the two give the “same” results. What does the “same” mean?  “Same” applies to two things: the average and the variation.  The Bland-Altman plot is designed to determine if two measurement systems can be used interchangeably.  In a Bland-Altman plot, the differences between paired measurements are plotted against their averages.  The bias between the two measurement systems is also determined as well as the 95% confidence limits. All this is put in the Bland-Altman plot to give you information on how much agreement there is between the two measurement systems. 

Example Data

Suppose you want to compare two test methods: A and B.  The Bland-Altman technique involves paired samples measurements.  You take the same sample or part and measure it in test method A and then in test method B.  It is best to have at least 25 to 30 samples.  The data for the 25 samples  are shown in Table 1.

Table 1: Paired Sample Measurements

Sample 1 was measured in test method A with a result of 25.  It was also measured in test method B with a result of 24. Each of the 25 samples is measured in both test methods.

Bland-Altman Calculations

The metric we are interested in is the difference between the two results. That is the measure of how much the two test methods are alike or different.

Calculation 1: Calculate the difference (D_i) between test method A and test method B for each sample.

D_i = Test Method A result for sample i  – Test Method B result for sample i

For sample 1, D₁ = 25 – 24 = 1; for sample 2, D₂ = 24 – 22 = 2; for sample 3, D₃ = 20 – 21 = -1 and so forth. If the two methods are similar, you would expect about half the differences to be positive and half to be negative when the difference is not zero.

Calculation 2: Calculate the average (D_i) between test method A and test method B for each sample. For sample 1,

D₁ = (Test Method A result – Test Method B result)/2 = (25+24)/2 = 24.5

The rest of  the averages for the other samples are calculated the same way.

Calculation 3: Calculate the average difference (D)

D = SD_i/k

where k is the number of samples.  So,

D = SD_i/k = 10/25 = 0.4

The average difference, also called bias, is 0.4.  Test method A tends to give slightly higher results than test method B.

If the two test methods were identical, you would expect the difference to be 0.  Of course, it is not the result in most cases.  Our old friend variation is present.  The question you have to ask and answer in this case is the following:

Is the average difference really different from 0?

You answer this question, not statistically, but practically.  Does the difference of 0.4 make a difference to me?  More on this later.

Calculation 4: Calculate the standard deviation of the differences (s_D).

The easiest way to calculate the standard deviation of the differences is to use the Excel function STDEV.  Doing that gives the following:

s_D = 1.19

Calculation 5: Construct the 95% confidence interval for the differences (also called the Limits of Agreement)

The equations for the 95% confidence interval are:

Upper Confidence Limit  =  D + 1.96 s_D = 0.4 +1.96(1.19) = 2.73

Lower Confidence Limit  = D – 1.96 s_D = 0.4 – 1.96(1.19) = -1.93

The Limits of Agreement represents the range within which you would expect the differences between the two measurement methods to be. 

Bland-Altman Plot

We now have enough information to create the Bland-Altman plot.  Table 2 is Table 1 with the sample differences and averages added.

Table 2: Paired Sample Results

The Bland-Altman plot is shown in Figure 1.

Figure 1: Bland-Altman Plot

The difference in each pair of results is plotted against the average for each pair of results. For sample 1, the difference is 1 and the average is 24.5.  Those two results are plotted as a point on the Bland-Altman plot.  The average difference (0.4) is plotted as well as the 0 line.  The Limits of Agreement are then added.  These are shown as the red dotted lines in Figure 1. 

Now that we have the Bland-Altman plot, what does it mean?

Bland-Altman Plot Interpretation

The Bland-Altman plot provides a visual picture of how two measurement systems compare.  Here are some things to consider when interpreting the Bland-Altman plot.

First, consider the bias.  How big is this?  The bias is the average difference.  Is the difference large enough to be considered important?  Note this is not a statistical question. It is a question you have to consider in terms of the process.  Would using one test method over the other make a difference in your process?

Next consider the Limits of Agreement. If the interval is narrow, then the two test methods are close in their measurements.  This assumes that the bias is small enough in your view.   If they are wide, then the two test methods have less agreement.  Again, this is not a statistical test.  It is based on your knowledge of the process.   These limits tell you how much the two test methods can differ while still being considered the “same.”

Since the differences are plotted all together, it is easier to detect possible outliers that indicate a problem with one of the test methods for that point. Don’t assume that points outside the confidence limits are outliers.  That is not necessarily true.

Does there seem to be a trend in the chart?  If there is, this means that the variation in the results gets larger or smaller as the average sample difference increases. Then one test method is less reliable over the range of differences.  Figure 1 does not show any trends.

Summary

This publication introduced the Bland-Altman plot. This plot provides a visual way to compare two measurement systems.  Paired samples are used.  The chart plots the difference in a paired sample versus the average of the paired sample. This gives you an idea of how much variation there is between the two measurement systems.  The average difference is also plotted.  This gives you an idea of the bias between the two measurement systems.  The Limits of Agreement around the differences is also added to the plot. 

The Bland-Altman plot does not tell you if something is “statistically” significant.  The user has to make decisions if the bias or the amount of variation is large enough to conclude that the two measurements are not the same and cannot be used interchangeably.