Folks,
With the advent of modern imaging devices it is now possible to measure the transfer efficiency (TE) of thousands of SMT solder paste stencil prints. This situation makes the concept of being statistically significant often meaningless. Let me explain by example.
Let’s say 30 years or so ago someone measured the TE of 25 prints for solder paste 1 and obtained an average of 90% with a standard deviation of 10%. Solder paste 2, also with 25 prints, gave her a TE average of 95% with a standard deviation of 15%. Using software like Minitab®, it is easy to show that these two sets of data are not statistically significantly different, at a 95% confidence level. This situation is easy to see visually by plotting the confidence interval of the mean (CIM). The two are only statistically different if the confidence intervals do not overlap, which as seen in Figure 1 is not the case.
Figure 1. Since the CIMs overlap the means are not statistically significantly different.
Today, the experiment would likely be run with 5,000 prints. Assume, in this case, that solder paste 1 has a mean TE of 94.5% with a standard deviation of 10% and solder paste 2 has a mean of 95% and a standard deviation of 15%. Even with this small difference of only 0.5%, the difference is statistically significant. This discovery is supported by plotting the CIMs as seen in Figure 2. They do not overlap. The reason for this dramatic change is that the standard error of the mean (SEM) goes as one over the square root of the sample size. As the sample size gets large, the SEM gets smaller and the CIM gets tighter.
Figure 2. With a sample size of 5,000, even a difference in the means of only 0.5% is statistically significant.
This raises the question, is the difference meaningful? Probably not. With a large enough sample size any difference will become statistically significant. In these cases we need to ask ourselves, "Is it practically significant?" Only we can define what it means to be "practically significant." I think most would agree that a 0.5% difference is not practically significant. This situation would be especially true if solder paste 1 had other properties that were much better than solder paste 2.
So, always ask yourself, "If a difference is statistically significant, is it also practically significant?"
Cheers,
Dr. Ron