Systematic and Statistical Errors*

In any measurement the numerical value that you obtain from your instrument is always somewhat different from the true value of the physical quantity. Your goal is the come to term with that fact of life, and characterize the numerical value accordingly: estimate the error of your measurement. Here are a few concepts related to this issue:

 

The difference between statistical and systematic errors is illustrated in an example here.  Assume you aim arrows at the bull's eye.  You may find the magnitude of statistical and systematic errors by looking at the distribution of the shots:

Statistical or Random Error

Calculating/estimating the statistical error is an important part of your job in the lab.  How do we know if  random errors dominate the measurement?  Consider, for example, measuring the time required for a weight to fall to the floor.  A random error may occur when an experimenter attempts to push a button that starts a timer simultaneously with the release of the weight.  To decide if the error is truly random, we have to repeat the measurement many times, just like in the example above we had to shoot many times (say N = 200 times).  Then we look at the measured times, and we find the smallest and the largest (say 0.44s and 0.76s).  We divide the time in between to equal intervals, called "bins".  In our example, the first bin, t1 is for times between  0.44s and 0.46s, the next one, t2, is from 0.46s to 0.48s and so on.  We then count how many of our measured times fall into a given bin - call these numbers ni.  Finally we plot the ti bins on the horizontal axis and the  ni numbers on the vertical axis.  The curve we obtain is a histogram (grades in large classes are also assigned by histogram).  The error is random if our histogram looks like an ideal bell-shaped curve (called a Gaussian distribution).  Of course, to do this we need a really large number of measurements.

The best estimate of the true fall time t is the mean value (or average value):

taver

If the experimenter squares each deviation from the mean, averages the squares, and takes the square root of that average, the result is a quantity called the "root-mean-square" or the "standard deviation" sigma of the distribution. It measures the random error or the statistical uncertainty of the individual measurement ti:

stddev

About two thirds of all the measurements have a deviation less than one sigma from the mean and 95% of all measurements are within two sigma of the mean. In accord with our intuition that the uncertainty of the mean should be smaller than the uncertainty of any single measurement, measurement theory shows that in the case of random errors the standard deviation of the mean sigmam- is given by:

sigmam

where N again is the number of measurements used to determine the mean. Then the result of the N measurements of the fall time would be quoted tres.

Whenever you make a measurement that is repeated N > 20 times, you are supposed to calculate the mean value and its standard deviation as just described. Needless to say, the procedure can get very tedious.  Many times we take a shortcut, and use a simplified  prescription for estimating the random error. Assume you have measured the fall time about ten times. In this case it is reasonable to assume that the largest measurement tmax is approximately +2sigma from the mean, and the smallest tmin is -2sigma from the mean. Hence:

sigmakort

is a reasonable estimate of the uncertainty in a single measurement. The above method of determining sigma is a rule of thumb if you make of order ten individual measurements (i.e. more than 4 and less than 20).

Systematic Errors

Systematic errors result when characteristics of the system we are examining, or the instruments we use are different from what we assume them to be. Your watch being off by five minutes may cause systematic error.  If a meter stick we are using expanded over time by 5%, then every reading we record will have a systematic error of 5%.  The experimenter causes systematic error if he/she is pushing  the timer accurately at the end of the measurement, but he/she is always late at the beginning .

Clearly, taking the average of many readings will not help us to reduce the size of this systematic error. If we knew the size and direction of the systematic error we could correct for it and thus eliminate its effects completely.

Instrumental Precision

To report that the time is about 3 PM is less precise than to say the time is 3:02:45. If your watch displays hours only, you are limited by instrumental precision.  But being more precise does not always mean being more accurate - your watch may show seconds, but it could be off by five minutes.

The error due to instrumental precision is a systematic error and cannot be improved by repeating the measurement many times. For example, assume you are supposed to measure the length of an object. The precision will be given by the spacing of the tick marks on the measurement apparatus (the meter stick). You can read off whether the length of the object lines up with a tick mark or falls in between two tick marks, but you could not determine the value to a precision of l/10 of a tick mark distance. Typically, the error of such a measurement is equal to one half of the smallest subdivision given on the measuring device. So, if you have a meter stick with tick marks every mm (millimeter), you can measure a length with it to an accuracy of about 0.5 mm.

While in principle you could repeat the measurement numerous times, this would not improve the accuracy of your measurement.  This assumes, of course, that you have not been sloppy in your measurement but made a careful attempt to line up one end of the object with the zero of the meter stick as accurately as you can, and that you read off the other end of the meter stick with the same care. If you want to judge how careful you have been, it would be useful to ask your lab partner to make the same measurements, using the same meter stick, and then compare the results.

Accuracy

If you measure, for example, the gravitational acceleration, and you compare your value to the textbook number, you may conclude that your accuracy was "within 30% of the accepted value".  Under no circumstance can this comparison replace a true error estimate.  The error you report should be calculated from the errors of the time and length you measured.

Any discrepancy between a generally accepted value and your measurement is NOT AN ERROR but either an indication that you have not fully understood all sources of error in your measurement, or that you made a new discovery!  You can always calculate (estimate) the error.  But if you do not know the true value, it is impossible to determine to what extent is your measurement accurate.

    Propagation of Errors

Even simple experiments usually call for the measurement of more than one quantity. The experimenter inserts these measured values into a formula to compute a desired result. He/she will want to know the uncertainty of the result. Here, we list several common situations in which error propagation is simple, and at the end we indicate a general procedure. If you are faced with a complex situation, ask your lab instructor for help.

Additive Formulae

When a result R is calculated from two measurements x and y, with uncertainties x and y, and two constants a and b with the additive formula:

R = ax + by ,

and if the errors in x and y are independent, then the error in the result R will be:

(∆R)2 = (a x)2 + (b y)2 .

The reason why we should use this quadratic form and not simply add the uncertainties ax and by, is that we don't know whether x and y were both measured too large or too small; indeed the measurement errors on x and y might cancel each other in the result R! Independent errors cancel each other with some probability (say you have measured x somewhat too big and y somewhat too small; the error in R might be small in this case). This partial statistical cancellation is correctly accounted for by adding the uncertainties quadraticallyNote: a and b can be positive or negative, i.e. the equation works for both addition and subtraction.

Multiplicative Formulae

When the result R is calculated by multiplying a constant a times a measurement of x times a measurement of y (or divided by y), i.e.:

R = axy     or    R = ax/y,

then the relative errors x/x and y/y add quadratically:

(R/R)2 = (x/x)2 + (y/y)2 .

Example: Say quantity x is measured to be 1.00, with an uncertainty x = 0.10, and quantity y is measured to be 1.50 with uncertainty y = 0.30, and the constant a = 5.00 . The result R is obtained as R = 5.00 x1.00 x l.50 = 7.5 . The relative uncertainty in x is x/x = 0.10 or 10%, whereas the relative uncertainty in y is y/y = 0.20 or 20%. Therefore the relative error in the result is dr/r or 22%,. The absolute uncertainty of the result R is obtained by multiplying 0.22 with the value of R: R = 0.22 x 7.50 = 1.7 .

More Complicated Formulae

If your result is obtained using a more complicated formula, as for example:

R = a x2 siny ,

there is a very easy way to find out how your result R is affected by errors x and y in x and y. Insert into the equation for R, instead of the value of x, the value x+x, and find how much R changes:

R + Rx = a (x+x)2 siny .

If y has no error you are done. If y has an error as well, do the same as you just did for x, i.e. insert into the equation for R the value for y+y instead of y, to obtain the error contribution Ry. The total error of the result R is again obtained by adding the errors due to x and y quadratically:

(R)2 = (Rx)2 + (Ry)2 .

This way to determine the error always works and you could use it also for simple additive or multiplicative formulae as discussed earlier. Also, if the result R depends on yet another variable z, simply extend the formulae above with a third term dependent on Dz.

Fitting a Straight Line through a Series of Points

Frequently  you perform a series of measurements of a quantity y at different values of x, and when you plot the measured values of y versus x you observe a linear relationship of the type y = ax + b.  Your task is now to determine, from the errors in x and y, the uncertainty in the measured slope a and the intercept b. There is a mathematical procedure to do this, called "linear regression" or "least-squares fit". Such fits are typically implemented in spreadsheet programs and can be quite sophisticated. If you have no access or experience with spreadsheet programs, you want to instead use a simple, graphical method, briefly described in the following.
 
 

Plot the measured points (x,y) and mark for each point the errors x and y as bars that extend from the plotted point in the x and y directions. Draw the line that best describes the measured points (i.e. the line that minimizes the sum of the squared distances from the line to the points to be fitted; the least-squares line). This line will give you the best value for slope a and intercept b. Next, draw the steepest and flattest straight lines, see the Figure, still consistent with the measured error bars. From these two lines you can obtain the largest and smallest values of a and b still consistent with the data, amin and bmin, amax and bmax. From their deviation from the best values you then determine, as indicated in the beginning, the uncertainties a and b.

    Significant Figures

In light of the above discussion of error analysis, discussions of significant figures (which you should have had in previous courses) can be seen to simply imply that an experimenter should quote digits which are appropriate to the uncertainty in his result. The above result of R = 7.5 ± 1.7 illustrates this. It would not be meaningful to quote R as 7.53142 since the error affects already the first figure. On the other hand, to state that R = 8 ± 2 is somewhat too casual. It is a good rule to give one more significant figure after the first figure affected by the error.



* http://solidstate physics.sunyb.edu/teach/phy132/lab_instructions

 


 


Information from http://solidstate.physics.sunysb.edu/teach/phy132/lab_instructions/