Initial errors like offset and gain errors can be significantly reduced or fully eliminated with a two point or a three point calibration. However, nonlinearity cannot be easily calibrated. Integral nonlinearity (INL) is a measure of the deviation of the system's actual output from the best output, given a certain input. INL is measured by applying a series of dc test voltages along a straight line computed from the slope and offset transfer function of the system. INL is the maximum of the difference in the actual output from the best output for the full scale input range. The INL calculation uses the least-sqaured error best fit method to determine the new straight line to minimize the root-sum-square of the INL errors above and below the original end-point line.

Figure 4-7 shows the measured INL for one unit across temperature (with a fixed gain of 1V/V) and across gains (at 25°C).

Figure 4-7 INL Error Versus Input Voltage

The typical INL listed in the ADS127L21 data sheet is 0.4ppm, with a maximum of 1.8ppm across the range 0°C to 70°C. For the PGA855, the typical INL listed is 2ppm, with a maximum of 10ppm, which is consistent with the data collected for one unit, as shown in Figure 4-7.