The Art of Submicron Measurements

There are inherent challenges in measuring parts to submicron tolerances. We have previously discussed the need for a climate-controlled environment and absolute cleanliness, but we have only scratched the surface (so to speak). Special attention must also be paid to the selection of the gage and readout as well as to mastering.

If you are checking parts for tolerances of 0.5 micron or checking gage blocks, rings and discs to less than 1.0 micron, you will need 0.1-micron, or even 0.01-micron, resolution (minimum grad value) on the gage readout. But beware of excessive magnification. Some gage manufacturers create the appearance of submicron accuracy by supplying an electronic amplifier, with units reading in parts of a micron on a garden-variety gage. The average shopfloor gage is mechanically repeatable to only 1 micron or so; what you really get is a highly magnified look at the gage’s repeatability error.

The resolution of the readout should accurately reflect the precision (repeatability) of the gage itself. This can be checked by repeatedly measuring a part under controlled conditions. If the reading varies by 1 micron or more between trials, the gage may be incapable of handling micron inspection duties.

When measuring high-accuracy inner diameters, gages typically contact the workpiece with as much as 120 grams of force. At submicron tolerances, this can produce measurable deflection of the jaws that might be unavoidable. Repeatability demands that it remain constant from one trial to the next. A frictionless mechanism, such as a reed spring arrangement, is essential to maintaining constant gaging force within 0.5 gram.

We normally rely on a master to ensure the accuracy of a gage. However, if we are using the gage to measure a master, we have to go one step further and refer to certified gage blocks to master the gage. This is relatively straightforward for outside measurements but more complex for inside measurements. This is especially true when using inner-diameter/outer-diameter comparators, but not so on precision universal-length machines.

On comparative gages, it is necessary to set up a stack of gage blocks to set the reference size. With this method, a single gage block or a stack of blocks is used to set a pair of caliper blocks at a precise distance. The blocks are held together with clamping rods, with the caliper blocks extending over both sides of the gage blocks. The comparative gage is mastered to the distance between the caliper blocks, checking at both ends for parallelism error.

Today, however, universal measuring machines are replacing the older comparative systems. The process of maintaining multiple sets of gage blocks, spending hours stacking and checking blocks—along with all the potential sources of errors—results in too much wasted time. The universal measuring machines offer the needed level of performance for measuring standards, but they do not require a comparative master setup with their long measuring range. Rather, one master ring is used to set a zero location and the system can conduct high-performance measuring on a wide variety of sizes in just one setup.

In this case, it is critical to have a set of certified master rings to set various zero locations within the range of the machine. The setting rings have to be of the best uncertainty to ensure the best possible reference point.

This brings us to the actual point of measuring master rings: It is necessary to account for geometry errors as well as surface finish, scratches, waviness and other submicron imperfections. Measure class XX masters (tolerance = 0.5 micron for sizes between 2.6 and 21 mm) at a depth of 2 mm in from both ends, and in the middle of the ring. This avoids the bellmouth conditions likely to exist near the ends, and detects most barrel-shape, hourglass and tapered conditions.

For Class XXX masters (tolerance = 0.25 micron for sizes between 2.6 and 21 mm) the drill is even more rigorous. Confine all measurements to a 6-mm-tall band in the center of the ring and take a total of six measurements: at the top, middle and bottom of the band, in both north/south and east/west orientations.

The point here is not to find and discard masters that show variation of 0.5 micron or more. When working at the submicron level, a certain degree of uncertainty is inevitable. The objective is to minimize the uncertainty that the master contributes to the overall measurement process. By using the fewest number of masters, one reduces potential sources of errors.