Visible Touch: Tactile Sensor Enables Multi-Purpose Inspection

“If you see a little scratch on a part, but you also want to do a profile of the surface roughness, and then you want to measure the diameter of a hole, how would you do it today?” asks Youssef Benmokhtar, CEO of tactile sensor company Gelsight. One could tackle this challenge in a number of ways, but it would most likely require removing the part from the machine and using a stylus profilometer or microscope to measure the surface roughness and a CMM or gage to measure the hole. The scratch would be more difficult to measure, depending on how reflective the material is — a high-end microscope might do the trick. But regardless of how it’s done, the part will require multiple inspection methods.

Because different features of a part require different inspection methods, shops must maintain a variety of inspection systems. But what if surface roughness, defects, cleanliness, dimensions and more could all be measured using one device? A new inspection system consisting of an elastomeric sensor and a camera could make part inspection more efficient.

Making Contact

Vision science professor Edward Adelson co-founded Gelsight in 2011 with his postdoctoral student Kimo Johnson, and local technologist and entrepreneur Janos Rohaly, as a spinoff of his work at MIT. According to Benmokhtar, Adelson was inspired by watching his infant daughter use her sense of touch to explore the world around her. “That led him to ask the question, ‘Is there a sensor that mimics human touch and feel?’” he explains. “And the answer was no, there was no such thing. He’s one the very few people in the world that can take that answer and say, ‘Well, okay, if it doesn't exist, I'm just going to invent it.’”

Gelsight’s solution features a tactile sensor made of an elastomeric material that’s designed to conform on contact with an object in the same way human skin does. A camera located behind the sensor captures multiple images of what the sensor is detecting, and an algorithm stitches them together to create a real-time 3D map of the object’s surface. “It’s basically like having a camera inside your finger,” Benmokhtar explains. “We make touch visible, so you can actually see details of textures.”

While the goal was to create a sensor that could replicate a human’s sense of touch, Gelsight’s sensor actually exceeds that. Benmokhtar says the sensor is capable of submicron precision thanks to its use of photometric stereo, a technique where multiple photos of the same object under different lighting conditions are stitched together. This can provide pixel-level information on the depth of the object’s surface. This method is also, like human touch, agnostic to the optical properties of the material it’s measuring. “Vision systems will struggle with black, transparent or highly reflective surfaces,” he explains. “You can be blinded, but you can still touch and get a good understanding of the texture. So that's a key advantage that we bring to this space.”

Detecting Defects and More

Once the sensor creates the 3D map, Gelsight’s software interprets it. This process varies based on the end user. “In aerospace, for example, it's extremely critical to characterize scratches and dents,” Benmokhtar explains. Gelsight has a suite of applications that can identify different types of defects including scratches, surface roughness and pits that indicate corrosion.

The company is increasing the types of defects it identifies as it gains customers. “Over time, we’re adding application after application to meet as many use cases as possible,” Benmokhtar notes. “Our hardware is like good wine. It gets better over time because we add functionality to our software every three to four months.” One recent feature is Automatic Defect Detection. Users can either scan the entire field of view for defects or select a specific defect or area. The software then analyzes any defects and automatically provides the deepest points, x, y and z measurements, and volume.

Inspection results are designed to be easy to interpret, no matter what the user is inspecting. The software can even be programmed to automatically identify parts that pass, fail or need rework. “You don't need to send it to an engineer to analyze it,” Benmokhtar says. “All of that could be preprogrammed into routines in the Gelsight software.”

Gelsight can also be used beyond defect detection. “Our product is really a platform, because at the end of the day, it all starts with that really precise 3D map,” Benmokhtar says. “On top of that 3D map, you can do almost an infinite number of things.” The company started with detecting scratches and dents because its initial customers were in the aerospace industry. It eventually expanded to include surface roughness, corrosion detection and part features such as diameters and curvatures. A more recent application with a space company involves particle counting for parts cleaning specifications. “They want to make sure that the surfaces are super clean, so they have very tight specs,” he explains. “Our system actually is used to characterize the number of particles on a surface.”

Users can export data from Gelsight to Excel or other programs for trend analysis. Gelsight can also automatically generate PDF reports users can then feed into their quality systems for material requirements planning (MRP). The map itself can also be saved in different file formats, including STL, which can then be uploaded into CAD for parts that need rework.

From the Lab to the Manufacturing Floor

The company offers several versions of its sensor. Two of them, the Gelsight Mobile and Gelsight Mobile Shock Protected Edition, are designed to be portable and can be used anywhere, not just in a metrology lab. “We bring lab performance to wherever the customers want. It could be on assembly lines, for example. It could be mounted to a robot, to a fixture, or even in a hangar or on the tarmac right at an airport, like we have some of our customers do,” Benmokhtar notes. “It's one of our strengths.”

This portability enables the Gelsight sensor to be robotically automated. The company has released several application programming interfaces (APIs) to enable users to integrate the sensor with robots and cobots. For example, users can now trigger scans remotely and see a live view of what the sensor is measuring. It’s also partnering with Flexxbotics, which makes software that controls robotic systems, to help users integrate Gelsight sensors with robots and cobots.

This is just the first step in increasing applications for Gelsight. “Ultimately, we want this to be mounted on CNC machines, CMMs, and portable arms,” Benmokhtar says. Integrating a Gelsight sensor into a CNC machine could make machine tools more productive versus offline inspection. This is especially true given that Gelsight can perform multiple inspection tasks at once.

Gelsight could eventually be part of a closed-loop machining system that adjusts machine tool parameters based on inspection data. “I've seen with my own eyes in CNC shops, where people will take the part, measure the roughness and put it back on the machine to rework or fine tune,” Benmokhtar says. “The end result could be completely eliminating that step, and you can just keep the part on the machine.”