Until recently, computer vision — used most widely in manufacturing — and mainstream computing technology have existed in parallel worlds. Along with other factory floor technologies, computer vision tends to be machine-specific, hardware driven, and makes little if any use of the Internet. Many the advances we take for granted in modern computing — ubiquitous connectivity, unlimited data storage in the cloud, insights drawn from massive unstructured data sets — have yet to be applied systematically to the factory floor in general and to computer vision specifically.
It’s no surprise when you consider that until recently most computer vision software was written by computer vision hardware makers, built on embedded systems without open APIs. What comes to mind when you think of the software that came bundled with your scanner, your Wi-Fi router, your car’s navigation system? Balky, inflexible and unintuitive. The software isn’t much more than a utility to run the hardware.
But this closed world is being broken open by a convergence of emerging technologies:
- The proliferation of cheap, high pixel-density camera sensors
- Open implementations of vision algorithms, machine learning, and statistical tools
- Large amounts of cheap computing power, becoming virtually limitless in the cloud
These technologies offer all the raw materials needed for a massive shift in how computer vision is practiced. It’s a shift from focusing on the raw material of visual data — the pixels and bitmaps generated by specific cameras — to extracting data from images and using statistical and data science techniques to draw insights.
This new approach to computer vision has a powerful application amid an American manufacturing renaissance emphasizing rapid product cycles and mass customization. Whereas the archetypal American factory was built around systematic, repeatable function, modern manufacturing is about flexibility, adaptability and high efficiency. We’ve gone from Henry Ford’s “any colour he wants so long as it is black” to Google’s Moto X phone — customer-configured, manufactured in the U.S. and delivered within four days.
Unrelenting Quality Demands
But that need for flexibility on the manufacturing line is in tension with unrelenting quality demands that manufacturers face across industries and down supply chains. Despite huge investments in quality control, automakers recalled nearly as many cars as they sold in the U.S. in 2012. Ford and GM made warranty payments of $5.7 billion in 2012, more than half of the $10.5 billion they reported in net income. Automakers are now paying suppliers prices based on benchmarks like defects per million, terminating those who fall below thresholds, and pushing liability for warranty claims down to their suppliers.
While automation has transformed much of manufacturing, a surprising amount of quality control is still done by hand or otherwise relies on human judgement. Many types of inspection require visual evaluation, but manfacturers’ experience with computer vision in quality control has been a frustrating one. Walk into a factory and ask the manager about computer vision, and you are likely to hear a variant of, “Oh yeah, we tried that, it didn’t work very well, we had to throw it out.”
Existing machine vision uses a 30-year-old architecture that’s capital-intensive and severely constrained in its abilities. Today’s computer vision systems operate as stand-alone islands, rarely connected to the Internet. Every time needs change, each installation has to be manually reprogrammed, unit by unit.
Worse still, little data is kept, making it difficult to spot trends or find correlations among multiple variables. Most manufacturing quality inspection by machine vision today is pass/fail. If the initial inspections of a production run pass the quality inspection, the machines are turned on and the testing data overwritten.
The New Computer Vision
The new computer vision, liberated from its hardware shackles and empowered by connectivity, unlimited data storage and Big Data-style statistical analysis, is beginning to change the role of vision in manufacturing. Instead of being a reactive tool to detect defects, computer vision is becoming a data collection tool supporting defect prevention initiatives, improving understanding of complex processes, and enabling greater collaboration across entire supply chains in real time.
With modern web services, once the data is collected it is easily aggregated into dashboards and distributed to production workers, quality engineers, and management, locally or around the globe. Manufacturers can share data with supply chain partners, making it easier to monitor their suppliers or to satisfy reporting requirements for customers.
One of our customers, a large manufacturer of high-quality bolts and other fasteners to automakers, is bringing this vision to life. Their system uses computer vision to analyze the grain pattern of bolts. If the pattern is wrong — if the grain lines end on a load-bearing surface — the bolt head can shear off when a factory worker torques it down, or worse, when it’s already holding an engine block in place.
The company is capturing images using a $100 scanner purchased at Best Buy. All the intelligence is in the software, running remotely on Amazon’s cloud computing platform. The system compares each image to thousands of other metal grain photos stored in the cloud, looking for patterns that correlate with part failure.
The bolt maker is now exploring the extension of its the computer vision system to its steel supplier, which will capture images of metal grain from each batch of steel rods it ships to the fastener maker. The fastener maker will then be able to analyze increasingly massive data sets to correlate grain patterns in the steel rods with quality measurements in the finished bolts.
Instead of examining only a single station, large data sets let companies trace complex interactions down the production line and across the supply chain. Upstream stations may produce parts that are technically within tolerance, but when certain ranges of acceptable variation are combined, they cause downstream defects after installation.
For our bolt-making customer, the raw material (a steel rod) and the batch of bolts made from that rod may each be well within spec, but retrospective data analysis may show that certain combinations of grain pattern in the steel rods lead to higher failure rates on bolts used for specific applications.
As automakers adapt the system it will gain even more power. Should an automaker report that the fastener-maker’s bolts are breaking and leading to warranty repairs, the parts supplier now has the analytical tools to determine the source of the problem. They can run analysis to determine whether the failed bolts came from a particular batch of steel rods, or were made on a day when their line was adjusted to a specific tolerance – or whether the problem wasn’t with the bolt itself, but rather with the worker on the left side of the assembly line who consistently overtorques the engine bolts.
Once the captured data is in the cloud, such systems can store an unlimited amount of data indefinitely, for reanalysis and retrieval anytime. They let plants run correlations over time, track trends and identify root causes, and as new variables of interest arise, go back and analyze previously acquired data.
As each plant gets smarter, the whole system gets smarter. Like Google learning more about consumers with their every search and click, we’re able to aggregate our learnings from quality issues common across industries.
Ultimately, vision can turn physical world challenges into Big Data problems. We know how to solve these Big Data problems better and better every day.
(Written by Jon Sobel, CEO and co-founder of Sight Machine Inc.)