Article

Machine Vision: Automated Vehicle Maintenance & Vehicle Inspection Systems

Posted September 20, 2022 | Industry | Amplify
Machine vision inspection
In this issue:

  AMPLIFY  VOL. 35, NO. 8
  
ABSTRACT

Cutter Expert Curt Hall explains how machine vision systems using machine learning (ML) and other artificial intelligence techniques are gradually spreading from use in large auto manufacturers to being incorporated in aftermarket businesses. Increased accuracy of ML-based cameras, edge computing systems, machine vision startups targeting the market, and MaaS are all driving the expansion. One main area of use, Hall explains, is in camera-based robotic systems to automate basic maintenance operations like tire changing. Another area is automated vehicle inspection that can be used by repair shops, dealership service departments, online vehicle marketplaces and auction sites, and auto parts recyclers. Hall says the trend is just getting under way, so we’ll see a steady rise in the use of these systems in the near future.

 

The auto industry helped pioneer the use of machine vision systems to automate various vehicle manufacturing processes. Today, the technology has evolved to the point where it’s spreading beyond vehicle manufacturing into automotive aftermarkets. This represents a significant development in the evolution and application of machine vision technology, as well as the use of the technology in the automotive market in general.

Today, auto aftermarket product and service providers are turning to advanced machine vision systems employing machine learning (ML) and other artificial intelligence (AI) techniques to automate various operations. Examples include automating basic maintenance and repair activities like tire inspection and replacement (in conjunction with robotics technology) and automating the vehicle inspection and repair order–generation processes.

Companies are also using machine vision to assess and present the status of a used vehicle to customers of online auto resale marketplaces and auctions as well as to identify and classify parts on salvage vehicles to optimize auto recycling operations.

What’s Driving This Trend?

Various technical and industry developments are driving the use of AI-powered computer vision and imaging in the automotive aftermarket and online resale marketplaces, including:

  • Increased accuracy of ML-based camera systems

  • Availability of cloud and edge computing systems, including service-based offerings

  • Machine vision startups targeting the auto aftermarket

  • Partnerships between large automakers and machine vision startups

  • Mobility as a service (MaaS)

Increased Accuracy of Machine Vision Systems

By employing deep learning neural networks that have been trained on millions of vehicle images (including various examples of damages), the accuracy of the algorithms used by computer vision systems has increased considerably over the past five years. Camera-based computer vision systems can now identify, classify, and precisely locate multiple types of defects in a single image. This includes damaged parts like bumpers, fenders, and grilles; fluid leaks; corrosion; and even underinflated or worn-out tires.

Cloud-Based Systems & Edge Computing

The availability of computer vision applications in the form of cloud-based solutions and the use of edge computing are helping accelerate technology use. The former makes the technology more accessible to repair shops, dealers, fleet operators, and other aftermarket businesses. The latter helps reduce the latency associated with image processing and analysis. For example, models can run autonomously on or near the edge, including on self-contained, high-resolution, smart camera components installed in the workshop areas of service centers. Additionally, camera-based vehicle inspection systems are now integrated with car dealership management systems, online auto selling/auction platforms, and other applications, making their use even more appealing for aftermarket applications.

Machine Vision Startups

Vendors now offer advanced machine vision systems, including solutions targeted at auto aftermarket uses. Such products are available as self-contained components for easier installation and integration into existing operations at repair shops, dealers, fleet operators, and online marketplaces.

Big Auto & Startups

Partnerships involving machine vision startups and the large automakers are on the rise, mainly in an effort to expand the use of the technology into aftermarket operations. Examples include computer vision–based vehicle inspection systems from UVeye and Tractable (discussed further in the auto inspection section).

MaaS

In the not-too-distant future, the rise of on-demand, personalized transportation services — MaaS — in the form of ride sharing and car sharing and (perhaps most importantly) in combination with autonomous vehicles will require fleet operators to quickly service vehicles to get them back on the road. This is expected to lead to significant disruptions in the auto servicing industry, in which automation will play a major role.

Automated Vehicle Maintenance Systems

Automakers have used systems combining machine vision and robotics to automate key manufacturing operations for decades. Adoption of the technology to automate aftermarket operations has been slow, but service shops are starting to use camera-based robotic systems to automate basic repair and maintenance operations.

For example, tire changing is one of the most common and time-consuming vehicle maintenance jobs. RoboTire, a Detroit, Michigan, USA–based robotics and automation company, has developed a robotic tire-changing system (see Figure 1).1 Using high-definition cameras, ML, and advanced robotics, the system automates much of the tire-changing process. The robot can remove lug nuts, lift off old tires, and install new tires and wheel assemblies with limited human assistance.

Figure 1. RoboTire automated robotic tire-changing system (source: RoboTire)
Figure 1. RoboTire automated robotic tire-changing system (source: RoboTire)

Using a combination of high-definition cameras, ML, and pressure-sensitive sensors, the system accurately identifies the placement of lug nuts for removal and reinsertion, tightening them to the desired degree during remounting.2 Cameras and sensors prevent the robot from coming into contact with the body of the vehicle it is servicing. The company claims the system can change four tires on a passenger vehicle in approximately 25 minutes, a job that takes about an hour when performed manually.3 Tire pressure monitoring and tire balancing must still be performed by human mechanics.

RoboTire’s sophisticated camera system can identify issues such as stripped/damaged lug nuts or corroded hubs and alert a mechanic to the problem. In addition, the robot is programmed to allow time for mechanics to complete their visual safety inspections, including the need to clean the wheel hubs, if necessary.

Automated robotic tire-changing systems bring several benefits to auto repair providers and fleet operators. They can increase the number of customers a shop can service and enable the completion of more work with fewer staff. Freeing up mechanics lets them focus on more complex services and repairs, helping get vehicles serviced and back on the road sooner, which translates into a better customer experience. Robotic systems also lessen the possibility of injuries to workers related to manually lifting and mounting heavy tires and wheels.

Of course, any time robots need to operate around people, their use becomes considerably more complicated due to human-machine interaction: auto repair shops are busy environments with many safety hazards. To mitigate this issue, RoboTire uses sensors, cameras, and an ML component to interpret and adapt to the environment within which it’s functioning — including in response to humans in its immediate vicinity.

Discount Tire, an independent tire and wheels retailer, has installed and is operating a RoboTire system in its Fountain Hills, Arizona, USA, store. It will be interesting to see if Discount Tire deploys the system in any of its other 1,100 store locations.

RoboTire has indicated it is looking at developing robotic systems to automate other vehicle maintenance operations, such as brakes, oil changes, and battery replacements.4

Automated Vehicle Inspection in the Aftermarket

Machine vision has been a staple of auto manufacturing in automating quality-control processes, enabling quick and accurate identification of defects in paint finishes, body welds, and improperly aligned parts as vehicles proceed along the production line.

Today, dealer service centers and independent repair shops are using machine vision systems to automate the vehicle inspection process for various aftermarket uses, including:

  • Maintenance and servicing

  • Online used car marketplaces and auctions

  • Vehicle-salvaging and parts-recycling operations

Tel Aviv, Israel–based UVeye offers a drive-through, ML-based vehicle inspection system designed to automate service department operations for dealerships, fleet operators, car rentals, used car marketplaces, and repair shops (see Figure 2).5

Figure 2. UVeye’s machine vision–based automated vehicle inspection system (source: UVeye)
Figure 2. UVeye’s machine vision–based automated vehicle inspection system
(source: UVeye)

The system features comprehensive vehicle inspection capabilities, including an underbody scanner for detecting problems like frame damage, oil leaks, and rust. The system’s tire inspection function can identify the brand of tire on a vehicle and its specifications and status, including air pressure, tread depth, and sidewall damage, even if a vehicle’s tires are mismatched. External inspection capabilities provide a 360-degree exterior scan of a vehicle’s parts, including bumpers, door locks, grilles, mirrors, and windows.

In a key development, UVeye’s vehicle inspection system is now integrated with CDK Global’s Fortellis Automotive Commerce Exchange application, a popular platform that connects auto dealers, manufacturers, developers, and lenders.6 The combined system offers an integrated platform for automating the vehicle repair process — from initial vehicle inspection to repair order generation.

UVeye says it takes about 15 minutes for a vehicle to pass through its vehicle inspection system and have a repair order generated, about 45 minutes less than the same process performed manually. As a customer’s vehicle drives through a UVeye inspection lane (which takes just a few seconds), a service writer using a tablet can view a series of photos and a list of problems detected by the system. Serious problems (e.g., a worn tire, a damaged muffler, or a fluid leak) are flagged in red on the service advisor’s screen. Customers are shown photos of the issues and work with the service advisor to determine which repairs to carry out. On approval, the system auto-generates a repair order (including an estimate) based on information from the inspection report.

A number of US dealerships are using this system. In March 2022, Volvo Car USA initiated a program to use UVeye’s camera-based inspection system at select dealers.7 Volvo says it gives retailers the ability to quickly and cost effectively evaluate trade-ins and to examine the condition of customers’ cars as they come in for servicing.

Volvo hopes to expand use of the system to more of its 280+ independent US retail locations. Volvo has been involved with UVeye since 2019, when it became a strategic investor in the company and installed UVeye’s body inspection technology on its assembly lines to support its manufacturing quality assurance program.

In June 2022, UVeye received an investment from the capital venture arm of General Motors to help fund the vehicle inspection technology.8 UVeye is also working with GM to explore expanding UVeye’s system to GM dealerships. The two companies are undertaking other vehicle inspection projects, including those involving used car auctions, fleet operations, and auto dealership sales. In the future, UVeye plans to incorporate electric vehicle and autonomous driving platforms into its inspection system.

Visual AI company Tractable offers machine vision inspection solutions for insurance agencies, auto service departments, and others.9 Tractable’s vehicle inspection platform uses ML and image classification to conduct a pixel-by-pixel assessment of a photo of a car. It can determine vehicle damage down to the individual part level, classify the condition or the degree/severity of damage, and generate a detailed estimate, which can include a certainty score.

In March 2022, Tractable teamed with Black Widow to integrate its AI visual inspection system with Black Widow’s 4K imaging system.10 Black Widow is an eight-camera, drive-through system designed to capture high-definition vehicle images that are auto-assembled into a 360-degree virtual tour of the inside and outside of a vehicle. These images can be edited and published to online vehicle marketplaces and auction sites.

Tractable’s system applies ML-based analytics to images captured by Black Widow to detect external damage and generate a vehicle condition report that includes estimated repair costs. This enables auctions and dealerships to more accurately and consistently assess each vehicle and make better decisions during the vehicle remarketing and merchandising process. Additional benefits include improving the customer experience by offering a more comprehensive and transparent view of the vehicle being considered for purchase.

Tractable’s systems are also being used to automate parts recycling — specifically, the processes involved in identifying suitable parts on salvaged vehicles and ensuring those parts can be recycled and reused. The system can also help repair shops source needed parts.

LKQ North America is using the Tractable system to accelerate and optimize its auto parts recycling and distribution operations for procuring salvaged vehicles.11 Tractable’s algorithms visually analyze and classify images of parts on salvaged vehicles that come up for sale, allowing LKQ to assess the damage of a body part on a particular vehicle and determine which parts can be resold. Tractable then matches the salvaged parts to the needs of repair shops and notifies them about availability.

Conclusion

Machine vision is one of the most successful commercial applications of AI. This is particularly true when it comes to the auto industry, where it is used extensively to automate manufacturing operations. The technology has now evolved to the point where companies are increasingly applying machine vision to the aftermarket, most noticeably for automated inspection applications and, to a lesser extent, in combination with robotics systems for automating basic servicing tasks. This trend is just getting under way, but we’re likely to see the use of machine vision systems in auto aftermarkets increase steadily in the near future.

References

1    Next Generation Tire Changing System.” RoboTire, 2022.

2    Hope, Graham. “Robotic Tire Changing System Cuts Install Time.” IoT World Today, 10 August 2022.

3    RoboTire. “System Overview — Extended Length.” YouTube, 29 October 2020.

4    RoboTire.” Y Combinator, accessed August 2022.

5    Automotive Solutions.” UVeye, accessed August 2022.

6    CDK on Fortellis: Powering Automotive Innovation and Collaboration Through APIs and Apps.” CDK Global, accessed August 2022.

7    Volvo Car USA Looks to Improve Customer Satisfaction and Business Efficiencies with New Automated Vehicle-Inspection Technology from UVEye.” Press release, Volvo Car USA, 10 March 2022.

8    UVeye Enters into Strategic Collaboration with General Motors to Expand Technology to Global Dealerships.” Cision PR Newswire, 22 June 2022.

9    See my Cutter article on the insurance solution: Hall, Curt. “Computer Vision and Imaging in the Insurance Sector.” Cutter Consortium Technology Advisor, 29 June 2021.

10  Tractable and Black Widow Collaborate to Offer Enhanced Imaging Solution with Detailed Vehicle Condition Insights.” Tractable, 22 March 2022.

11   LKQ North America Partners with Tractable to Accelerate Auto Parts Recycling and Distribution with Artificial Intelligence.” Tractable, 29 June 2021.

About The Author
Curt Hall
Curt Hall is a Cutter Expert and a member of Arthur D. Little’s AMP open consulting network. He has extensive experience as an IT analyst covering technology and application development trends, markets, software, and services. Mr. Hall's expertise includes artificial intelligence (AI), machine learning (ML), intelligent process automation (IPA), natural language processing (NLP) and conversational computing, blockchain for business, and customer… Read More