Big Data, ML, and Geo Mapping for Real-Time Traffic Monitoring and Analysis

Posted December 15, 2020 in Data Analytics & Digital Technologies
Big Data, ML, and Geo Mapping for Real-Time Traffic Monitoring and Analysis

Increasingly, government agencies, businesses, universities, and other organizations are working together to build artificial intelligence (AI) systems that analyze geo-mapped and other location-based data. There is considerable innovation in this area, particularly involving the implementation of traffic planning and safety applications utilizing machine learning (ML) and predictive algorithms. One of the most impressive is a new ML application that uses Uber driver data to track and alleviate urban traffic congestion. TranSEC (Transportation State Estimation Capability) was developed at the US Department of Energy’s Pacific Northwest National Laboratory (PNNL). It is designed to give urban transportation analysts and traffic engineers access to information about city traffic patterns in order to relieve bottlenecks, chokepoints, and other problems.

What’s So Cool About TranSEC?

According to PNNL representatives, most publicly available traffic information at the street level is sparse and incomplete. Consequently, traffic engineers typically have had to rely on isolated traffic counts, collision statistics, and speed data to determine roadway conditions.

TranSEC, in contrast, integrates anonymized GPS traffic datasets sourced from hundreds of thousands of online Uber drivers with other publicly available traffic sensor data to map street-level traffic flow over time. This big data integration and ML analysis capability, running on PNNL’s computing resources, provides a more fully encompassing picture of city traffic, including a very detailed method that shows how traffic is actually moving around cities.

In short, TranSEC provides street-level estimation over a large metropolitan area. Moreover, the tool (and its models) is portable, and cities can apply it to any urban area where aggregated traffic data is available.

Uber’s data sharing is part of the company’s “Uber Movement” program where, by sharing GPS data sourced from its driver vehicles, Uber seeks to assist city officials and transportation planners around the world to better understand and address urban transportation challenges. (It’s good PR, too, and a great way for the company to ingratiate itself with city leaders and other officials.) Cities can share their transportation data with Uber, too.

So, some may ask, “What’s so cool about TranSEC? I use the WAZE app on my smartphone to plan my route all the time....” But apps like WAZE are designed for individual drivers trying to get from one point to another. City traffic engineers, in contrast, are concerned about how to help all vehicles get to their destinations efficiently. The differences can be considerable. For example, a route that seems efficient for an individual driver could possibly lead to too many vehicles trying to access a road that was not built to handle that volume of traffic.

TranSEC also has capabilities most other traffic-monitoring applications lack: It can analyze sparse and incomplete information. It uses ML to connect segments with missing data; in effect, allowing the tool to make near real-time street-level traffic estimations.

In one application scenario that used public data from the entire 1,500-square-mile Los Angeles, California, metropolitan area, the PNNL research team claims it was able to reduce significantly the time needed to create a traffic congestion model — from hours to minutes. Of course, this was done using the high-performance computing at PNNL. Still, it is significant, because is shows that real-time (or near-real-time) traffic analysis is feasible.

TranSEC’s ML Component

ML helps improve the accuracy of analyses over time. As the tool ingests and processes more data, it becomes more refined. And here the importance of ML analysis in traffic-monitoring applications shows its worth, by providing the ability to understand how disturbances spread across networks. Given enough data, the ML component will be able to predict impacts so that traffic engineers can create strategies to work around them.

PNNL researchers indicate they used a graph-based model combined with novel sampling methods and optimization engines to learn both the travel times and the routes. Arun Sathanur, a PNNL computer scientist and a lead researcher on the team, noted: “The method has significant potential to be expanded to other modes of transportation, such as transit and freight traffic. As an analytics tool, it is capable of investigating how a traffic condition spreads.”

Users can upload real-time data and update TranSEC on a regular basis in a transportation control center. Engineers can use short-term forecasts for decision support to manage traffic issues. The approach is also extensible to include weather or other data that affect conditions on the road.

What About Computing Requirements?

While running a full-scale city model requires high-performance computing resources, TranSEC is scalable. For example, a road network with only the major highways and arterials could be modeled on a powerful desktop computer.

PNNL plans to make TranSEC available to municipalities nationwide. And researchers could also apply it to other scenarios, such as autonomous vehicle routes, according to the team.

TranSEC is an example of an innovative application combining big data, ML analytics, and the use of geo-mapped data for traffic planning. It is also interesting in that it involves collaboration between a government agency, a commercial transportation business, and city traffic planners and managers. Expect to see more similar collaborative efforts in the future.

Finally, I’d like to get your opinion on today’s Advisor, and on the use of AI, big data integration, and geo-mapped data in general. As always, your comments will be held in strict confidence. You can email me at or call +1 510 356 7299 with your comments.

About The Author
Curt Hall
Curt Hall is a Senior Consultant with Cutter Consortium’s Data Analytics & Digital Technologies and Business & Enterprise Architecture practices and a member of Arthur D. Little’s AMP open consulting network. He has extensive experience as an IT analyst covering technology trends, application development trends, markets, software, and services. Mr. Hall's expertise includes artificial intelligence (AI), cognitive systems, machine… Read More