By Nanjing Zhongshe Aviation Technology Development Co., Ltd.
The prerequisite for intelligent highway management is to be “visible.” With the maturity and popularization of advanced image acquisition technologies such as drones and satellite remote sensing, full road visualization, real-world simulation, and digitization have become important development directions, using image recognition technology for intelligent analysis and decision-making.
From 2017 to 2019, Jiangsu Province made comprehensive efforts to carry out five actions for environmental remediation along traffic trunk lines, addressing issues of “dirt, disorder, and poor quality,” clearing illegal facilities, and enhancing greening levels. Starting in 2020, this was transformed into a long-term management effort. During this period, the Jiangsu Provincial Transportation Comprehensive Law Enforcement Supervision Bureau adhered to the principle of “Seeing is Believing.” A piece of text or a string of statistics is insufficient as a decision-making basis for road environment remediation or road administration law enforcement, making it difficult for management departments to make objective and fair analyses and decisions. Facing the large-scale highway management work in the province, the Jiangsu Provincial Transportation Comprehensive Law Enforcement Bureau established a “Provincial Road Environment Management Platform” to create a “Real-time Visualization Map of Provincial Road Environments,” allowing for site observation from the office and utilizing AI image recognition technology for intelligent analysis, alleviating manpower shortages and forming a scientific and efficient intelligent management approach for road environment law enforcement.
Jiangsu Provincial Road Environment Management Platform
The “Jiangsu Provincial Road Environment Management Platform” is an intelligent business platform developed by Nanjing Zhongshe Aviation Technology Development Co., Ltd. (referred to as “Zhongshe Aviation”) that comprehensively applies drone aerial photography, machine vision, and geographic information management technologies. The platform uses drones and satellite remote sensing technologies to acquire three-dimensional, all-angle image data of road environments and achieves map-based deployment, allowing users to “look wherever they click” on the provincial map, analyzing it using AI image recognition technology to objectively identify issues in road environment law enforcement along highways. Additionally, it is linked to daily inspection work, ensuring continuous updates, ongoing analysis, dynamic reporting, and traceability, providing long-term, complete, and detailed data for understanding provincial dynamics, investigating key issues, macro analysis, decision-making, and evaluating law enforcement effectiveness.
Integrated Air and Ground: A Real-time Visualization Map of Provincial Roads
The “Real-time Visualization Map of Provincial Roads” is a geographic information image database established by Zhongshe Aviation that locates and models drone videos, orthophotos, and oblique photography on GIS layers, linking it with basic information data of provincial highways, ordinary national and provincial trunk roads, and rural roads. It provides a comprehensive and intuitive display of ground-level clarity within a 200-meter bandwidth along highways. The distribution of remediation targets along highways is complex, extensive, and large in scale, making traditional “vehicle-based inspections” inefficient and prone to blind spots. For example, under-bridge spaces, concealed greenery, building facades, and blocked ditches are often overlooked. Zhongshe Aviation’s self-developed “integrated air and ground” data collection method utilizes 45-degree aerial video dual-direction collection, 90-degree orthophotos, and ground vehicle video dual-direction collection to cover the entire road area comprehensively and without blind spots. Through regular inspection updates, the platform is equipped with the latest highway image data, with all data recording the time of capture, geographic coordinates, and highway pile numbers. The visible images can be measured with centimeter-level accuracy, providing the latest and most reliable data, addressing the challenges of documentation and traceability.
Display of Road Environment in One Map
In 2020, the “Real-time Visualization Map of Provincial Roads” was established as a working platform for evaluating law enforcement along provincial highways, ordinary national and provincial roads, and rural roads, gradually covering nearly 5,000 kilometers of highways, 14,000 kilometers of ordinary national and provincial roads, and 143,000 kilometers of rural roads in Jiangsu Province, promoting real-time visualization and intelligent management of road environment law enforcement across the province.
AI Image Recognition: Intelligent Judgment Throughout the Process
Another core innovation of the platform is the application of AI image recognition technology, based on convolutional neural networks (CNN) for low-resolution small-sample semi-supervised image recognition. This technology learns the mapping relationship between low-resolution and high-resolution images through convolutional neural network training, obtaining parameters for each layer of the network. A recurrent layer is added to the convolutional neural network, increasing network depth while fully utilizing prior image information, effectively avoiding the phenomenon of local edge blurring in the reconstructed images. Using the convolutional neural network as a feature extractor for images, data augmentation transformations are introduced into the original small dataset to expand the sample range; on this basis, a source pre-trained model on a large-scale dataset is fine-tuned on the target small dataset to extract model weights and image features, excluding the last fully connected layer. Combining the features extracted from the source pre-trained model, the layer freezing method is used to fine-tune the convolutional model on the target small-scale dataset, resulting in final classification and recognition outcomes.
During the acceptance work of the five actions, AI image recognition technology played a crucial role. It identified a total of 100,000 images of law enforcement traffic elements with an accuracy rate of 95%. Over three years, drones collected data from over 45,000 kilometers of provincial trunk roads, using algorithms for deep machine learning and big data analysis to assist in determining key projects, regions, and types for law enforcement. Combined with drone data collection for the entire road surface and long-distance ground data, it processed and analyzed image data with high efficiency, successfully assisting provincial trunk roads in identifying discrepancies, precise investigations, acceptance checks, and remediation during the five actions. The main issues identified by the AI image recognition technology include illegal junctions, unpaved access roads, illegal buildings, non-standard advertisements, debris, street vendors, trash bins, pollution sources, and car wash points, with a reported recognition accuracy of 95%. With a total transportation network exceeding 186,000 kilometers and trunk roads exceeding 22,000 kilometers, the demand for daily road administration inspections is unprecedentedly high, and AI image recognition technology provides a unified, fair, and just evaluation method, further improving the efficiency and accuracy of road law enforcement supervision.
Multi-point Practice: Preparing for the National Evaluation
In November 2017, the Environmental Remediation Five Actions Work Leading Group Office along Xuzhou’s traffic trunk lines first collaborated with Zhongshe Aviation, becoming the first city to attempt using the “Jiangsu Provincial Road Environment Management Platform.” Zhongshe Aviation strictly followed the relevant indicators of the “Five Actions for Comprehensive Environmental Remediation along Provincial Traffic Trunk Lines” to objectively assess existing law enforcement issues along highways, assisting frontline remediation personnel in targeted efforts. Using the “Jiangsu Provincial Road Environment Management Platform” for comprehensive inspections, the coverage was extensive and the inspection efficiency high, completing video recordings of 1,012.6 kilometers of ordinary national and provincial roads and 307.1 kilometers of highways within 15 days, identifying 2,270 law enforcement issues in the road environment. By effectively combining text, numbers, and real-world visuals from multiple data types, a “three-in-one” real-time visual electronic ledger was established, showcasing Xuzhou’s remediation data in one map and determining the remediation task list for each county (city, district), providing data support for implementing a dynamic tracking assessment management approach.
Since the successful pilot application in Xuzhou, Zhongshe Aviation has subsequently provided services to 11 cities including Nanjing, Wuxi, Xuzhou, Changzhou, Suzhou, Nantong, Lianyungang, Yancheng, Zhenjiang, Taizhou, and Suqian, collecting a total of 12,000 kilometers of real-time highway image data and identifying over 70,000 law enforcement issues in the road environment, effectively assisting frontline personnel in implementing precise remediation and helping management personnel supervise issue resolution. The platform played a crucial role in the provincial assessment and acceptance of the five actions, ensuring standards were publicly fair, processes transparent, and results just. The real-time environmental images and acceptance results displayed on electronic maps became the sole basis for the acceptance of ordinary national and provincial roads and highways in 13 municipalities. In preparation for the national evaluation work, the platform will continue to enhance the supervisory management capabilities of the “Eight No’s” of highways and assist in the informationization and intelligent management of highways.
Future Outlook: Cutting-edge Applications of Artificial Intelligence
Zhongshe Aviation is currently developing AI-based intelligent rapid inspection products for highway surface defect detection, leveraging rapidly advancing machine vision technology. The AI rapid inspection product connects multifunctional road condition detection vehicles with traditional human-vehicle inspection modes, enhancing maintenance inspection efficiency and reducing the difficulty of manual data processing while maintaining high analytical accuracy. This significantly improves the ability of local roads for preventive maintenance while enabling concentrated handling of surface defects through small and medium repair projects and major maintenance projects.
Currently, there are over 730,000 kilometers of ordinary national and provincial trunk roads and over 3.96 million kilometers of rural roads nationwide, with unprecedented demand for daily maintenance inspections. The AI rapid inspection product timely provides a complete set of standardized operational procedures for intelligent inspections, highly aligning with industry needs and easily replicable for promotion. The AI rapid inspection product is capable of providing services for grassroots highway management units in preparation for national evaluations, effectively supporting the “13th Five-Year Plan” for national trunk road maintenance evaluation work.
This article is selected from
“China Highway” magazine, Issue 19, 2020
For more content,
please subscribe to “China Highway” magazine
Subscription phone: 010-84990501
Editor: Miao Tingjie
Review: Yu Dapeng Fan Yuanyuan
If there is any infringement, please contact the editor for deletion.
Contact: 18701334157
This article is organized and edited by the New Media Development Center of “China Highway” magazine, please indicate the source at the beginning of the articleReprinted from the WeChat public account “China Highway”.
