Pedestrians in focus

Crosswalks with traffic lights are intended to ensure safe and comfortable crossing of busy roads. According to current studies, however, the green phases do not take into account the required crossing times of different groups of people. With current technology, the detection of crossing intent at simple traffic signal installations (LSA) is done by a pushbutton that has to be actuated by pedestrians: This type of demand request provides little information about the number or needs (e.g., vulnerable persons) of pedestrians, which allows for individual optimization of switching times. In addition, the length of green phases is said to have an impact on the willingness not to cross these streets during a red phase, which may be safety relevant. The research partners hope that the success of the project will lead to a demand- and situation-oriented control concept that can reduce the waiting time for high volumes of people by an average of 30% and the number of dangerous crossings in violation of traffic regulations by up to 25%.

Challenges / research questions

Against this background, the manufacturer of light signal systems, Stührenberg GmbH from Detmold, and the developers of Fraunhofer IOSB-INA from Lemgo have asked themselves the following research questions, among others, for their new project: How can lighting-independent and DSGVO-compliant person recognition and tracking based solely on LiDAR sensors be achieved using AI and implemented in an embedded system in real time? What waiting time reduction is possible at pedestrian LSAs with high volumes of people without holding up car traffic for too long? How can an improvement of traffic safety at pedestrian LSA be achieved by means of AI, especially for vulnerable persons? How can an integration into existing traffic lights be carried out cost-effectively as a so-called "retrofit"? The state of North Rhine-Westphalia already describes the necessity of a corresponding optimization in its Road Safety Program 2020.

Solution approach with AI

Intelligent, optimizing traffic light control systems could react dynamically to pedestrians, reduce waiting times and at the same time contribute to traffic safety. In its search for a technology to meet this challenge, Fraunhofer IOSB-INA intends to develop, among other things, an algorithm for intelligent image recognition in this project. While the LiDAR method generates data in a completely data-protection-compliant and anonymous manner, not only people and groups of people but also baby carriages, wheelchair users and other road users are to be recognized on this basis and their crossing intentions evaluated. The chosen approach should be robust to the greatest possible extent with regard to lighting influences and harmless due to a DSGVO-compliant overall system approach. Furthermore, it should be a cost-effective alternative to other approaches. The robustness refers especially to adverse conditions (e.g. backlight, darkness, reflections) as well as the saving of resources in the implementation.The self-imposed goal: lighting-independent person recognition and tracking using AI, based exclusively on LiDAR sensors, with a recognition performance of at least 90%. Carsten Fischer, Managing Director of Stührenberg GmbH, is looking forward to the new research project: "The introduction of an AI traffic control system is also a first and big step for our company. AI marks the beginning of a new and modern era in the field of road traffic technology, which will bring many more changes in the future."

Differences to previous approaches

Previous approaches for optimized control of pedestrian LSAs, which are often based on video-optical sensors, have proven to be disadvantageous with regard to usability in adverse lighting conditions as well as acceptance due to data protection that is not visible from the outside. Current AI methods are often biased in their recognition quality, since mainly images from Western culture are used for training. This can lead to unintentional discrimination against people of color, who are not recognized by the AI algorithm because of their skin color. The desired solution should be largely independent of lighting influences, DSGVO-compliant by design, easily retrofittable as an embedded system in existing LSAs, and non-discriminatory.

Next steps

Initially, the research partners will conduct various sensor tests on the Fraunhofer IOSB-INA site under different, if necessary, simulated lighting conditions to determine detection performance. In the implementation phase, the Fraunhofer Institute and Stührenberg GmbH will record data with LiDAR sensors at selected pedestrian traffic lights in Lemgo and Bielefeld and develop an AI-based prediction of people's intention to cross on this basis. At the same time, the integrability into existing traffic light systems will be investigated. As part of the evaluation, the suitability for intelligent control at the above-mentioned locations will be verified and the robustness to different lighting conditions will be investigated. Project manager from Fraunhofer IOSB-INA, Dr. Dennis Sprute is looking forward to the cooperation: "With Stührenberg we once again have an experienced and competent partner at our side - and with Lemgo and Bielefeld progressive and open-minded venues. We are particularly pleased that the project is also supported by Straßen.NRW. One can hardly wish for much better framework conditions for such a project." Director of Fraunhofer IOSB-INA and project initiator, Prof. Dr.-Ing. Jürgen Jasperneite, emphasizes the potential benefits of the technology: "With these 'thinking' traffic lights, we can make an important contribution to increasing pedestrian safety and traffic flow."

About the BMVI's mFUND:

As part of the mFUND innovation initiative, the BMVI has been funding data-based research and development projects for digital and connected mobility 4.0 since 2016. Project funding is supplemented by active professional networking between stakeholders from politics, business, administration and research and by making open data available on the mCLOUD portal. For more information, visit www.mfund.de.