Computer Vision

Automated image and point cloud data analysis for the smart factory and smart city

Figure 1: Detection of defective components of a combine harvester in a thermal image as part of the “Datenfabrik.NRW” project (https://www.datenfabrik-nrw.de/)
Figure 2: Recognition of road users in a 3D point cloud for intelligent traffic light control as part of the “KI4PED” project (https://www.iosb-ina.fraunhofer.de/de/aktuelles_news/2021/ki4ped.html)

Image and point cloud data represent the environment in the form of 2- or 3-dimensional data. For example, color cameras generate visual (human-visible) 2D color images, thermal imaging cameras capture thermal signatures in the environment (see Figure 1), and 3D LiDAR sensors use laser beams to scan their environment in the form of 3D point clouds (see Figure 2).

 

It is often very easy for a human to extract higher-value information from this rich image and point cloud data and thus generate added value. However, this is very time-consuming and costly and requires expert knowledge. We support the automation of this task using state-of-the-art computer vision processes to enable innovative applications and thus save you costs and valuable staff time in the medium term. For example, we use 2D colour images to automatically assess the quality of a workpiece in production, reveal energy potential in systems using thermal images and detect pedestrians in compliance with the GDPR for optimized traffic light circuits in our cities of tomorrow.

Our services focus on the two application areas of Smart Factory and Smart City.

Our goal:

Automation of tasks using a combination of cameras (or camera-like sensors) and modern computer vision processes in the fields of industry (e.g. visual quality control) and smart cities (e.g. environmental monitoring)

Our range of services:

  • Potential and feasibility analyses for the integration of systems for the automatic evaluation of image or point cloud data in your company or city
  • Conceptual design of systems for image or point cloud evaluation from data acquisition to provision
  • Manufacturer-independent use of components without vendor lock-in
  • Implementation of algorithms on different hardware platforms, e.g. embedded hardware for edge computing based on commercial and open source toolchains
  • Training your employees on the topic of „AI-based optical quality control“
  • Participation as an application-oriented research partner in research and development projects in the consortium with a focus on computer vision for embedded hardware

Core benefits for companies and municipalities:

  • Smart Factory: Automation of quality inspections in production
    • Increase in production quality
    • Cost savings through reduction of returns
    • Increased efficiency in production through faster and more precise processes
    • Competitive advantage through the use of state-of-the-art technologies
    • Better traceability and documentation of production processes
    • Scalability of solutions for growing production requirements
    • Reduction in training costs for employees thanks to intuitive AI-supported system
Figure 3: Optical quality inspection
  • Smart City: Real-time monitoring of vehicles in road traffic
    • Vehicle counting to determine the volume of traffic
    • Lane-based traffic flow detection as the basis for intelligent traffic light systems
    • Determining the occupancy of parking spaces
    • Optimization of traffic flow through intelligent traffic light control
    • Greater acceptance of urban mobility solutions thanks to modern GDPR-compliant technologies
    • Positive impact on the cityscape and citizens' quality of life
    • Data-based decision-making for urban planning and development
    • Integration into existing urban infrastructures for seamless implementation
Figure 4: Real-time vehicle counting

Equipment:

  • SmartFactoryOWL with imaging sensors for developing and piloting applications for object recognition and tracking
  • Mobile robot station with video-optical camera and safety laser scanner for implementing image-based handling applications in the production environment
  • SmartCityBox for implementing imaging applications in the smart city
  • Wide range of imaging sensors: video-optical industrial cameras, 3D stereo cameras, infrared/thermal cameras and solid-state LiDAR
  • Ring and area lights for targeted illumination of test objects
  • Mounting systems for temporary inline integration of inspection systems in production facilities
  • All-in-one inspection station for optical quality control for training purposes

References / Publications:


  • Zamboni, Pedro Alberto Pereria; Hendrickx, Hanne; Sprute, Dennis; Flatt, Holger; Rushdi, Muhtasimul Islam; Brodrecht, Florian; Eltner, Anette: Synergistic image and point cloud processing of UAV data for urban flood modeling: point cloud smart thinning and curb mapping. In: The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLVIII-2/W8-2024 8th International ISPRS Workshop LowCost 3D - Sensors, Algorithms, Applications, Brescia, Dec. 2024
  • Wittke, Christian; Liebert, Artur; Friesen, Andrej; Flatt, Holger; Niggemann, Oliver: Potato-Glow: Utilizing Glow for Vision-Based Anomaly Detection in an Industrial Context: A Comparative Benchmarking Approach. In: Bildverarbeitung in der Automation (BVAu), Lemgo, Nov 2024
  • Basile, Dennis; Sprute, Dennis; Dörksen, Helene; Flatt, Holger: Evaluation of 3D-LiDAR Based Person Detection Algorithms for Edge Computing. In: Forum Bildverarbeitung 2024, Karlsruhe, Nov. 2024
  • Senke, Hanna; Sprute, Dennis; Büker, Ulrich; Flatt, Holger: Deep Learning-Based Localisation of Combine Harvester Components in Thermal Images. In: Forum Bildverarbeitung 2024, Karlsruhe, Nov. 2024
  • Sprute, Dennis; Hufen, Florian; Westerhold, Tim; Flatt, Holger: 3D-LiDAR-based Pedestrian Detection for Demand-Oriented Traffic Light Control. In: IEEE 21st International Conference on Industrial Informatics (INDIN), 2023
  • Sprute, Dennis; Westerhold, Tim; Hufen, Florian; Flatt, Holger; Gellert; Florian: DSGVO-konforme Personendetektion in 3D-LiDAR-Daten mittels Deep Learning Verfahren. In: Bildverarbeitung in der Automation (BVAu), Nov 2022
  • Gutknecht-Stöhr, M.; Friesen, A.; Flatt, H.; Habeck, T.; Großehagenbrock, J.: Automatisierte Qualitätskontrolle: Kartoffeln, KI und Roboter. Atp magazin 10/2019, S. 72 ff, 2019

Reference projects:

 

I4KMU-FlexRoG

 

Datenfabrik.NRW: Thermal image-based anomaly detection on combine harvesters

 

KI4PED

 

KI4LSA

 

Image-based quality inspection processes for laminated safety glass

 

Evaluation of self-learning optical methods for quality control

 

Pedestrian frequency measurement in real time

 

UAV-SRGK