artificial intelligence for smart sensors and actuators, m.eng.

Motivation

In infrastructure projects, an essential requirement is to name the individual project documents according to a standardised procedure, e.g. according to VGB-S832. However, the documents created are not always named correctly according to the specifications. In addition, due to the large number of project documents, there are sometimes discrepancies between the document list and the documents actually created.
With the help of AI, in particular Natural Language Processing (NLP), the management of project documents is to be partially automated so that resources are freed up for other activities.

Project Objectives

• Classification of documents
  The document classes are to be recognised independently of the document name
  The recognition of document classes should be applicable to text documents as well as scanned documents and drawings in German and English
• Renaming of files according to VGB-S832, if necessary
• Automatic creation of the document list based on the existing documents and identification of deviations from existing document lists

Approach

• Clean up the existing project documentation, especially clarification of partially wrong classifications and avoidance/reduction of "imbalanced datasets"
• Creation and testing of different classification models with different settings for class recognition
• Selection and use of the best models and implementation of file renaming
• Automatic creation of the deviation list

Computer-assisted chatbots are technical dialogue systems based on natural speech recognition and are used to answer user queries automatically and without direct human intervention in real time.

Project Objectives

A student project group designed an extensive catalogue of questions about studying at Campus Cham. In cooperation with the respective university departments, the project group defined correct answer patterns for the questions asked and fed them into the system. The technical application recognises the user input, compares the predefined answer patterns and should, for example, help prospective students navigate better through the wealth of information on the website in the future.  

Images

• chatbot dialogue box

Project Overview

With increasing traffic across the globe and the number of vehicles, traffic supervision has become complex. Artificial intelligence (AI) technology can reduce the complexity as well as increase the throughput.

The project “Traffic Supervision System’’ was initiated in the Sensor Lab, Campus Cham. The objective was to build a prototype “Watch box” (Integrating AI Hardware with object detection software). It can monitor the traffic as well as vehicle features in real time. The project plan is to connect a centralised system through cloud direct connect.  

The YOLOv5 object detection model is used for training of vehicle classification and its manufacturer.  Deep sort algorithm is used for tracking and counting (figure 1). Figure 2 shows the video capture by the camera. The vehicle characteristics viz type, count, manufacturer and color as well as the time instance has been stored in excel file (Figure 3). The model has been evaluated on the test data with an accuracy of 89%. The AI hardware used are Jetson Nano and camera (Figure 4). The watch box design for the hardware is shown in Figure 5.

Images

• Figure 1: object detection model and the results are stored in Excel file

• Figure 2: video capture by the camera.

• Figure 3: results stored in an Excel file.

• Figure 4: AI hardware used is Jetson Nano and camera.

• Figure 5: watch box design.

Keywords

Traffic Supervision, Artificial Intelligence, Machine Learning, YOLO Object Detection Model, Jetson Nano

The project deals with the detection of the direction of gaze on a specific object with simultaneous object recognition. As a preliminary stage to this application, the determination of the direction of gaze and the recognition of the corresponding object was implemented with the help of a convolutional neural network.

Project Objectives

Where do our eyes wander to first when shopping for our daily needs, where do they linger the longest and what consequently appeals to us the most on the supermarket shelves? How should a supermarket, for example, optimally position or design its product range in order to appeal more to its customers?

The student is always focused on a pair of scissors, when the focus of vision and scissors are recognised by the neural network.

Images

• The gaze of a student is directed at the scissors here. The direction of gaze and the scissors are recognized by the neural network.

Measurement and visual representation of magnetic fields are often necessary in development and production of products in connection with magnets. Among many others, the following can be mentioned as application examples for a "Magnetic Field Mapper":

• Quality control in the production of permanent magnets or products in which permanent magnets are built in (e.g. speakers)
• Checking and verification of computer models of magnetic fields
• Investigation of the manufacturing stability and aging behavior of magnets used for qualification or in the application for magnetic sensors (example: angle sensors in cars)

The project used a commercially available 3D printer (CTC 16450, Table, Fig. 1) where essentially the extruder unit was replaced by a Hall sensor from Infineon Technologies (Hall sensor TLV493D-A1B6). Above all, the 3D printer offers a sufficiently large installation space to be able to investigate larger magnetic fields in later applications, and at the same time a comparatively good positioning accuracy.

Data of the 3D printer:

The movements in x, y and z direction are done by stepper motors. An Arduino Mega 2560 microcontroller controls the stepper motors and receives and processes the sensor data.

• The first application for which the Magnetic Field Mapper is used is the measurement of magnetic fields of diametrically magnetised disc magnets, such as those used for magnetic angle sensors (Fig. 2).

• Abb. 2: Diametrically magnetised disc magnets

For this measurement, a measuring grid of 0.25 mm was set over a field of 14 mm x 14 mm (Fig. 3).

• Fig. 3: Measurement grid and measurement overview

For the visual representation of an x-y measurement at a constant z-value, various display options can be selected. As an example, Fig. 4 shows the course of magnetic field lines in the x-y plane and Fig. 5 the corresponding equipotential curves including a measurement artefact. Figure 6 shows the z-component of the magnetic field directed out of the x-y plane or into the x-y plane.
The Magnetic Field Mapper project will be continued in further project and final work. Examples of improvements to the system would be:

• More flexible graphical representation
• Use of a significantly smaller measurement grid
• Development of a user interface
• Simple exchange of the sensor used

• Fig. 4: Magnetic field lines.

• Fig. 5: Representation of the equipotential lines.

• Fig. 6: Representation of the z-component.

The detection of road damage is essential to maintain road quality for road users. In particular, the detection of the severity of road damage is also important for the authorities to decide where and what to prioritise for repair. In the project "Road Condition Detection", a CNN model for the detection of road damage was created and trained. The project includes a user-friendly interface and the automated creation of a road condition report. The CNN model classifies the condition of the road surface into four categories with two levels of severity. The CNN model further uses images and GPS data from a GoPro dashcam and the weather status as input (Fig. 2).

• Fig. 1: Project overview.

The final dataset is based on a combination of different publicly available datasets with a total of 11,947 images and associated 26,191 labels. By means of augmentation, the training set was increased to over 30,000 images and over 100,000 labels using the online annotation tool Roboflow. Mosaic and cropping augmentation was applied to the entire dataset to increase the number of images from about 10,000 to about 30,000. There was a significantly lower amount of original images of potholes, which is why additional techniques such as colour change, scaling, flipping, translation etc. as well as combinations of these techniques were used (Fig. 2).

• Fig. 2: Augmentation.

For image recognition, the YOLOv5 (You only look once) model was used. YOLOv5 is "open source" and can be used as an efficient starting point for fast and good results. The framework offers five different options. A recognition rate of about 80% was achieved for all classes.

In order to make the road condition recognition as user-friendly as possible, a "user interface" was programmed (Fig. 3).

• Abb. 3: User interface.

After a monitoring run has been completed, the software automatically generates a report on the road condition. The report contains small statistics and shows the roads travelled on a map (fig. 4). In addition, the captured images of the road damages as well as the corresponding exact GPS data are recorded in the report (fig. 5).

• Fig. 4: Report overview.

• Fig. 5: Detailed damage recording.