Ensuring comfortable driving, a traffic sign reminder might be helpful to display the currently allowed maximum speed limit. Andreas Sampl and Michael Teuschler wanted to develop a system that can be built into cars not having such a device on board. So, the main goal was to select the fitting hardware, to write the software and to train an artificial intelligence to get the wanted results. To achieve that they had to prepare and plan the whole system from scratch.
Chosen Hardware
After having done a lot of research which devices could fit best for this application the students decided to use a platform called NVIDIA Jetson Nano (minicomputer with a powerful graphic card running under Linux). One of the reasons is the huge amount of documentation and training material when using this device. To generate an input video stream a camera has been attached to the hardware platform. Furthermore, an LCD screen is connected via HDMI to display the recognized traffic sign which might help the driver to know the actual speed limit. This setup allows a portable, easy to handle device which can be installed in every car in order to assist the driver.
Generating training data
It is obvious that real data had to be available so what could be better than using the journey to FH Kapfenberg to gain these data? In fact, more than 60 GB of video material were recorded for training and testing the artificial intelligence to fulfill the requirements. Furthermore, an algorithm was programmed which produces simulated training data in order to be able to gain even more data for the AI. The programming of the software has been done in Python. Afterwards it has been transferred and executed on the portable device using remote connections like SSH and VNC.
Image classification and testing of the system
Then every picture was assigned to a category. In this use case it is either the traffic sign with the speed limit 100 km/h, 70 km/h etc or it is a non-classifiable picture. This can be achieved by using object detection. The system recognizes a small section of the whole picture which might contain traffic signs and allocates it to the category.
After a few more loops of improving the system the students created a GUI (graphical user interphase) for an easy handling, and to represent the recognized traffic signs on the screen.
Now it’s getting interesting: does the system work well? The system was tested on the road between Kapfenberg and Graz and it worked very well. Of course, some improvements still can be made i.e. recognition of more traffic signs or implementing of a more sensible camera but students and their supervisor are very happy with the outcome of this demanding project.