New AI technology can monitor crowds and pick out particular individuals, by MITSUBISHI.

Mitsubishi Electric develops AI crowd surveillance.

Electronics maker wants to introduce system by Tokyo Olympics.

Mitsubishi Electric has developed a system using artificial intelligence technology that can monitor crowds and pick out particular individuals.

Applications for the system include scanning commercial areas for people acting suspiciously or simply needing assistance.

The Japanese electronics maker said the video-based system utilizes deep learning technology, through which a computer can learn the characteristics of specific objects.

The company hopes to market the system by the 2020 Tokyo Olympics, when huge numbers of visitors are expected to descend on the Japanese capital.

The system monitors video feeds from security cameras and detects individuals that fit predetermined characteristics automatically and in real time. It can detect, for example, someone carrying a suspicious item, such as a plastic fuel can, someone pushing a stroller or an elderly person walking with a cane. It can also recognize specific movements so that it can alert security personnel to someone walking erratically.

One demonstration video shows several people walking. As soon as a person with a fuel can comes into view, the system highlights them with a white rectangular frame.

Hidenobu Kanda, chief of Mitsubishi's security systems department, said the use of deep learning technology makes programming easier to define attributes for people with specific characteristics.

In order to be able to recognize a stroller, the system first memorizes images taken of the object from various angles. Using this knowledge, it can automatically spot someone pushing a stroller passing in front of the security camera. Precision in detecting a specific image improves as the system builds on previous experience.

Conventional image recognition systems, in contrast, need to learn many more details of the objects' characteristics, necessitating complex programming. 

In the case of a person, it would need to first learn that a human pedestrian is an object with a width-to-height ratio of 2-to-8 that moves at a speed of about 20kph at the most, and so on. It would then need to learn the details of the stroller before it can recognize a person pushing it.

Mitsubishi said its system is particularly useful in commercial facilities and event venues, enabling crowd-management staff to come to the aid of people who need assistance or security guards to monitor people acting suspiciously.

Mitsubishi also eyes other applications for the system. In a joint effort with researchers from the University of Tokyo, the company is developing an image-analysis system that predicts how congested different routes between an event venue and the nearest station will become. For this, the company plans to use the AI crowd monitoring system to extract a demographic profile of pedestrians so that organizers can address the different needs of attendees.

Although Mitsubishi aims for a commercial introduction of the system by 2020, some issues remain to be solved.

One is the question of where to set the degree of detection precision amid varied needs expected by customers. Although facility owners will want a high precision when it comes to detecting specific individuals entering a venue, detecting individuals acting suspiciously will require a different level of precision.

Kanda said the company would need to feed more data into the deep-learning system to raise accuracy, but this will result in higher costs.

In terms of the number of images the system needs to learn, Kanda said opinions are divided among the development team with some saying 100 to 1,000 images per object would be enough, while others insist on over 10,000 images.