Anti-Covid Glasses.Device with camera and sensors can help people with visual impairments to protect themselves from contagion by the new coronavirus.A system consisting of a sensor kit that can be attached to any type of glasses and a processor that uses artificial intelligence could help visually impaired people to move safely during the Covid-19 pandemic. With an efficiency of 95% in the testing phase, the accessory provides three fundamental pieces of information for those who see nothing or almost nothing to prevent the Sars-CoV-2 virus. The device informs you if the user is less than 1.5 meters (m) away from someone; if the person around you wears a mask correctly; and if her body temperature is altered. In Brazil, about 6 million individuals with low vision and 500,000 blind people could potentially benefit from innovation.
The project is part of the professional master’s degree in software engineering that the industrial mechatronics and software engineer from Ceará, 38 years old, Sandro Mesquita, is about to complete at the Center for Advanced Studies and Systems of Recife (Cesar), a non-profit research institution Porto Digital, one of the country’s leading innovation ecosystems, located in Recife ( see Pesquisa FAPESP n the 299 ).
The device, which is in the final stage of development, consists of three parts. The first is a device with a microcamera and two sensors (infrared and thermal), designed to be attached to the stem of an ordinary pair of glasses. Weighing 80 grams (g), it is shaped like a rectangular box 10 centimeters (cm) long, 4 cm wide and 3 cm high.
The Brazilian prototype consists of a kit with camera and sensors ( blue box ) and a processor ( white and red device ) that detects if there are people nearby without a mask and with feverSandro Mesquita
The second component consists of a tiny Raspberry Pi 4 computer and a lithium battery. The size of a small cell phone, the Raspberry Pi receives via Wi-Fi the data generated by the camera and sensors and does its processing. The set, weighing less than 500 g, was designed to be installed in a pouch, of the pouch type, that goes around the user’s waist.
Finally, the system has a wireless headset, which captures signals processed in the bag via bluetooth, and alerts the user to risky situations ( see infographic on the page below ). “We use artificial intelligence resources to process the information captured by the microcamera and sensors, which enable the identification of faces and the correct use of a mask”, explains Mesquita.
A similar technology, however, with fewer resources, is being developed at the Karlsruhe Institute of Technology in Germany. The German prototype uses a commercial glasses model, the Chinese KR-Vision, combined with an RGB-D camera, similar to Kinect technology, used by the Xbox video game console to capture the user’s movements and transmit them to the action of the games. While reading the colors (RGB system), differentiating human objects beings, D stands for the camera has a sensor capable of making a depth reading ( depth , in English), informing the blind proximity to other individuals.
The German system, however, does not provide information about the use of a mask or the body temperature of those around you. It is bigger and heavier than the kit developed by Mesquita. The glasses with the attached camera weigh almost 400 g and the processor and battery, accommodated in a backpack on the back, about 3 kilos (kg). “The next step is to miniaturize our prototype, replacing the laptop, where the information is processed, by a portable device, the size of a cell phone,” said Pesquisa FAPESP, mechatronic engineer Manuel Martinez Torres, who leads the German initiative.
Adjusting the focus
Started in 2018, the Brazilian project was born with another focus. “It all started with my high school students in electrotechnics at the Pedro Queiroz de Lima State School of Professional Education, in Beberibe, Ceará. We built a pair of glasses with common sensors to help blind people to identify obstacles along the way, such as poles or garbage cans. When one of them was detected, the glasses vibrated and beeped ”, remembers Mesquita.
The following year, when he started his master’s degree at Cesar, he decided to continue the development of the glasses, with the partner of Tiago Diógenes de Araújo, Mesquita’s student in the computer science course at the Federal Institute of Education, Science and Technology of Ceará. When the pandemic broke out, the researcher redefined the scope of the project. “One day I heard a health authority on TV directing the population to keep a distance of 1.5 m from other people and to avoid approaching anyone who did not wear a mask or was sneezing. I thought, how does a blind man observe these things? There I saw the opportunity to change the direction of work ”, remembers Mesquita.
Although the Cesar prototype showed good results in the tests, lawyer Marcelo Panico, from the Dorina Nowill Foundation for the Blind, from São Paulo, warns of a challenge that needs to be considered in the creation of assistive technologies for the blind. “People who are blind or have low vision and learn OM techniques [Orientation and Mobility] should be very attentive, especially on the streets, to the other senses. The excess of information, instead of helping, can make it difficult to understand the space ”, he says.
The German model uses glasses with a camera attached and the processing is done by a laptop carried in the backpackReproduction of the article “Helping the blind to get through Covid-19: Social distancing assistant using real-time semantic segmentation on RGB-D video
Panico, who lost his sight in 2003, when he was 33, uses his own experience to illustrate the argument. “I myself have participated in tests with glasses that identify faces and have worked well indoors, with little noise. However, they proved to be inefficient in open places, with a lot of public, making it difficult for me to understand the space where I was. ”
Mesquita says he knows these difficulties. “The individual who cannot see usually has a very sharp ear. The alerts coming from the device compete with the hearing of one of the ears, hindering its orientation ”, recognizes the engineer. “A solution to this problem could be to test a phone that emits sound through bone conduction. The accessory is in contact with the skin, but outside the ear, without obstructing the user’s ear canal. Thus, the ear that receives the messages can pick up the sounds of the environment without interference. ” This solution has not yet been developed by him.
At the same time that there are challenges to improve the invention, there is potential for technologies developed for glasses to have other purposes, with emphasis on facial reading attesting to the correct use of the mask. “The algorithms for facial recognition, programmed to look for a mask on the person’s face, are already used, for example, to allow people to enter Cesar during the pandemic”, explains software engineer Victor Hazin da Rocha, Mesquita’s advisor . “The technology can also be used in cameras to monitor the use of masks in different environments, such as supermarkets, airports and parks.”
Mesquita’s next challenge is to develop specific sensors for each function of the system and integrate them into the same circuit, also designed exclusively for the device – today, the prototype uses generic sensors. To this end, it hopes to obtain support from funding agencies or private companies. Then, it plans to make the kit production on a large scale possible. “Our ultimate goal is to develop social technology at a more affordable cost, around R $ 800, and make it available to people with visual impairments who today are not being met in their needs and are facing difficulties because of the Covidean pandemic. 19. ”