Many iPhone users prefer to interact with a virtual QWERTY keyboard placed like a hologram on a table instead of the phones with QWERTY keyboards that is presently available on their device. These virtual keyboards are the same size as a desktop computer's QWERTY, giving its user more room to type. The text input performed by smartphone users would be quicker and more precise. What if you spritz something on your hands and start typing quickly without a keyboard? That is the concept behind a new spray-on smart skin that enables you to type and operate apps on your computer and phone even without touching them.
How to Type Without Keyboard?
The researchers of Stanford University have developed an inventive spray-on smart skin that uses AI to detect what you'll type on a computer keyboard, building on the work of MIT researchers who created an electronic skin that can sense signals. This biocompatible new skin spray solution may be sprayed on your hand like sunscreen and an electrical network to detect skin stretches and movements. Machine learning assists in detecting where you want to type on the keyboard, track the physical gestures, and engage with apps. This outbreaking technology may revolutionize the way we engage with multiple technologies in the future.
The chemicals in the skin smart spray are stretchy honeycomb-like nanomesh that is flexible and easy to move with the skin. It comes with electrical sensors that track these motions and transmit them to a smart type AI system for further analysis. Machine learning technology allows fast gesture mapping and helps understand every user's erratic hand movements. Millions of silver and gold-plated nanowires made the mesh and produced electrical paths when contacted. This breathable and biocompatible mesh will remain on the user’s skin until it is cleaned with soap and water. Moreover, this skin smart spray on the skin can sense each finger and its folds and wrinkles. The mesh can also be connected to a portable Bluetooth module to transmit signals wirelessly.
The author stated:
"with the help of machine learning , electronic devices including electronic gloves and electronic skins can track the movement of human hands and perform tasks such as object and gesture recognition.”
Techniques Used to Make Movements Possible
Several interesting techniques have been intensively researched to enable different hand movements and physical gestures. These techniques include wristbands, monitoring muscle electrical impulses, or wearing gloves. These gadgets are expensive because it takes several sensory systems to detect motions at every joint. Due to these problems, it was inconvenient to use this old technology.
ThinkML
Why Spray-On Smart Skin Has Gain Popularity?
There are several reasons of its popularity including:
- This spray-on skin is significantly less heavy, which makes it considerably easier to use than a glove or other e-skins products.
- The mentioned technique is the first one that is adaptive and simple enough to work for virtually any user, even with the least data.
- The number of sensory components of existing techniques need to read each finger joint makes them large. The new device also employs a simpler approach to software to enable rapid learning.
- The proposed technology is significant in VR applications to give the feeling of real experience with high precision.
- This electrically sensitive mesh is embedded in a durable material called polyurethane. Because of its stretching ability, it makes staking wheels and protects woody hard floors.
- Moreover, smart spray technology might be employed in various other contexts, including medical departments where doctors may use robots with spray-on smart skin to perform operations from a distance.
- The researchers have created a technique that works similarly to a wearable Google Lens; the skin can identify different things by only touching them.
It is also true that you will not see a spray bottle of smart skin in front of you in future soon. Further study and experimentation are required to use this product on a commercial scale. So, we must use the same traditional keyboards and phone screens to operate our computers and smartphones.
The Virtual QWERTY and How It Performs?
Let us examine the virtual QWERTY and how it might perform. Based on electrical conductivity patterns and machine learning, the algorithm learns the motion of how to type the letter "X ."The need for an actual keyboard ends when the algorithm becomes "trained" to detect motions for all numbers, punctuation, and letters. These virtual keyboards will be simpler and easier to use than the phone QWERTY keyboard.
According to Zhenan Bao Lee, senior founder of research and a professor of Chemical Engineering:
"As the fingers bend and twist, the nanowires in the mesh get squeezed together and stretched apart, changing the electrical conductivity of the mesh. These changes can be measured and analyzed to tell precisely how a hand, finger, or joint is moving."
Benefits of Spray-On Type of Smart Technology
- You don't need artifacts and substrate for the attachment; you must spray the mesh directly onto your hand's skin.
- Further, only a single trace of conductive mesh is enough to generate the information and data by the movements of all fingers.
According to Kyun Kyu "Richard:
" "The first author of the work is Kim, a post-doctoral researcher in Bao's team, "We brought the aspects of human learning that rapidly adapt to tasks with only a handful of trials known as 'meta-learning.' It allows the device to rapidly recognize arbitrary new hand tasks and users with a few speedy trials."
Kim said:
"Moreover, skin smart spray is surprisingly a simple approach to this complex challenge that means we can achieve faster computational processing time with fewer data because our nanomesh captures subtle details in its signals."
ThinkML
- The spray-on device can adjust to any hand size or shape and can also be modified to the face to collect facial emotions. This technology can birth new computer science techniques and virtual meetups showing more real-like facial gestures and hand movements.
- Using machine learning, computers can link lively tasks and expressions to changing conductivity patterns and identify them and type without a keyboard. For instance, using the moving or changing patterns in electrical conductivity, the algorithm identifies the motion of typing a "Z" on a keyboard. The same rules can be employed to identify sign language or things by tracing their outside surfaces.
- Smart skin technology can learn fast and efficiently with only a little data, in contrast to conventional technologies, which are cognitively demanding and require an enormous amount of data to be trained and interpreted.
- It increases its viability for everyday use with a variety of real-world applications.
- In telemedicine, smart skin technology may be used to examine a patient's hand motions and gestures to diagnose the present condition and track the level of recovery.
- Additionally, this smart technology might be used to examine athletes' movements during sports training sessions to enhance performance.
- The skin smart spray technology has the potential to produce more authentic and engaging virtual reality experiences in the gaming sector.
- While in robotics, it may help industries to produce more intelligent and exceptional robots than the present ones.
The scientists have created a prototype that can identify simple things by touch and even predictively type with both hands while using a non-visible keyboard. The system was able to type without keyboard, "No legacy is so rich as honesty" from William Shakespeare and "I am the master of my fate, I am the captain of my soul" from William Ernest Henley's poem "Invictus."
What is the Expected Future of Spray-On Smart Skin Technology?
The future of Spray-On smart skin technology is very promising.
- This revolutionary technology has the potential to revolutionize the way we interact with our environment and our digital devices.
- Shortly, Spray-On smart skin will create highly sensitive, interactive surfaces that can detect and respond to touch, pressure, and other environmental forces.
- This technology could be used to create interactive walls, furniture, and clothing and to create more intuitive interfaces with digital devices.
- As technology advances, Spray-On smart skin will become even more versatile, allowing for more accurate, precise, and responsive interactions with our environment.
- Ultimately, the future of Spray-On Smart Skin technology will depend on how quickly it can be commercialized and how readily available it is.
The technology is still in its early stages, and it will take some time to become widely available. However, with the right investments and development, it could become an indispensable tool for consumers and businesses in the near future.
Brief Note on Spray-On Smart Skin
This spray smart skin is an innovative new technology that uses artificial intelligence (AI) to enable users to type without a keyboard. The technology works by using a spray-on conductive material applied directly to the skin. The conductive material is then connected to a computer that uses AI to recognize the user's skin movements and convert them into words. It means the user can type without using a keyboard or any other input device. Technology can revolutionize how we interact with computers and could be used for various applications such as gaming, communication, and even medical and healthcare.
The spray on new skin is still in the early stages of development, but the potential applications of this technology are already exciting. It could help people with physical disabilities or impairments who cannot use traditional input devices. It could also help those with mobility issues who may find typing with a keyboard difficult or uncomfortable. The technology could even be used to create a new type of input device that could be used for virtual reality gaming.
Overall, this spray skin is a revolutionary new technology that could revolutionize how we interact with computers and other electronic devices. It has several exciting applications and could help to make technology more accessible and easier to use for everyone.
It is an exciting development in Artificial Intelligence and could open up a world of possibilities for the future of input devices.
ThinkML
Other Examples of Relevant Technology
1. An Invisible AI-Powered Keyboard from Samsung for Your Phone
We now work from our phones more than ever, yet typing lengthy emails, and texts still takes longer than using a keyboard. Samsung has an alternative: it tracks your hands as they type on an unseen keyboard using AI and the camera on your phone. This SelfieType system was an excellent experiment, even in its demo stages. It was introduced during Samsung's C-Lab program, offering amazing ideas for the in-house incubator. Some of these exciting products got commercial-level importance shortly. SelfieType may be a great resource if it functions as similarly as this advertisement claims, but we need to know what strategy Samsung has for it.
However, the typing itself will be the proof. Laser projection already allows for creating "invisible keyboards," but these devices are more of a novelty than a useful utility. They are frequently sluggish and unreliable, so if you're already lugging about a little laser projector, you may go all out and replace it with a reliable Bluetooth keyboard.
SelfieType will at least make bringing around attachments unnecessary because it uses the camera on your phone to take pictures. It sounds challenging to use machine vision to monitor the precise movement of your fingers, and you'll probably need to keep your hands still for things to operate properly, which is only possible with the tactile feedback of a keyboard.
We've never heard of this approach before, but we're intrigued to give it a shot. Engineers have explored a variety of strange keyboard designs throughout the years to replace the tried-and-true keyboard. Maybe AI can help us solve this issue once and for all.
ThinkML
2. Researchers Train an Invisible AI Keyboard Using GPU
A South Korean company has developed an invisible AI-enabled keyboard that expedites one of the most common jobs of our time: reading through messages on mobile devices. Users of the Korea Advanced Institute of Science and Technology's (KAIST) Invisible Mobile Keyboard (IMK) can type 51.6 words per minute while experiencing less pressure.
Users begin typing on the keyboard to begin using it. No keyboard covers the display. Written content is shown in the corresponding text field. Alternatively, users can read a stream of text to verify correctness. Instead of providing users with an on-screen keyboard, IMK relies on the user's recollection of the location of each fictitious key about all the others.
The Self-Attention Neural Character Decoder, which the researchers call the system's brain, was trained on GPUs and fed by a sizable user input dataset.
There are two decoder models in it.
- The touch input sequence is sent into the geometric decoder. It then uses the touch points on the invisible keyboard to translate them into a character sequence.
- The semantic decoder then uses semantic interpretations to rectify decoding mistakes in the character sequence calculated by the geometric decoder.
The One Billion Word Benchmark, developed by Cambridge University, the University of Edinburgh, and Google in 2014, served as the training ground for the semantic decoder.
The two models were then adjusted jointly by the researchers. Both were trained using a GeForce GTX 1080 Ti GPU and the free PyTorch 1.4.0 framework. Whether utilizing applications in landscape or portrait mode, users may type. Users can enter text on the entire screen in any case. Invisible Mobile Keyboard users could type 157.5% faster than those using third-party soft keyboards on their cell phones.
Conclusion
The Stanford team has created a learning strategy that is significantly more computationally efficient to address this demanding computing challenge and the requirement to calculate enormous data. There are several uses for this technology, including new computer animation methods and avatar-led virtual meetings that feature more lifelike facial expressions and hand movements. However, this virtual spray skin isn't limited to operating mobile phones with keyboards. Researchers claim that it can even be used to operate VR applications and smartphone apps. While it may not allow you to feel objects in virtual reality, it looks nicer on your hands than Facebook's VR Glove.
Furthermore, if you can better control augmented reality, technology like this might pave the way for many new advancements, not only in the area of the mobile device interface. Apple's intention to direct AR visuals right into your eyes might contribute to creating more immersive experiences when coupled with other devices.