Virtual keyboard

Virtual keyboard on a Pocket PC PDA

The four main approaches to enter text into a PDA were: virtual keyboards operated by a stylus, external USB keyboards, handwritten keyboards, and stroke recognition. Many early PDAs were not primarily focused on virtual keyboards. Microsoft's mobile operating system approach was to simulate a complete functional keyboard, which resulted in a slightly overloaded keyboard layout.[12] The main problem that early PDAs faced was support for multi-touch technology, and as a result, usability problems for the user.

When Apple presented the first iPhone in 2007, the decision not to include a physical keyboard was seen as a detriment to the device.[13] But Apple brought the multi-touch technology into their new device, which enabled them to overcome the usability problems of PDAs. Apple's virtual keyboard design pattern has become a standard on mobile devices today.

Android Keyboard Settings

The Android SDK provides a so-called InputMethodService.[14] This service provides a standard implementation of an input method, which final implementations can derive from and customize, enabling the Android development community to implement their own keyboard layouts. The InputMethodService ships with it on Keyboard View.[15] While the InputMethod Service can be used to customize key and gesture inputs, the Keyboard Class loads an XML description of a keyboard and stores the attributes of the keys.[16]

As a result, it is possible to install different keyboard versions on an Android device, and that the keyboard is only an application, most frequently downloaded among them being Gboard and SwiftKey; a simple activation over the Android settings menu is possible.[17]

Apple iOS UIInputViewController

Apple also provides the possibility for the community to develop custom keyboards, but does not give any access to the dictionary or general keyboard settings. Further iOS is automatically switching between system and custom keyboards, if the user enters text into the text input field.[18][19]

The UIInputViewController is the primary view controller for a custom keyboard app extension. This controller provides different methods for the implementation of a custom keyboard, such as a user interface for a custom keyboard, obtaining a supplementary lexicon or changing the primary language of a custom keyboard.[20]

Next to the classic virtual keyboard implementation Android, iOS and custom keyboards, such as SwiftKey for example, are providing different features to improve the usability and the efficiency of their keyboards.

The Android platform offers a spelling checker framework that offers the possibility to implement and access spell checking in the application itself. The framework is one of the Text Service APIs offered by the Android platform. Based on provided text, the session object returns spelling suggestions generated by the spelling checker.[21][22]

iOS is using the class UITextChecker, an object used to check a string (usually the text of a document) for misspelled words, commonly known as Apple's autocorrection. UITextChecker spell-checks are using a lexicon for a given language. It can be told to ignore specific words when spell-checking a particular document and it can learn new words, which adds those words to the lexicon.[23]

Users may be able to add a custom dictionary of whitelisted terms that are treated by auto correction as usual words, and specify "aliases" or "text shortcuts", where entering a specified text string causes it to get replaced with a target text string, or the target text string appears as suggestion. The former means they are not replaced with other terms but may be corrected to from other terms. It may be possible to exclude unwanted existing suggestions.[24][25]

Diverse scientific papers at the beginning of the 2000s showed even before the invention of smart phones, that predicting words, based on what the user is typing, is very helpful to increase the typing speed.[26][27] At the beginning of development of this keyboard feature, prediction was mainly based on static dictionaries. Google implemented the predicting method in 2013 in Android 4.4. This development was mainly driven by third party keyboard providers, such as SwiftKey and Swype.[28] Both provide powerful word search engine with corresponding databases.[29] In 2014 Apple presented iOS 8[30] which includes a new predictive typing feature called QuickType, which displays word predictions above the keyboard as the user types.

iOS and Android allow developers to replace its keyboard with their own keyboard apps. This has led to experimentation and new features, like the gesture-typing feature that's made its way into Android's official keyboard after proving itself in third-party keyboards. Research by Google itself confirmed that gesture-typing is increasing the typing rate by 22% and is decreasing the error rate near to 0%.[31] Google further showed that the gesture-typing method is also useful on smart watches. Their scientific research is primarily based on research made by I. Scott MacKenzie and papers about modeling finger touch with fitts' law.

Haptic feedback provides for tactile confirmation that a key has been successfully triggered i.e. the user hears and feels a "click" as a key is pressed. Utilising hysteresis, the feel of a physical key can be emulated to an even greater degree. In this case, there is an initial "click" that is heard and felt as the virtual key is pressed down, but then as finger pressure is reduced once the key is triggered, there is a further "unclick" sound and sensation as if a physical key is respringing back to its original unclicked state. This behaviour is explained in Aleks Oniszczak & Scott Mackenzie's 2004 paper "A Comparison of Two Input Methods for Keypads on Mobile Devices" which first introduced haptic feedback with hysteresis on a virtual keyboard.[32]

An optical virtual keyboard was invented and patented by IBM engineers in 1992.[33] It optically detects and analyses human hand and finger motions and interprets them as operations on a physically non-existent input device like a surface having painted keys. In that way it allows to emulate unlimited types of manually operated input devices such as a mouse or keyboard. All mechanical input units can be replaced by such virtual devices, optimized for the current application and for the user's physiology maintaining speed, simplicity and unambiguity of manual data input.

The basic idea of a virtual keyboard in an augmented reality environment is to give the user a text input possibility. A common approach is to render a flat keyboard into the augmented reality, e.g. using the Unity TouchScreenKeyboard. The Microsoft HoloLens enables the user to point at letters on the keyboard by moving his head.[34]

Another approach was researched by the Korean KJIST U-VR Lab in 2003. Their suggestion was to use wearables to track the finger motion to replace a physical keyboards with virtual ones. They also tried to give an audiovisual feedback to the user, when a key got hit. The basic idea was to give the user a more natural way to enter text, based on what he is used to.[35]

The Magic Leap 1 from Magic Leap implements a virtual keyboard with augmented reality.[36]

The challenges, as in augmented reality, is to give the user the possibility to enter text in a completely virtual environment. One big issue is that most augmented reality systems on the market are not tracking the hands of the user. So many available system provide the possibility to point at letters.[37]

In September 2016, Google released a virtual keyboard app for their Daydream[38] virtual reality headset. To enter text, the user can point at specific letters with the Daydream controller.[39]

In February 2017, Logitech presented experimental approach to bring their keyboards into the virtual environment. With the Vive Tracker and the Logitech G gaming keyboard it is possible to exactly track every finger movement, without wearing any type of glove. 50 of such packages were send to exclusive developers, enabling them, in combination of Logitech's BRIDGE developers kit, to test and experiment with the new technology.[40][41]