Industrial augmented reality

Although the origins of augmented reality dates from the 1960s, when Ivan Sutherland created the first head-mounted display [9] it did not gain strength until the early 1990s, when the David Mizell and Thomas Caudell developed the first industrial AR at Boeing. They used a head-mounted display (HMD) to superimpose a computer-generated diagram of the manufacturing process with a real-time world registration and the user's head position calculation. They coined the name augmented reality for this technology.[2][10]

Contemporary several prototypes were proposed to demonstrate AR's application to manufacturing:[11] a laser-printer maintenance application was proposed in 1993 by Steven K. Feiner and coauthors by introducing the concept of knowledge-based AR for maintenance assistance (KARMA).[12][13] Whitaker Ross et al. proposed a system to display the name of the part pointed by the user in an engine.[14]

By the 2000s, the interest in AR had grown considerably. Some important groups were funded:[10] the largest consortium for IAR backed by the German's Federal Ministry of Education and Research (ARVIKA), with the aim of researching and implementing AR in relevant German industries,;[15] the European Community founded several projects including Service and Training through (STAR), which is a collaboration between institutes and companies from Europe and the US,[16] and Advanced Augmented Reality Technologies for Industrial Service Applications (ARTESAS) derived from ARVIKA, focused on the development of AR for automotive and aerospace maintenance.[17] Likewise, from other countries such as Sweden, Australia and Japan with the aim of encourage the IAR development.

From the beginning of 2010 until today, advances in hardware devices such as the wearable Google Glass, the reduced cost of mobile devices, and the increasing user familiarity with this technology.[7] Besides the increasing product-development complexity where products are becoming more versatile and intricate, with multiple variations and mass customization. Opened new scenarios for this technology.[18]

Industrial part visual characteristics duplex ball valves

One of the most promising fields of AR application is industrial manufacturing, where it can be used to support some activities in product development and manufacturing[19] through providing information available to reduce and simplify the user's decisions.[20] The general issues of the development of an AR system can still be classified into:[21]

MannGlas and GoogleGlass1

There are technologies needed to build AR systems. Some of them are directly related with the performance of the software and hardware that enable the deployment of AR, such as displays, sensors, processors, recognition, tracking, registration among others.[21] Thus AR uses different approaches to integrate the virtual and real worlds, where several technologies influence the usability and applicability.[22]

Some common unsolved issues concern tracking systems suited for industrial scenarios which mean: poorly textured objects with smooth surfaces and strong light variation; object recognition using natural features when it is not possible to use markers;[7] the improvement of accuracy and latency of registration,[4] and 3D context scene capture to allow context awareness.[6]

The limited understanding of human factors is likely to be obstructing the spreading of IAR beyond laboratories prototypes.[4] Their study is challenged to overcome technological issues (deficiencies in resolution, field of view, brightness, contrast, tracking systems, among others) in order to separate the AR performance from interface factors and technological issues.[23]

It was suggested by[10] that for an IAR application to be successful in a commercial environment, it has to be "user friendly", meaning that it needs to be easy and safe to setup, learn, use and customize and the user should feel free to move with an AR system.[8] As well as the use of natural interfaces in order to control AR by using natural movements of the body have also motivated a good deal of research. The reason for this is that the usability not only depends on the system's stability but also on the control interface's quality.[22]

Further, the user interface should avoid overloading the user with information and prevent over relying on it in order to avoid losing important cues from the environment.[24] Other issues are also related with improving multiple user collaboration[6]

It is the final challenge, given an ideal AR system (hardware, software and an intuitive interface) to be accepted and become a part of a user's daily life.[21]

Consequently, one of the most important factors related with the adoption of any new technology is the perception of usefulness, and AR needs to show a clear cost-benefit relation.[25] Some studies suggest that, in order for AR to be perceived as useful the task should be high enough to require its use.[26]

Other non-addressed but important issues for technology acceptance are related with fashion, ethics, and privacy.[21]

Assembling is the process of putting together several separate components in order to create a functional one. It can be performed in different stages of the product's life.[27] Even though nowadays many assembly operations are automated, some of them still require human assistance as, in many cases, their bits of information are detached from the equipment. Thus it is necessary to alternate their attention which leads to decreasing productivity and increasing of errors and injuries[18]

The use of AR is encouraged by the premise that instructions might be easier to understand if instead of being available as manuals they are super imposed upon the actual equipment.[11] Some of the uses of AR in the support of the assembly can be categorized into:[6]

• Assembly guidance
• Assembly training
• Assembly simulation, design and planning

Similarly, by using AR it is possible to simulate the user's motion during assembly to acquire an accurate and realistic movement of virtual parts.[28]

On the other hand, some of the critical issues of the support assembly task are related with the dynamic reconfiguration of the state diagram, which allows to automatically identify the step of assembly and also adapt to unexpected actions or errors of the user.[6][7] Thus, defining 'what', 'where', and 'when' to display information becomes a challenge since it requires a minimum understanding of the surrounding scene.[29]

Like assembly, maintenance serves as a natural application for AR because it requires keeping the user's attention on a specific area and the synthesis of additional information such as complex sequences, component identification, and textual data.[30]

Furthermore, the efficiency and speed of maintenance processes can be improved through AR by quickly displaying relevant information about an unfamiliar piece of equipment to a technician.[31] Similarly, AR can support maintenance tasks by acting as an "x-ray" like vision, or providing information from sensors directly to the user.[24]

It can also be employed in repairing tasks. For instance, in the diagnosis of modern cars whose status information can be loaded via a plug-like connector. AR can be used to immediately display the diagnosis on the engine[3]

Many industries are required to perform complex activities that needs previous training. Therefore, for learning a new skill, technicians need to be trained in sensorimotive and cognitive of sub-skills that can be challenging. This kind of training can be supported by AR.[32]

Additionally, the possibility to use AR to motivate both trainees and students through enhancing the realism of practices has been suggested.[33] By providing instructions with an AR, the immediate capability to perform the task can be also accomplished.[34]

Other advantages of using AR for training are that students can interact with real objects and, at the same time, have access to guidance information, and the existence of tactile feedback provided by the interaction with real objects.[32]

By displaying information about the components of manufacturing in real time. For instance, Volkswagen has used it to verify parts by analyzing their interfering edges and variance. AR has also been used in automobile development to display and verify car components and ergonomic test in reality. By superimposing the original 3D model over the real surface, deviations can easily be identified, and therefore sources of error can be corrected.[35][36]

The use of AR have been proposed to interact with scientific data in shared environments, where it allows a 3D interaction with real objects compared to virtual reality, while allowing the user to move freely in the real world.[37] Similar systems for allowing multiple user's with HMD can interact with dynamic visual simulations of engineering processes[38]

In the same way, AR simulation of working machinery can be checked from mobile devices as well as other information such as temperature and, time of use, which can reduce worker's movements and stress.[19]