Towards smarter aircraft interfaces with Virtual Reality

Virtual Reality For The Aircraft Industry
Today, aeronautic and aerospace industries want to reduce product development times and costs but face a major obstacle: the need to build several physical prototypes for verifying various concerns during the design process. The consideration of human factors in the design process plays a crucial role in the reliability and resilience of the systems involved, from an operational and error-tolerant point of view. This is especially true for aircraft cockpits. The answer to this challenge? Virtual Reality.

The i-Vision Project
Drawing on this, a European consortium decided to develop a project dedicated to the needs of aircraft manufacturers: i-VISION. i-VISION stands for Immersive Semantics-based Virtual Environments for the Design and Validation of Human-centered Aircraft Cockpits. By advancing the methodologies with requirements from modern operating conditions, i-VISION is meant to progress the current status of human factors analysis of operations in aircraft cockpits using Virtual Reality technologies. The project gathers seven major partners from the aerospace and virtual reality fields:

OPTIS, the leading virtual reality expert for industrial systems,
Two partners from the aerospace industry: AIRBUS Group Innovations Germany and France,
Three academic partners: the Laboratory for Manufacturing Systems & Automation of the University of Patras, the Institute of Applied Informatics and Formal Description Methods of the Karlsruhe Institute of Technology and the Transportation Group of University of Southampton,
The EASN, international association of academic institutes.

How Does It Work?

i-VISION consists of a semantic engine attached to the usual Virtual Reality framework, dramatically increasing the usefulness and relevance of manipulated data to human operators. This makes virtual cockpits and interacting with them smarter than with simple simulators. Human factors are evaluated both at cockpit design time and simulation time, with dynamic flight procedure generation and identification. To this end, state-of-the-art interaction methods for Virtual Reality are used, including finger tracking and eye tracking.

Through a unique combination of research in the area of human factors, semantics and virtual design, i-VISION is made to enable designers and engineers to visualize, manipulate and interact with the digital mock-up, allowing for decisions to be taken at the initial stage of the design process. Thus, it helps reduce costly errors. Additionally, pilots are integrated in the design loop, in a Virtual Reality environment connected to a flight simulator, providing even more qualitative human factors results. To this aim, i-VISION delivers three distinct technological components that are integrated and validated together with the industrial partners:

Human-cockpit analysis: Human task and cognitive work analysis for cockpit operations during various flight procedures acts as the basis for the human task and cognitive workload analysis. Several human factors methods have been demonstrated in the prototype of the project: hierarchical task analysis represents simulated or target procedures, critical path analysis points out the minimum time required to run a procedure or gives insights to the workload, error analysis helps identifying cockpit layout issues, self-assessment provided important feedback from pilots, and physical human factors use virtual humans to compute ergonomics evaluations for cockpit design.
Semantic virtual cockpit: Semantics virtual scene-graph with knowledge-based reasoning of objects and intelligent querying functions provides the semantic-based scene-graph and human task data processing and management engine. Semantic events recorded from simulations can directly be interpreted by the human factors analyst, without requiring a deep knowledge of aircraft and cockpit subsystems. The engineer query system supports cockpit designers by highlighting in a 3D or Virtual Reality view of the cockpit the elements that match a flight procedure or a composed semantic query.
Virtual cockpit design environment: Cockpit design using semantic technologies for assessing human qualities at procedural handling will be the advanced design environment for interactive and intuitive virtual prototyping of modern aircraft cockpits, facilitating knowledge-based simulation technologies. This virtual environmentis provided by HIM,the OPTIS human ergonomics evaluation software, with the help of Airbusâ€™ virtual flight simulation software to develop a new general user interface for cockpits. The project connects HIM,to recreate the instrumental interfaces, directly to the flight simulator for an increased immersion.
Vincent Hourdin, PhD R&D engineer and specialist of the i-VISION project at OPTIS, explains:

“i-VISION provides a cockpit evaluation and crew training tool for next generation airliners, using Virtual Reality. This tool reduces the cost of cockpit design: Virtual Reality allows the design to be evaluated mostly on virtual prototypes, reducing the number of expensive physical mock-ups necessary to create the product. One step further, i-VISION also offers collaborative training scenarios based on virtual cockpits, which enable low-cost devices and set-ups to be used instead of expensive and hard to access simulators. Airbus is the industrial end-user of the project. Their designers, engineers and pilot trainees around the world will benefit from it.

Human factors analysts will also benefit from the highly efficient way of building complex task analysis models and generating user activity reports from them. The i-VISION tool aims at becoming the industrial standard for virtual evaluation of cockpits and other complex systems involving humans in the loop. It is also tailored for the future, as any increased precision in virtual reality devices will only improve the toolâ€.

This system records the behavior of the pilot during flight and displays flight instruments and cockpit displays provided by the flight simulator. The pilot is placed in a virtual cockpit by the means of a head-mounted display or large 3D screens and can interact with the virtual interfaces around him thanks to a fine finger tracking system. HIM is used to identify and interpret the actions of the pilot and react accordingly. The i-VISION tool suggests the tasks to perform and the new information to display, in real time.

An Innovation For All The Transportation Industries
i-VISION delivers an innovative and practical solution that has been created both by aerospace professionals and by leading experts in their field. The knowledge-based technologies of i-VISION will accelerate the design process through the systematic reuse of knowledge, while allowing for faster and more flexible prototyping of aircraft cockpits. In addition, the analysis of human operations will result in highly competitive cockpits from the end-userâ€™s perspective and will increase the utilization of future aircraft by allowing human pilots to operate in extreme weather and traffic conditions.

While the I-VISION project is focused on aerospace problematics, the outcome of the project is now applicable to all transportation industries. Indeed, cars, trucks, boats, trains, all share the same issues and questions related to cabin design and the interaction between the cabin interface, the external environment with various lighting conditions,and the pilot or driver. The project started with aeronautics since it is the sector where the needs were the most critical regarding safety and product development. Drawing on this, various types of cabins can now benefit from the human task and cognitive workload analyses performed during the project. Virtual Reality can be used to reproduce any cabin design, create smarter virtual cabins and ease the access of the user to them compared to common physical simulators. The result will be the same as for aircraft cockpit design: performing these intuitive product experiences early in the process will help these industries reduce their product development times and costs using virtual prototypes.
â€ƒ

About OPTIS
OPTIS, the virtual prototyping company, brings life and emotion to all industrial projects. Its world leading solutions pave the way for a revolutionary design process: towards zero physical prototype. Since 1989, OPTIS offers its know-how in light and human vision simulation into famous CAD/CAM software and dedicated virtual immersive solutions. This synergy creates true-to-life virtual mockups which are used as real decision-making tools. Today, more than 2500 clients in over 50 countries already trust OPTIS and innovate day after day with its solutions to ensure the look and safety of their designs, reduce their ecological footprint and bring their future products faster on the market.For more information about OPTIS, please visit www.optis-world.com.

About i-VISION
Immersive Semantics-based Virtual Environments for the Design and Validation of Human-centred Aircraft Cockpits, this project has received funding from the European Unionâ€™s Seventh Framework Programme for research, technological development and demonstration under grant agreement No 605550.Coordinator: Laboratory for Manufacturing Systems and Automation, University of Patras, Greece.For more information about i-VISION, please visit www.ivision-project.eu.