Future directions for AR and VR

Mixed reality has been available for many years now, maybe it is time to revisit where virtual reality (VR) & (AR) simulators may fit into mining industry activity. Potentially, VR & AR simulators are a valuable tool for training and retraining mine workers in safe work procedures and frequent and infrequent mining activities. However, despite VR mining simulators being around since the mid-nineties, the industry is yet to embrace the technology for more advanced applications as described by industry researchers (1). The availability of VR head-sets, low-cost projectors and HDTV technologies, combined with advances in software development kits, makes VR & AR technology more accessible than ever.

Here’s some thoughts on how and where simulator technology should be applied and why the mining industry should take advantage of more advanced simulator capability.

In recent decades, mining activities have become more sophisticated. Work organisation has changed with multi-skilling and hi-tech equipment commonplace. During the previous boom (in Australia), there was a large labour mobilisation and regular changes in equipment and work procedures. This meant that risk associated with equipment or safety procedures changed frequently. Hence, personnel must be constantly trained and retrained to ensure safe human responses on site.

Within Australia (and elsewhere), comprehensive rules and regulations relating to the safe operation of mines exist. However, despite this, accidents still occur suggesting deficiencies within the rules and procedures themselves, the training delivery method, worksite culture, or all aspects.

Also, extraneous influences may contribute too. For example, work procedures may be ineffective due not encapsulating learning experiences gained from previous operations and incidents. Or, information is available, but not in a form easy for personnel understand, collate into training experiences, or distribute efficiently. Training sessions may be high quality, but ineffective due to variability between trainer styles and trainee inattention or later risk-taking activity. There are many variables to consider. What is needed, is consistency in the training experience, materials and method of delivery.

Achieving safe human responses in potentially hazardous mine environments requires the timely and effective training, retraining and continuous improvement of the skills of operators and maintenance personnel. This may be possible via advanced simulator training modules. Modules that train workers about the location in which they will work, the activities they will perform and how to identify hazards and risks that may be present. Importantly they can be trained how hazards may be removed or managed. Interactive simulators developed as a serious game, may be able to provide the necessary training consistency and user engagement along with a sense of being there.

The new wave of VR and associated software and applications means that virtual reality and hence simulators are even more accessible than ever before, offering an innovative way to assess hazards and workplace environments. Information can be placed in context and presented in a consistent manner to users. This is particularly important for training and the development of risk assessments of inaccessible underground mine environments. Ideally, simulators should be tailored to meet the needs of the local industry and use a mix of generic and site specific virtual environments to provide the necessary experiences. They must also have the capacity to integrate knowledge and skills to achieve safe human responses and importantly, enhance existing training methods.

Hence, a simulator is not just a head set, a container enclosing based motion base and projection system, or a large screen visualisation system per se. It is much more than that. A model simulator should provide these elements, as a minimum, to be effective,

1. Consider human factors.

2. Provide access to a Knowledge Management System.

3. Contain examples of ‘Best Practice’ Safety Management Plans.

4. Provide examples of ‘Best Practice’ Safe Working Procedures.

5. Include advanced functions to challenge and train production and maintenance personnel and assess their competency and proficiency following training.

6. Include the ability to identify risk taker behaviour and facilitate behaviour modification.

7. Where needed, have an ability to provide a resolution and fidelity that suspends disbelief.

8. Ability to engage industry experts with the virtual environment to develop the most realistic of interactive experiences, not just a passive 3D video experience.

Ideally, the application of simulators should be strategic and targeted, and not developed just because it can be. Where it is safe and practicable to use reality for training and education, this should be the preference. Where is possible to use digital images and interactive video, acquired via easy to gain and consistent site access, this media these should be the preference. Where safe and consistent site access is not possible, such as underground mine sites, or future operations, simulators should be used. This is because the effort expended in acquiring expert input, reference materials, images and so-on to build meaningful virtual environments can add a significant human overhead and hence significant cost to the development of interactive simulators. Environments that should be replicated in simulators should include: high-risk equipment and activities; environments that are difficult to access safely; environments that do not yet exist; and infrequent or emergency events. The goal is to provide a realistic site experience to the user.

There must also be a return on the investment. However, how this is measured is still difficult to quantify. The cost of one mine accident can result in the loss of the mines social licence to operate, so the financial benefits may not be immediately realised. Hence, a simulator, should not be viewed as a fancy headset, a large screen visualisation or a one-use equipment simulator that provides an initial but unsustainable wow factor. Simulators (and their 3D video derivatives) should be an essential tool that routinely replicates work place environments and presents trainees or teams with a problem based learning exercise. The trainee’s ability to identify and remedy risks can then be quantified and proficiency assessed.

Where appropriate, a simulator should show graphically the consequences of poor decision-making or risk-taking behaviour. It should also include feedback on how close a serious incident was and how it could be avoided. However, the intention is to educate not traumatise. The interaction with the simulator should be based upon sound educational principles and its application designed around human factors guidelines. In summary, simulators still have a lot of unexplored possibilities for mine site operations as shown in the figure, particularly for training the new wave of skilled workers that will be required when the industry leaves the current downturn.

Simulators also have massive application for Mine Information Modelling…but that’s another article.

Phillip Stothard

(1). ”Stothard P, Squelch, E van Wyk, R. Stone, Kizil M, Schofield D, Fowle K. 2015. Taxonomy of Interactive Computer-based Visualisation Systems and Content for the Mining Industry - Part 2. Volume 124, Issue 2 (June 2015) Mining Technology (Trans. Inst. Min. Metall. A) Print ISSN: 1474-9009 Online ISSN: 1743-2863. View here ”)