Virtual & Augmented Reality Assisted Robot Programming
Overview
The VARobot (Virtual & Augmented Reality Assisted Robot Programming) project group aims to develop a solution combining both the advantages of AR and robotic simulation in order to set up a programming environment that assists robot programming. The project group was led by the Computer Science Masters class of Paderborn University from March 2020 to March 2021.
As robot programming is a very complex task, requiring specialist skills in multiple fields, the goal of this project group is to make the programming and therefore the use of robots easier and therefore more accessible for people without programming experience. For this, VARobot uses multiplatform AR as a solution to program different types of robots as well as to simulate them on a virtual robot displayed in the real environment. In this way, collisions can be detected on the virtual model, which eliminates the risk of damaging the real robot (or its surroundings).
Our research is based on an earlier approach which was also developed at Paderborn University. It extends this approach by introducing more flexibility in the selection of target robots. Our work also turns this basic solution idea into a multiplatform AR approach, running on both head-mounted and / or handheld AR devices. As we want our research to remain practically applicable and usable in realistic situations, the framework also connects to physical robots in order to directly deploy the developed programs.
It can be very difficult to visualize robot movements when programming a robot, which can lead to program errors. This takes time because programmers have to program the robot, run it at the actual workplace, and make adjustments based on the actual test, then start all over again. It also poses the risk of the real robot hitting an obstacle if the program is not perfectly correct, which can damage the real robot.
Robot programming has traditionally been done in very complex development environments that require a high level of expertise, making trained professionals necessary. Plus, these programming environments usually have very complex interfaces to cover all the low-level details. This leads to a very steep learning curve, making it more difficult for beginners to get into robot programming.
Existing development frameworks are often only available on a single device (often only on PCs). This restricts the user's choice and requires the availability of the single device supported by the development frameworks at all times.
Robot programs are only useful for a specific robot. Programming a new robot usually requires a new development environment which must also be learned.
The traditional programming of robots by lines of code offers no choice on input modality. The only way for the user to interact with the development framework is with the mouse and keyboard. These may not be suitable at all times, or may not always correspond to the user's preferences.
Traditional robot programming by lines of code requires a steep learning curve. It's a never-ending cycle of coding, testing on the real robot, then adjusting the code and so on.
Robot simulation using AR makes it possible to see how the robot would react virtually in the real space in which the robot is expected to operate. This eliminates the need to test on a real robot, which saves time as the real robot can also keep working undisturbed while developing the robot program. Collisions, i.e. damage to the real robot, can also be avoided because they can be easily predicted in AR.
Robot programming by demonstration allows even beginners to teach the robot how to perform a task by demonstrating it instead of programming it through many lines of code. This is intended to help users who don't have the programming skills (or the time) to program a robot. The compact design of our framework also helps reduce the complexity of the development environment, flattening the learning curve.
The multi-platform interface of our framework allows the programming of robots using both head-mounted and / or handheld AR devices. The latter offers the possibility of programming in the AR space as well as on the screen space.
The diverse target robots functionality of our framework allows the programming of different robots using a single development environment.
Our framework offers a variety of new ways for the user to interact with the programming environment in order to successfully program a robot. That is by gestures, using the user interface or even by voice via predefined voice commands.
Session management makes it possible to easily switch between different scenarios, save progress or access old states in order to continue learning from a previous point.
The NXT robot has to grab balls located at different positions and drop them in the pit.
The NXT robot has to push a cube through a labyrinth.
Project Leader
Scrum Master
Configuration Manager
Representative Project Leader
Documentation Expert
Supervisor
(AR) Domain Expert
Representative Project Leader
Quality Manager
(Robot) Domain Expert
Technology Expert
Do you need help or additional information? Send us an email and we will get back to you as soon as possible!
