Robot programming by demonstration is a technique for teaching a robot how to perform tasks instead of programming it through lines of code. This is intended to help users that do not have the programming skills (or the time) to program a robot. Demonstrating how to accomplish the task with examples allows one to teach the robot without specifically programming each detail. An AR environment for PbD is used in our project. Here, path following operations were taught by kinesthetics teaching. Thus, a tracker labelled sphere (target sphere) was used as a demonstration tool.

Steps involved:

The coordinates of the waypoints are displayed on the AR DoBot code panel.

The main menu can be easily accessed by simply tapping on the icon present on the top-right corner of the device screen. It is designed to hold all submenu items which in turn will hold up all features portrayed over the AR space. Apart from that, it holds additional options too. It contains a ‘Debug’ mode button which is specially meant for running the application in debug mode and / or running mode. The main menu contains a special session management feature which enables us to save the current scene as it is and then safely navigate to other scenes as and when required without the need to restart the application all over again. This is to safely handle and operate multiple scenes presented in the application. The main menu supports a scrollable menu and maintains a safe screen occupying ratio. Each of the submenu options carries an AR toggle button which can then immediately send the sub-menu item to AR space from screen space and vice versa. This operation can be carried out collectively for all sub-menu items or individually. The sub-menu items and special functions are all categorised for ease of understandability.

The feature ‘Touch Input’ needs a special mention as we use this feature to actually operate the robots in the concerned scenes. It can be accessed via tapping the ‘Touch Input’ option available in the list of ‘Main’ menu. The on-screen version of this feature is somewhat different from the AR space version. It is due to the required compactness that has to be addressed on-screen. While the AR version is capable of moving each command via the dedicated upward or downward buttons, the on-screen version is capable of managing this just by dragging the command to the position that the user wants. The on-screen version is capable of adding different types of commands specifically meant for the corresponding robots via the dedicated named buttons inside the ‘Touch Input’ panel. This action can be carried out in the AR space version by tapping on the dedicated ‘+’ button that is painted in the syntax color of the corresponding command. The movement buttons (both upward and downward) and the command addition buttons are included with each line of the commands in AR space.

Although the UI design has been created with as many considerations as was possible, change is inevitable. With this concept in mind, the entire compact UI design and extension can be easily configured using Unity game engine. The syntax colors introduced for commands can be configured and changed as and when required, with minimal effort. The AR space command addition button colors can be correspondingly modified to match any new changes in syntax colors of the commands. The button icons and button color can also be easily modified. The total arrangement of the UI panels and features can have alternate orientations as required with possible introductions of new features in the future.
Since the application also presents multiple scenes meant for different robots, we have the variants for the same sub-menu items which are tailored for the corresponding scene and the corresponding robots.