Integrating natural language instruction with computer vision for collaborative robot programming

Abstract

Industry 4.0 is booming due to its multiple benefits ranging from optimizing resources, managing quality, and reducing manufacturing times. Different technologies are combined and implemented to give life to Industry 4.0. Recently the use of collaborative robots, also called cobots, has aroused great interest thanks to their versatility and low cost, bringing with it the automation of processes in a short time, weeks, or even days. However, the great potential that cobots can offer has not yet been exploited, mainly because the programming of cobots is still only for a select group of technical users and experts on the subject due to the complexity of task programming.

The consequence of a gap in the development of a straightforward and robust user interface allows intuitive and clear programming, promoting collaborative development in an accessible way to non-technical users.

Therefore, this proposal presents a prototype version of a voice user interface (VUI) for non-technical users to program and control collaborative robots. This implementation is planned to reduce the hardship of programming a collaborative robot and, through an experiment, measure the usability of the VUI and the reliability of natural language programming for collaborative robotics.