Virtual Reality and Haptics for Product Assembly

—Haptics can significantly enhance the user’s sense of immersion and interactivity. An industrial application of virtual reality and haptics for product assembly is described in this paper, which provides a new and low-cost approach for product assembly design, assembly task planning and assembly operation training. A demonstration of the system with haptics device interaction was available at the session of exp.at’11

INTRODUCTION Virtual reality (VR) and haptics are new innovative and promising technologies that emerged in recent years, and rapidly evolved into wide applications, from medicine to industry, from education to training, from entertainment to military [1]. Virtual assembly is one of the most challenging applications of virtual reality in engineering. The first objective of virtual assembly is to test the feasibility of the assembly operations at the design stage of the product. The second objective is to generate optimal assembly plans including resource allocation, assembly time and cost estimation, assembly operation training and maintenance ergonomics [2]. Haptics is particularly important for virtual assembly, because it can increase the users sense of immersion and interaction, help the users to get a better understanding of virtual objects, to feel more secure and more confident in the real world assembly process, and thus improve task efficiency [3]. In this paper, a typical virtual assembly application system based on virtual reality and haptics is introduced for industrial products.
II. SYSTEM CHARACTERISTICS Model and data can be integrated with commercial CAD systems. The product, tools and fixtures are designed in a commercial CAD system such as Pro/Engineer or SolidWorks, and then an automatic data integration interface can be developed to transform the data and information from CAD to VR, as shown in Fig.1. Four types of data are mainly taken into account, including geometry data, topology data, assembly data and physics data. These models and data are then input into virtual reality environment.
A multi-modal virtual environment must be constructed for virtual assembly simulation, including vision, audio and haptic feedback (Fig.2). A hierarchical constraintbased data model is proposed to represent parts and objects, which is composed of product layer, subassembly layer, part layer, feature layer, surface layer and polygon layer. For elements in the same layer, there exist geometry constraint relationships, and for elements in different layer, there exist hierarchical mapping relationships.

. Haptics interaction in virtual environment
A hierarchical scene graph structure can be also generated. The user can select different layer object to operate, for example, he can select a single part to operate, and he can also select a subassembly as a whole to assemble or disassemble. Because there are multi-modal feedbacks in virtual environment, a multi-thread mechanism is realized. There are three separate threads in the system: haptic rendering thread, physical calculation thread and graphical rendering thread. The haptic rendering thread is responsible for communicating with the PHANTOM device, launching at a high priority and high frequency (about 1000 Hz). The physical calculation thread performs all the work including collision detection, physics computation, dynamic simulation of realistic part behaviour, and geometry constraint recognition etc, which is running at a second priority and frequency (about 100Hz). The graphic rendering thread is mainly responsible for visualizing the entire scene and virtual objects, and it runs at a low frequency of about 30 Hz.
A new approach based on physics modeling and haptics feedback is realized for virtual assembly operation. As shown in Fig.3, this process can be divided into two stages: contact simulation state and assembly simulation state. Contact state is mainly referred to simulate the collision reaction and dynamic behavior of the virtual part, and assembly state is mainly referred to simulate the mating or insertion process of the virtual part. In virtual environment, when two parts are close enough to each other and the distance and orientation of their assembly features reach a specified range, the assembly simulation state can be activated. Otherwise, the contact simulation state is executed. During contact simulation, a real-time collision detection algorithm is supported to calculate the collision position, direction and penetration depth, and then the collision force and torque can be computed and sent back to the user to get the sensation of realism. A spring-mass model is used to simulate the dynamic contact behavior of virtual parts. During assembly simulation, in order to avoid unnecessary computation and improve system efficiency, the collision detection is closed. The virtual part is adjusted to the precise position by geometry constraint. A real-time guiding force or repulsive force can be also generated. The guiding force can be used to help the user to assemble the part along the free-collision path to the correct position. If the user deviates from this position, a repulsive force or torque can be also generated to prevent the user's action, to make the user feel that he is assembling the virtual part as he would the real products. Shown in Fig.4 is a haptics guided assembly operation, and Fig.5 is the recorded force information during the assembly process.
III. CONCLUSION Virtual reality and haptics is a promising and valuable application for product assembly in industrial environment. It can result in faster product development process, faster identification of assembly and design issues, and an efficient and low-cost approach for assembly planning and training without physical models. We will demonstrate the application of the system on the exhibition with the hardware and software, and the public can assemble the product with the haptics device.  Besides the research interests in Robotics, namely, force control, parallel manipulators and walking machines, he has also interests in pedagogical matters, distance learning and remote labs (email: aml@fe.up.pt).
Maria Teresa Restivo is with faculty of Engineering of University of Porto and UISPA, IDMEC-Polo FEUP. As Senior researcher she has been working in automation, instrumentation and control areas, with R&DI interests in sensors and transducers development, and applications wireless communication systems, new technologies for web remote control and interaction of haptics devices with remote and virtual reality experiments (email: tres-tivo@fe.up.pt).