1721_油箱注油口沖壓工藝及模具設計,油箱,注油,沖壓,工藝,模具設計 HAVE'2007 - IEEE International Workshop on Haptic Audio Visual Environments and their Applications Ottawa, Canada 12-14 October 2007Extending Blender: Development of a Haptic Authoring ToolSheldon Andrews', Mohamad Eid2, Atif Alamri2, and Abdulmotaleb El Saddik2 Multimedia Communications Research Laboratory – MCRLab School of Information Technology and Engineering - University of Ottawa, Ontario, KIN 6N5, Canada'sandrO71]@site. uottawa. ca, 2 teid, atifW abed] @ mcrlab. uottawa. CaAbstract：In this paper, we present our work to extend a well known 3D graphic modeler - Blender - to support haptic modeling and rendering. The extension tool is named HAMLAT (Haptic Application Markup Language Authoring Tool). We describe the modifications and additions to the Blender source code which have been used to create HAMLAT Furthermore, we present and discuss the design decisions used when developing HAMLAT, and also an implementation "road map" which describes the changes to the Blender source code. Finally, we concludewith discussion of our future development and research avenues. Keywords：Haptics HAML Graphic Modelers Blender Virtual Environments I. INTRODUCTION A. Motivation The increasing adoption of haptic modality in human-computer interaction paradigms has led to a huge demand for new tools that help novice users to author and edit haptic applications. Currently, the haptic application development process is a time consuming experience that requires programming expertise. The complexity of haptic applications development rises from the fact that the haptic application components (such as the haptic API, the device, the haptic rendering algorithms, etc.) need to interact with the graphic components in order to achieve synchronicity.Additionally, there is a lack of application portability as the application is tightly coupled to a specific device that necessitates the use of its corresponding API. Therefore, device and API heterogeneity lead to the fragmentation and disorientation of both researchers and developers. In view of all these considerations, there is a clear need for an authoring tool that can build haptic applications while hiding programming details from the application modeler (such as API, device, or virtual model).This paper describes the technical development of the Haptic Application Markup Language Authoring Tool (HAMLAT). It is intended to explain the design decisions used for developing HAMLAT and also provides an implementation "road map", describing the source code of the project.B. BlenderHAMLAT is based on the Blender [1] software suite, which is an open-source 3D modeling package with a rich feature set. It has a sophisticated user interface which isnoted for its efficiency and flexibility, as well as its supports for multiple file formats, physics engine, modem computer graphic rendering and many other features.Because of Blender's open architecture and supportive community base, it was selected as the platform of choice for development of HAMLAT. The open-source nature of Blender means HAMLAT can easily leverage its existing functionality and focus on integrating haptic features which make it a complete hapto-visual modeling tool, since developing a 3D modeling platform from scratch requires considerable development time and expertise in order to reach the level of code into the HAMLAT source tree.HAMLAT builds on existing Blender components, such as the user-interface and editing tools, by adding new components which focus on the representation, modification, and rendering of haptic properties of objectsin a 3D scene. By using Blender as the basis for HAMLAT, we hope to develop a 3D haptic modeling toolwhich has the maturity and features of Blender combinedwith the novelty of haptic rendering.At the time of writing, HAMLAT is based on Blender version 2.43 source code. C. Project Goals As previously stated, the overall goal for the HAMLAT project is to produce a polished software application which combines the features of a modem graphic modeling tool with haptic rendering techniques. HAMLAT has the "look and feel" of a 3D graphical modeling package, but with the addition of features such as haptic rendering and haptic property descriptors. This allows artists, modelers, and developers to generate realistic 3D hapto-visual virtual environments. A high-level block diagram of HAMLAT is shown in Figure 1. It illustrates the flow of data in the haptic modeling. HAMLAT assists the modeler, or application developer, in building hapto-visual applications which may be stored in a database for later retrieval by another haptic application. By hapto-visual application we refer to any software which displays a 3D scene both visually and haptically to a user in a virtual setting. An XML file format, called HAML [2], is used to describe the 3D scenes and store the hapto-visual environments built by a modeler for later playback to an end user. Traditionally, building hapto-visual environments has required a strong technical and programming background. The task of haptically rendering a 3D scene is tedious 1108 since haptic properties must be assigned to individual objects in the scene and currently there are few high-level tools for accomplishing this task. HAMLAT bridges this gap by integrating into the HAML framework and delivering a complete solution for development of hapto- visual applications requiring no programming knowledge. The remainder of the paper is organized as follows: in Section 2, we present the proposed architecture extensions and discuss design constraints. Section 3 describes the implementation details and how haptic properties are added and rendered within the Blender framework. In Section 4 we discuss related issues and future work avenues. II. SYSTEM OVERVIEW AND ARCHITECTURE The Blender design philosophy is based on three main tasks: data storage, editing, and visualization. According to the legacy documentation [3], it follows a data- visualize-edit development cycle for the 3D modeling pipe line. A 3D scene is represented using data structures within the Blender architecture. The modeler views the scene, makes changes using the editing interface which directly modifies the underlying data structures, and then the cycle repeats.To better understand this development cycle, consider the representation of a 3D object in Blender. A 3D object may be represented by an array of vertices which havebeen organized as a polygonal mesh. Users may choose to operate on any subset of this data set. Editing tasks may include operations to rotate, scale, and translate thevertices, or perhaps a re-meshing algorithm to "cleanup" redundant vertices and transform from a quad to a triangle topology. The data is visualized using a graphical 3D renderer which is capable of displaying the object as a wireframe or as a shaded, solid surface. The visualization is necessary in order to see the effects of editing on the data. In a nutshell, this example defines the design philosophy behind Blender's architecture.In Blender, data is organized as a series of lists and base data types are combined with links between items in each list, creating complex scenes from simple structures.This allows data elements in each list to be reused, thus reducing the overall storage requirements. For example, a mesh may be linked by multiple scene objects, but the position and orientation may change for each object and the topology of the mesh remains the same. A diagram illustrating the organization of data structures and reuse of scene elements is shown in Figure 2. A scene object links to three objects, each of which link to two polygonal meshes. The meshes also share a common material property. The entire scene is rendered on one of several screens, which visualizes the scene.We adopt the Blender design approach for our authoring tool. The data structures which are used to represent objects in a 3D scene have been augmented to include fields for haptic properties (e.g., stiffness, damping); user interface components (e.g., button panels) which allow the modeler to change object properties have also been updated to include support for modifying the haptic properties of an object. Additionally, an interactive hapto-visual renderer has been implemented to display the3D scene graphically and haptically, providing the modeler or artist with immediate feedback about the changes they make to the scene. in the current version of the HAMLAT. the modifications to the Blender framework include: data structures for representing haptic properties, an editing interface for modifying haptic properties, an external renderer for displaying and previewing haptically enabled scenes, scripts which allow scenes to be imported/exported in the HAML file format. A class diagram outlining the changes to the Blender ramework is shown in Figure 3. Components which are ertinent to HAMLAT are shaded in gray. HAMLAT builds on existing Blender sub-systems by extending them or haptic modeling purposes. Data structures for representing object geometry and graphical rendering areaugmented to include field which encompass the tactile properties necessary for haptic rendering.To allow the user to modify haptic properties GUI Components are integrated as part of the Blender editing panels. The operations triggered by these componentsoperate directly on the d ata structures used for representing hatic cues and may be considered part of the editing step of the Blender design cycle.Similarly to the built-in graphical renderer, HAMLAT uses a custom rendlerer for displaying 3Ds scenes grphcal and haptcall, an is ineedn of the Blender renderer. This component is developed independently since haptical and graphical rendering must be performed simultaneously and synchronously. A simulation loop is used to update haptic rendering forces at a rate which maintains stability and quality. A detailed discussion of the implementation of these classes and their connectivity is given in the next section.III IMLIEMENTATIONA Data StructureA.1 Mesh Data TypeBlender uses many different data structures to represent the various types of objects in a 3D scene a vertices; a lamp contains colour and intensity values; and camera a object contains intrinsic viewing parameters.The Mesh data structure iS used by the Blender inframework to describe a polygonal mesh object. It iS of particular interest for hapic rendering since many solid objects in a 3D scene may be represented using this type of data structure. The tactile and kinesthetic cues, which are displayed due to interaction with virtual objects, are typically rendered based on the geometry of the mesh. Hptic rendering is performed based primary on data stored in this data type. Other scene components such as lamps, cameras, or lines are not intuitively rendered using force feedback haptic devices and are therefore not of current interest for haptic rendering.An augmented version of the Mesh data structure is shown in Figure 4. It contains fields for vertex and face data, plus some special custom data fields which allow data to be stored to/retrieved from disk and memory. We have modified this data type to include a pointer to a MHaptics data structure, which stores haptic properties such as stiffness, damping, and friction for the mesh elements (Figure 5).A.2 Edit Mesh Data TypeIt should be noted that the Mesh data type has a comPlimentary data structure, called EditMesh, which is used when editing mesh data. It holds a copy of the vertex, edge ,and face data for a polygonal mesh. when the user switches to editing mode, the Blender copies the data from a Mesh into an EditMesh and when editing is complete the data is copied back.Care must be taken to ensure that the hapic property data structure remains intact during the copy sequence. The EditMesh data structure has not been modified to contain a copy of the hapic property data ,but this may properties in edit mode is required). The editing mode is mainly used to modify mesh topology and geometry, not the haptic and graphical rendering characteristics, A.3 Haptic Properties In this section we'll briefly discuss the haptic properties which may currently be modeled using HAMLAT. It is important for the modeler to understand these properties and their basis for use in haptic rendering. The stiffness of an object defines how resistant it is to deformation by some applied force. Hard objects, such as a rock or table, have very high stiffness; soft objects, such as rubber ball, have low stiffness. The hardness-softness of an object is typically rendered using th spring-force equation: Where the force feedback vector f which is displayed to the user is computed using ks the stiffness coefficient (variable name stiffness)for the object and x the penetration depth (displacement) of the haptic proxy into an object. The stiffness coefficient has a range of [0,1], where 0 represents no resistance to deformation and 1 represents the maximum stiffness which may be rendered by the haptic device. The damping of an object defines its resistance to the rate of deformation due to some applied force. It is typically rendered using the force equation:Where kd is the damping coefficient (variable name}MHaptics; damping) anddepdt is the velocity ofthe haptic proxy as it;penetrates an object. The damping coefficient also has a range of [0,1] and may be used to model viscous behaviour of a material. It also increases the stability of the hapticrendering loop fordstiffmaterials.The static friction (variable name stjriction) and dynamic friction (variable name dyjriction) coefficient are used to model the frictional forces experienced whileexploring the surface of a 3D object. Static friction is experienced when the proxy is not moving over the object's surface, and an initial force must be used to overcome static friction. Dynamic friction is felt when the proxy moves across the surface, rubbing against it.Frictional coefficients also have a range of /0,1], with a value of 0 making the surface of a 3D object feel "slippery" and a value of 1 making the object feel veryrough. Frictional forces are typically rendered in a direction tangential to the collision point of the hapticproxy at an object's surface. B. Editing Blender uses a set of non-overlapping windows called spaces to modify various aspects of the 3D scene and its objects. Each space is divided into a set of areas andpanels which are context aware. That is, they provide functionality based on the selected object type. For example, if a camera is selected the panel will display components which allow the modeler to change the focal length and viewing angle of the camera, but these components will not appear if an object of another type is selected. Figure 6 shows a screen shot of the button space which is used to edit properties for a haptic mesh. It includes user-interface panels which allow a modeler to change the graphical shading properties of the mesh, perform simple re-meshing operations, and to modify the haptic properties of the selected mesh. HAMLAT follows the context-sensitive behavior of Blender by only displaying the haptic editing panel when a polygonal mesh object is selected. In the future, this panel may be duplicated to support haptic modeling for other object types, such as NURB surfaces. The Blender framework offers many user-interface components (e.g., buttons, sliders, pop-up menus) which may be used to edit the underlying data structures. The haptic properties for mesh objects are editable using sliders or by entering a float value into a text box located adjacent to the slider. When the value of the slider/text box is changed, it triggers an event in the Blender window sub-system. A unique identifier that the event is for the haptic property panel and the HAMLAT code should be called to update haptic properties for the currently selected mesh.C Hapto-Visual RenderingBlender currently support graphical rendering of scenes using an internal render or an external renderer (e.g., [4]). In this spirit, the haptic renderer used by HAMLAT has been developed as an exteral renderer. It uses the OpenGL and OpenHaptics toolkit [5] to perform graphic and hapic rendering ,respectively.The 3D scene which is being modeled is rendered using two passes: the first pass render the scene graphically, and the second pass renders it haptically. The second pass is required because the OpenHaptics toolkit intercepts commands send to the OpenGL pipeline and uses them to display the scene using haptic rendering techniques. In this pass, the haptic properties of each mesh object are used much in the same way color and lighting are used by graphical rendering they define thetype of material for each object. To save CPU cycles, the lighting and graphical material properties are excluded from the haptic rendering pass.Figure 7 shows source code which is used to apply the material properties during the haptic rendering pass. The haptic renderer is independent from the Blenderframework in that it exists outside the original source code. However, it is still heavily dependent on Blender data structures and types.D. ScriptingThe Blender Python (BPy) wrapper exposes many of the internal data structures, giving the internal Python scripting engine may access them. Similar to the datastructures used for representing mesh objects in the native Blender framework, wrappers allow user defined scripts to access and modify the elements in a 3D scene.The hapic properties of a mesh object may be accessed through the Mesh wrapper class. A haptics attribute has been added to each of these classes and accessed through the Python scripting system. Figure 8 shows Python code to read the haptic properties from a mesh object and export to a file. Similar code is used to import/export HAML scenes from/to files. An import script allows 3D scenes to be read from a HAML file and reproduced in the HAMLAT application; export script allows 3D scenes to be written to a HAML file, including haptic properties, and used in other HAML applications. The BPy wrappers also expose the Blender windowing system. Figure 9 shows a panel which appears when the user exports a 3D scene to the HAML file format. It allows the user to specify supplementary information about the application such as a description, target hardware, and system requirements. These are fields defined by the HAML specification [2] and are included with the authored scene as part of the HAML file format. User-interface components displayed on this panel are easily extended to agree with the future revisions of HAML.The current version of HAMLAT shows that a unified modeling tool for graphics and haptics is possible. Promisingly, the features for modeling haptic properties have been integrated seamlessly into the Blender framework, which indicates it was a good choice as a platform for development of this tool. Blender's modular architecture will make future additions to its framework very straightforward.Currently, HAMLAT supports basic functionality for modeling and rendering hapto-visual applications. Scenes may be created, edited, previewed, and exported as part of a database for use in by other hapto-visual applications, such as the HAML player [6]. However, there is room for growth and in there are many more ways we can continue leveraging existing Blender functionality.As per future work ,we plan to extend HAMLAT TO include support for other haptic platforms and devices.Currently, only the PHANTOM series of devices is supported since the interactive renderer is dependent on the OpenHaptics toolkit [5].In order to support otherd evices, a cross-platform library such as Chai3D or Haptik may be used to perform rendering. These libraries support force rendering for a large range of haptic hardware. Fortunately, due to the modularity of our implementation, only the