Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Grasp Planning Of Unknown Object For Digital Human Model

1 Dassault Systèmes Canada, 393 Rue Saint-Jacques #300, Montréal, QC H2Y 1N9, Canada ↗
2 Ecole de Technologie Supérieure, Department of Systems Engineering, 1100 Rue, Notre Dame O, Montréal, QC H3C 1K3, Canada ↗
Publication year
Digital Object Identifier10.54941/ahfe1001908

Abstract

ObjectiveGrasp planning is a popular topic in the fields of robotic and Digital Human Model (DHM) (4, 6, 7, 9, 10, 11). So far, the proposed planners do not consider the final posture of the DHM has a criteria when determining potential grasps. In (4), a grasping algorithm has been developed to automatically grasp known tools. The present work introduces a grasp planner for single-hand grasp on an unknown object, further referred as “part”.MethodThe grasp planner gives has a result a grasp pose (position + orientation) for the posture solver (Smart Posturing Engine) to reach. The input necessary to the grasp planner are the 3D model of the object to grasp and of the surrounding environment, and an initial manikin position that is automatically determines by the posture solver algorithm.First the part is approximated by its oriented bounding box (OBB), limiting the grasp poses to 6 (one for each face of the OBB). Then precise grasp types (5) and apertures are chosen based on the face’s dimensions (i.e. width and depth), ranging from a small face (i.e. pinch) to larger ones (i.e. medium wrap or precision sphere).To determine what is the best face of the OBB to grasp, accessibility checks are performed by validating that the space around the face is free of collision. The faces are checked using a specific order (i.e. top, right or left, bottom, front, back) that is determined using the relative initial position of the manikin. As soon as a face is found to be graspable and accessible, the algorithm stops and choose that face as the best one to grasp.Using the selected face target, the hand is positioned using an inverse kinematic solver, free to rotate around the target using extra hand degrees-of-freedom inside a limited range (4). Giving the posture solver more possibilities to find a realistic posture.ResultsThe grasp planner described above leaded to believable grasps for the simulated tasks as well as a believable overall DHM posture. Examples of postures will be shown on assembly tasks performed on a gearbox assembly line.DiscussionThe proposed grasp planner seems really promising. In its current form, it is most suitable for small parts and bigger ones well represented by their OBB. More complex and bigger parts may require further segmentation into multiple smaller sub-parts (9, 10), allowing to perform the proposed checks at more specific and believable locations on the object. This would allow to obtain grasps on a wider range of objects. The object weight is also important and is currently being added to grasp type selection. The present planner is used by the Smart Posture Engine (SPE) framework (1, 2 and 3) inside Dassault Systèmes application “Ergonomic Workplace Design”. With the Ergo4All (12) technology the SPE allows to assess and minimize ergonomic risks involved in simulated workplaces.1. Lemieux, P.-O., Barré, A., Hagemeister, N., Aissaoui, R.: Degrees of freedom coupling adapted to the upper limb of a digital human model. Int. J. Hum. Factors Model. Simul. 5(4), 314–337 (2017)2. Lemieux, P., Cauffiez,M., Barré, A., Hagemeister, N., Aissaoui, R.: A visual acuity constraint for digital human modeling. In: 4th Conference proceedings (2016)3. Zeighami, A., Lemieux, P., Charland, J., Hagemeister, N., Aissaoui, A.: Stepping behavior for stability control of a digital human model. ISB/ASB (2019)4. Bourret, Q., Lemieux, P., Hagemeister, N., Aissaoui, R.: Flexible hand posture for tools grasping. DHM (2019)5. FEIX, Thomas, ROMERO, Javier, SCHMIEDMAYER, Heinz-Bodo, et al. The grasp taxonomy of human grasp types. IEEE Transactions on human-machine systems, 2015, vol. 46, no 1, p. 66-77.6. BEKEY, George A., LIU, Huan, TOMOVIC, Rajko, et al. Knowledge-based control of grasping in robot hands using heuristics from human motor skills. IEEE Transactions on Robotics and Automation, 1993, vol. 9, no 6, p. 709-722.7. Holleman, C.; Kavraki, L.E.; A framework for using the workspace medial axis in PRM planners, in Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), IEEE, Vol 2, 2000, 1408-1413. https://doi.org/10.1109/ROBOT.2000.8447958. FEIX, Thomas, BULLOCK, Ian M., et DOLLAR, Aaron M. Analysis of human grasping behavior: Correlating tasks, objects and grasps. IEEE transactions on haptics, 2014, vol. 7, no 4, p. 430-4419. Díaz, C.; Puente, S.; Torres, F.; Grasping points for handle objects in a cooperative disassembly system, IFAC Proceedings Volumes, 40(2), 2007, 112-117. https://doi.org/10.3182/20070523-3-ES-4907.0002010. Miller, A.T., Knoop, S., Christensen, H.I. and Allen, P.K., Automatic grasp planning using shape primitives. in Robotics and Automation, 2003. Proceedings. ICRA'03. IEEE International Conference on, (2003), IEEE, 1824-1829.11. Goussous, Faisal Amer. Grasp planning for digital humans. The University of Iowa, 2007.12. Bourret, Quentin, et al. "Ergo4All: An Ergonomic Guid

INTRODUCTION

Digital Human Model (DHM) offers the unique possibility to simulate wor- ker task in a 3D environment. This is particularly useful in the manufacturing world, allowing to detect ergonomic problems before production lines are built. This does not replace traditional ergonomics, but can help detect pro- blems in the virtual stage of the design phase to avoid costly changes on the production line in the real world.

Today, different DHM are available in commercial products: DELMIA Ergonomics (Dassault Systemes), Jack™(Badler 1999), Santos® Pro (VSR 2004). (Zhou et al. 2009) explained that the biggest challenge in DHM applications is the low efficiency of the manikin positioning in 3D, due to manual posture creation, moving each joint separately, which is a very time-consuming process. Jack and IMMA proposed methods to automati- cally posture a manikin in a 3D environment (Cort 2019) (Hanson 2014). The posture prediction process is not fully automatic because the user must place the manikin close to the object manually before resolving the posture. However, this is a step forward to reduce the manikin posturing creation phase.

Dassault Systèmes released a new application called “Ergonomic Work- place Design” (EWD) that aims at helping manufacturing engineers to design

© 2022. Published by AHFE Open Access. All rights reserved. 120

Grasp Planning of Unknown Object for Digital Human Model 121 safe and efficient workplace in 3D. The Smart Posture Engine (SPETM) tech- nology was developed to reach that particular goal. The SPE is a framework that performs an autonomous posturing of a DHM based on minimal user inputs (Lemieux et al., 2016; Lemieux et al., 2017; Zeighami et al., 2019).

This paper focus on the grasp planning part of the automatic posture gene- ration. (Bohg et al. 2013) divided the grasp problem depending on three objects categories: 1) known objects, 2) familiar objects and 3) unknown objects. Known objects are those for which grasps have already been defi- ned. Familiar objects are new objects that can be grasped in a similar way as defined in first category. Finally, unknown objects are objects on which there is no prior grasp definition.

As mentioned by (Zhou et al., 2009), grasp planners usually try to find the best hand location on the object without considering the final digital human posture. This may end up in unrealistic final postures when reaching for the object.

In a previous paper, a grasping algorithm has been developed to auto- matically grasp tools, which belongs to the first category of known objects (Bourret et al., 2019). The objective was to have a better DHM posture when grasping tools by allowing a range of motion to the hand on the object. An initial algorithm has been proposed to automatically find grasping cues on familiar tools, allowing the grasp planner to grasp familiar objects (Macloud et al., 2019; Macloud et al., 2021).

The present work introduces a complementary grasp planner for single- hand grasp on unknown objects, further referred as “part”. Like for known and familiar object, this grasp planner considers different aspects of the DHM final posture when choosing the proper way to grasp the object. The objective is to propose visually plausible grasp on unknown objects in a manufacturing context.

METHODOLOGY

Virtual Environment Example This paper proposes a methodology to grasp unknown objects in a manu- facturing context. Unknown objects are parts that compose the product assembled on the production line. To explain and test the proposed grasp planning methodology, the following virtual production line and product will be used (Figures 1.a, 1.b).

Figure 1: a - Virtual production line used to demonstrate the methodology. b - Virtual product (gearbox) assembled on the production line.
Original PDF Figure 1:

Inputs and Outputs The inputs of this algorithm are: the 3D model of the object to grasp, the 3D model of the production line environment, an initial position of the DHM in the 3D environment and the hand to be used (right or left) see Figure 2.

Figure 2: Inputs of the grasp planner.
Original PDF Figure 2:

The outputs given by the grasp planner are the grasp type to use and a grasp target, later explained. In solving the DHM posture, the SPE framework will use the grasp type to determine the position and orientation of the upper limb end effector, which is the hand. The end-effector will reach the target onto the object with the DHM using inverse kinematic algorithms.

The grasp planner is composed of five steps that will determine the grasp for an unknown object (Figure 3). Each step is further detailed below.

Grasp Planner

Figure 3: Steps of the grasp planner.

Figure available in the original PDF paper.

Bounding Box and Target Calculation During the first step, the object is approximated by its minimum bounding box (Figure 4.a).

Figure 4: a - Object approximation by its minimum bounding box. b - Grasp potential targets position and orientation calculation.
Original PDF Figure 4:

Grasp Planning of Unknown Object for Digital Human Model 123

To define the orientation of the bounding box, the principal axis of inertia of the object are used (Figure 4.a). A potential grasp target is then associa- ted to each face of the bounding box. The geometrical center of the object is calculated. It is projected on each of the six faces of the bounding box. This gives the location of the grasp target. It follows heuristic that humans prefer to grasp an object close to its center of mass, probably to reduce the efforts on the joints (Bekey 1993). However, since the distribution of the mass is not available on the grasped objects, the geometrical center is used. The hand orientation for each target is defined by the orientation of the minimum bounding box.

Grasp Type Determination (Feix et al., 2015) described a taxonomy of the different grasps that a human can do. In their work, a statistical analysis was performed for different grasp characteristics measuring the object size and weight, as well as grasp freque- ncy for each grasp type. In the present grasp planner, three grasp types were chosen among the most used grasps to have a great coverage of the objects grasped in the manufacturing context (Figure 5.a).

For each grasp type, an open and close hand configuration are created (Figure 5.b) and are later used during the hand closure on the object.

Figure 5: a – Grasp type considered in this grasp planner. b – Hand closure example for medium wrap grasp. c – Bounding box face characterization.
Original PDF Figure 5:

To determine which grasp type to choose, the dimensions of the bounding box faces are used. Figure 5.c shows that for each face of the bounding box, three dimensions can be found. First D1 and D2 are the dimensions of the face edges (D1 ≥D2). Then D3 is the dimension of edge normal to the considered face.

Knowing the dimension for each face, it is possible to determine a rule defining the grasp type to use: If D1 < 60mm And D2 < 35mm

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

Grasp Type = Medium Wrap To conclude, a small object will be grasped with a pinch grasp. A bigger object that has a small D3 dimension (flat object) will be grasped using a precision

sphere grasp (using the tip of the fingers). Otherwise, a medium wrap grasp will be used.

Those values were initially chosen using (Feix et al., 2014) study and refined with empirical tests performed on different manufacturing parts.

Face Labeling To be able to later rank the grasps and use heuristics, it is necessary to label the faces of the bounding box.

Figure 6: Object bounding box labeling regarding the manikin initial position.
Original PDF Figure 6:

Each face is labeled depending on its position relative to the manikin initial position. The six faces labels are: Front, back, left, right, top and bottom.

Graspable Faces To determine which faces of the bounding box are graspable, two checks are performed.

Figure 7: Collision check using isolated hand model to determine each faces accessi- bility.
Original PDF Figure 7:

First, the accessibility of each faces is validated. By positioning the isolated hand at the target’s location in its open position, it is possible to check if the face of the bounding box is accessible. If a collision between the isolated hand and the environment around the object is detected, then the face is considered as not accessible and will be ignored at the final stage when choosing the final grasp. In the example of Figure 7, extreme right image, the bottom face is not accessible. All the other faces are still candidates for the final grasp.

The second check concerns the faces dimensions. For each of the accessible faces, if the dimension D2 > 100mm then the face is considered too big to

Grasp Planning of Unknown Object for Digital Human Model 125

be grasped. This limit value follows (Feix et al., 2014) observation regarding the limits dimension that a human can grasp.

Grasp Ranking The final step consists of ranking the faces of the bounding box.

Table 1. Bounding box faces ranking.

Table available in the original PDF paper.

Grasp Side Top Right/Left Bottom Front Back Rank 1 2 3 4 5

Table 1 shows the rank of the bounding box faces. Considering that the right hand has been chosen to grasp the object, the preferred face to grasp will be the top one. If it is not graspable (no free space or face too big), the next face will be checked for graspability until a graspable face is found (right, bottom, front, back). If no face is graspable, then the top face will be chosen.

If the left hand is used to grasp the object, then the second rank is the left side instead of the right side. In fact, when grasping an object with the right hand, the left side is considered as not graspable because it would end up in nonrealistic final DHM posture (the same is applied when grasping with the left hand).

Grasp Execution Once the optimal face to grasp is determined, the target associated with the selected face is given to the inverse kinematic (IK) solver. The solver will match the upper limb end effector frame with the target frame (Figure 8.a). To have a more probable posture, three rotations degree of freedom are allo- wed along each direction of rotation of the end effector. The rotations about the Z axis was kept unlimited (Figure 8.b) while the X and Y axis were limi- ted to some extent based on empirical tests (e.g. ±10 to ±30°). This gives the IK solver more room to find a visually plausible posture. Once the target has been reached, the hand closes on the object. The hand starts in its open configuration (Figure 8.b), then each finger is moved toward its closed confi- guration (Figure 8.c). When a collision is detected between the finger and the object to grasp, the closure ends for that finger. The closure continues until all fingers are in collision or until all fingers reach their closed configuration.

RESULTS

Figure 9 showcase grasps performed on a gearbox assembly line. The task consists of assembling the parts that compose the gearbox.

The examples show in Figure 9 provide a good representation of grasp that can be performed by the grasp planner with different grasp types and locations.

What is also interesting to see is the overall manikin posture. Thanks to the degree of freedom allowed to the upper limb end-effector, the IK solver has more room to find a visually plausible body posture.

Figure 8: Grasp execution steps: a. Target reach. b. Hand Degrees Of Freedom (DOF) on the object. c. Hand closure.
Original PDF Figure 8:
Figure 9: Examples of grasp results for different parts: a. Bearing cover. b. Housing. c. Screw. d. Flange.
Original PDF Figure 9:

DISCUSSION

The proposed grasp planner seems promising. In its current form, it is most suitable for small parts or bigger ones when they are well represented by their oriented bounding box. More complex and bigger parts may require further segmentation into multiple smaller subparts (Miller et al., 2003), allowing to perform the proposed checks at more specific locations on the object.

The present planner is used by the Smart Posture Engine (SPE) framework inside Dassault Systèmes application “Ergonomic Workplace Design”. With the Ergo4All (Bourret et al., 2021) technology, the SPE allows to assess and minimize ergonomic risks involved in simulated workplaces.

Grasp Planning of Unknown Object for Digital Human Model 127

REFERENCES

Badler, Norman I., Palmer, Martha S., et Bindiganavale, Rama. Animation control for real-time virtual humans. Communications of the ACM, 1999, vol. 42, no 8, p. 64-73. Bekey, G. A., Liu, H., Tomovic, R., & Karplus, W. J. Knowledge-based control of gra-

sping in robot hands using heuristics from human motor skills. IEEE Transactions on Robotics and Automation, 1993, vol. 9, no 6, p. 709-722. Bohg, J.; Morales, A.; Asfour, T.; Kragic, D.; Data-driven grasp synthesis— a survey, IEEE Transactions on Robotics, 30(2), 2013, 289-309. https://doi.org/10.1109/TRO.2013.2289018 Bourret, Q., Charland, J., Imbeau, D., Brouillette, D., Djire, J-B. “Ergo4All: An

Ergonomic Guidance Tool for Non-ergonomist.” Congress of the International Ergonomics Association. Springer, Cham, 2021. Bourret, Q., Lemieux, P., Hagemeister, N., Aissaoui, R.: Flexible hand posture for tools grasping. DHM (2019) Cort JA, Devries D. Accuracy of Postures Predicted Using a Digital Human

Model During Four Manual Exertion Tasks, and Implications for Ergono- mic Assessments. IISE Transactions on Occupational Ergonomics and Human Factors.7(1):43-58 (2019). Feix, T., Bullock, I. M., & Dollar, A. M. Analysis of human grasping behavior: Object characteristics and grasp type. IEEE transactions on haptics, 2014, vol. 7, no 3, p. 311-323. Feix, T., Romero, J., Schmiedmayer, H. B., Dollar, A. M., & Kragic, D. The grasp

taxonomy of human grasp types. IEEE Transactions on human-machine systems, 2015, vol. 46, no 1, p. 66-77. Hanson, L., Högberg, D., Carlson, J. S., Bohlin, R., Brolin, E., Delfs, N., ... & Rhen, I. M. IMMA–Intelligently moving manikins in automotive applications. Third International Summit on Human Simulation (ISHS2014) (2014). Lemieux, P., Cauffiez,M., Barré, A., Hagemeister, N., Aissaoui, R.: A visual acuity

constraint for digital human modeling. In: 4th Conference proceedings (2016) Lemieux, P.-O., Barré, A., Hagemeister, N., Aissaoui, R.: Degrees of freedom coupling adapted to the upper limb of a digital human model. Int. J. Hum. Factors Model. Simul. 5(4), 314–337 (2017) Macloud, A., Zeighami, A., Aissaoui, R., Rivest, L. Extracting grasping cues from

one-handed tools geometry for digital human models, International Conference on Human Systems Integration (HSI2019), Biarritz, France, 2019. Macloud, A., Zeighami, A., Aissaoui, R., Rivest, L. Extracting grasping cues from pistol-shaped tools for digital human models. Computer-Aided Design and Applications, 2021, vol. 18, no 6, p. 1167-1185. Miller, A.T., Knoop, S., Christensen, H.I. and Allen, P.K., Automatic grasp planning

using shape primitives. in Robotics and Automation, 2003. Proceedings. ICRA’03. IEEE International Conference on, (2003), IEEE, 1824-1829. V.S.R. Group. Technical report for project virtual soldier research. Center for Computer-Aided Design, the University of IOWA. 2004 Zeighami, A., Lemieux, P., Charland, J., Hagemeister, N., Aissaoui, A.: Stepping

behavior for stability control of a digital human model. ISB/ASB (2019) Zhou, W., Armstrong, T. J., Reed, M. P., Hoffman, S. G., & Wegner, D. M. Simulating complex automotive assembly tasks using the HUMOSIM framework. SAE Technical Paper, 2009.

Cover of the proceedings volume Digital Human Modeling and Applied Optimization

Explore the complete digital volume

Digital Human Modeling and Applied Optimization

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-22-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →