July 10, 2024 feature
Editors' notes
This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:
fact-checked
preprint
trusted source
proofread
A new model to plan and control the movements of humanoids in 3D environments
Humanoids, robotic or virtual systems with body structures that resemble the human body, have a wide range of real-world applications. As their limbs and bodies mirror those of humans, they could be made to reproduce a wide range of human movements, such as walking, crouching, jumping, swimming and so on.
Computationally generating realistic motions for virtual humanoid characters could have interesting implications for the development of video games, animated films, virtual reality (VR) experiences, and other media content. Yet the environments portrayed in video games and animations are often highly dynamic and complex, which can make planning motions for humanoids introduced in these environments more challenging.
Researchers at NVIDIA Research in Israel recently introduced PlaMo (Plan and Move), a new computational approach to plan the movements of humanoids in complex, 3D, physically simulated worlds. Their approach, presented in a paper published on arXiv preprint server, consists of a scene-aware path planner and a robust control policy.
"Controlling humanoids in complex physically simulated worlds is a long-standing challenge with numerous applications in gaming, simulation, and visual content creation," Assaf Hallak, Gal Dalal, and their colleagues wrote in their paper.
"In our setup, given a rich and complex 3D scene, the user provides a list of instructions composed of target locations and locomotion types. To solve this task, we present PlaMo, a scene-aware path planner and a robust physics-based controller."
Most previous studies aimed at planning the movements of humanoid characters in 3D simulated environments focused on the development of either a planner or a controller rather than both. As the tasks these two types of models perform (i.e., planning and executing a humanoid's motions) are interdependent, Hallak, Dalal and their colleagues set out to devise a computational approach that tackled them both.
"The path planner produces a sequence of motion paths, considering the various limitations the scene imposes on the motion, such as location, height, and speed," Hallak, Dalal and their colleagues wrote. "Complementing the planner, our control policy generates rich and realistic physical motion adhering to the plan."
The researchers tested PlaMo in a series of simulations, using it to plan and execute the movements of the SMPL humanoid, a humanoid virtual agent with a neutral body structure (i.e., no facial features, hair, clothes, etc.). They ran their simulations on IsaacGym, a physics-based simulation environment for reinforcement learning developed at NVIDIA.
The results of these tests were highly promising, as the PlaMo approach was found to effectively plan and execute the movements of SMPL humanoids in complex simulated landscapes, following textual instructions. Notably, the approach's planner component was found to account for the humanoid character's movements on uneven terrains, as well as static and dynamic obstacles in the environment.
The motion controller was, in turn, found to reliably track the paths planned by the planner, executing complex and scene-aware movements for the humanoids, such as crawling under a low-hanging ceiling or rapidly moving to avoid an approaching obstacle.
Overall, the combination of the two modules was highly effective, producing realistic movements in response to changes in the environment.
"PlaMo focuses on planning and control," Hallak, Dalal and their colleagues wrote. "It opens opportunities for combining it with modern language models and 3D scene understanding. With these expansions in mind, we see PlaMo as a stepping stone in a much greater system where non-playable characters (NPCs) are given roles to play, forming a rich simulated virtual world."
More information: Assaf Hallak et al, PlaMo: Plan and Move in Rich 3D Physical Environments, arXiv (2024). DOI: 10.48550/arxiv.2406.18237
Journal information: arXiv
© 2024 Science X Network
Citation: A new model to plan and control the movements of humanoids in 3D environments (2024, July 10) retrieved 10 July 2024 from https://techxplore.com/news/2024-07-movements-humanoids-3d-environments.html This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
Explore further
The MIT humanoid robot: A dynamic robotic that can perform acrobatic behaviors 39 shares
Feedback to editors
