Researchers at Cornell University have created a first-of-its-kind wristband that tracks the posture of the entire body in three dimensions using a tiny camera and a specialized deep neural network.
BodyTrak is the first wearable that can track a full body pose using just one camera. Cheng Zhang, an assistant professor of information science and the paper’s senior author, believes that BodyTrak could revolutionize the way that user body mechanics are monitored during physical activities where accuracy is crucial.
“Since smartwatches already have a camera, technology like BodyTrak could understand the user’s pose and give real-time feedback,” Zhang said. “That is practical, reasonably priced, and does not restrict the user’s range of motion.”
A corresponding paper, “BodyTrak: Inferring Full-body Poses from Body Silhouettes Using a Miniature Camera on a Wristband,” was published in the At UbiComp 2022, the international ACM conference on pervasive and ubiquitous computing, the Association for Computing Machinery (ACM) presented proceedings on interactive, mobile, wearable, and ubiquitous technology.
The newest body-sensing device from SciFiLab, which is housed in the Cornell Ann S. Johnson Medical Center, is called BodyTrak. Bowers College of Computing and Information Science, a team that has previously created and used comparable deep learning models to track hand and finger movements, facial expressions, and even silent-speech recognition.
The custom deep neural network that powers BodyTrak is just as important as the tiny camera on the wrist. This deep neural network – a method of AI that trains computers to learn from mistakes – reads the camera’s rudimentary images or “silhouettes” of the user’s body in motion and virtually re-creates 14 body poses in 3D and in real time.
In other words, according to Hyunchul Lim, a doctoral student in the field of information science and the paper’s lead author, the model precisely fills out and completes the incomplete images that the camera captured.
“Our research shows that we don’t need our body frames to be fully within camera view for body sensing,” Lim said. “There is a lot of information to infer in order to reconstruct the entire body if we can only capture a portion of it.”
When creating these technologies, Zhang and Lim said that maintaining privacy for bystanders close to someone wearing such a sensing device is a legitimate concern. Since the camera is pointed at the user’s body and only takes partial body shots of the user, they claimed BodyTrak reduces privacy concerns for onlookers.
They also acknowledge that although full body sensing is not currently supported by smartwatches due to their insufficiently large or powerful cameras or battery life, it may be in the future.
Along with Lim and Zhang, the paper also includes co-authors Matthew Dressa ’22, Jae Hoon Kim ’23, and Ruidong Zhang, a doctoral candidate in information science, as well as Yaxuan Li of McGill University and Fang Hu of Shanghai Jian Tong University.
Read More: Best Sony Camera For Video
Source: Cornell Chronicle
