Detect and save skeleton with Azure Kinect【Azure Kinect Practical course#2】

Introduction

This article describes how to detect the skeleton with Azure Kinect and how to save the skeleton position with csv format.

Please check here to check the specifications required for the environment and PC to run Azure Kinect.

Body Tracking SDK

In order to detect the skeleton, the Body Tracking SDK must be installed on the PC.

Download the latest version from the “msi” link and install it. If you can start the Body Tracking Viewer by typing “body” from the windows start button, the installation is complete.

When you actually start it and detect a person, the skeleton part is colored as follows.

Has the skeleton detection accuracy improved?

Skeleton detection was also possible with the conventional Kinect v2. Azure Kinect has further improved skeleton detection accuracy. Specifically, v2 uses only the difference in depth information around pixels as a feature, but Azure Kinect detects it on a deep learning basis, making it strong against disturbances due to environmental factors such as when holding an object. Furthermore, the front and back of the body can be identified.

From the comparison video below, you can see that the front and back of the body are properly recognized. Something like a low-pass filter is built in in the initial state, and noise at the estimated part of the skeleton is also suppressed.

How to save skeletal position

You can preview the skeleton position but not save it just by installing the SDK. If you install the following tools, you can save the 3D skeleton position in csv format and use it when analyzing data.

AKRecorder starts recording the skeleton coordinate value (csv) and the color image (png) from the Depth camera viewpoint when the person appears, and finishes the recording when the person leaves. The recorded file is saved for each directory with the date and the time stamp at the start of recording, and supports up to 6 people.

AKRecorder can be executed by opening Csharp_3d_viewer.sln in Visual Studio and building it. In PosSaver.csv in advance, specify the path to save the data (line 33). If you get an error at build time, most of the dependent packages are missing, so install them as appropriate. If you want to visualize the skeleton position during recording, comment out the part related to GUI drawing in the same code.

Conclusion

Described how to detect the skeleton and save the skeleton coordinate values ​​using the Azure Kinect Body Tracking SDK and AK Recorder.

Bio

正田 孝平

正田 孝平

学生ブロガー兼エンジニア(東工大院21卒)
AI,IoT,web最新技術の具体的活用術、キャッシュレス決済と資産運用の最新トレンドや語学の体系的学習法について発信。在学中は人力飛行機の製作、webアプリ開発に積極的に携わった。
【趣味】
球技、水泳、マラソン、スノーボード