 Open Access
 Authors : Pradnya Krishnanath Borkar , Marilyn Mathew Pulinthitha , Mrs. Ashwini Pansare
 Paper ID : IJERTV8IS100253
 Volume & Issue : Volume 08, Issue 10 (October 2019)
 Published (First Online): 31102019
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
Match Pose – A System for Comparing Poses
Pradnya Krishnanath Borkar Computer Department, FRCRCE
Marilyn Mathew Pulinthitha Computer Department, FRCRCE
Mrs. Ashwini Pansare Computer Department, FRCRCE
Abstract: The proposed system presents a simple method to compare users realtime pose with any selected pose. The system uses PoseNet algorithm for realtime pose estimations. The pose comparison algorithm used in system allows a person to compare his/her realtime pose to check if it is successfully imitated.
INTRODUCTION:
In this busy world, many people wish to go to yoga classes or gym but may not have time or money to do so. The proposed system can help such people to do yoga or dance steps at home without any need to spend a penny. Moreover, proposed system only needs a webcam and is accessible through any browser.
Proposed system allows the user to select an image which he/she wishes to imitate. The realtime poses of the user are then captured through webcam and processed using human poseestimation algorithm. The selected image from database is also processed using the same algorithm. At the end, the system compares both these poses and prompts the user whether he/she has successfully imitated the pose.
METHOD OVERVIEW:
The system is a web application which only requires a browser to run it. The system uses PoseNet algorithm to detect human poses. When the user selects a desired image that he wants to imitate, the selected image is processed using single pose algorithm and a set of 17 key points are stored in the system.
POSE ESTIMATION:
Human pose estimation algorithm allows us to estimate human poses accurately. A detailed overview between some of the human pose estimation algorithms are mentioned below.
Many of the Machine Learning systems as mentioned in [1], [2], [3] needs a 2D human pose estimation as their input. To help the research community OpenPose was released publicly. OpenPose[8] is a multiperson algorithm which successfully detects 135 keypoints on a single human image. It consists of three blocks namely a) body and foot detection b) hand detection c) face detection. It can run 22 FPS on a system with Nvidia GTX 1080Ti. A detailed explanation can be obtained in the [9].
PoseNet [10] is another example of human pose estimation algorithm which is widely used. PoseNet is able to detect 17 keypoints in a single human image. PoseNet offers single pose algorithm which can detect keypoints of one human at a time Or multipose algorithm which detects multiple person at a particular point of time. A detailed explanation can be seen in [11].
Figure 1: 17 keypoints using PoseNet [12]
Now the webcam captures the realtime pose of the user. This feed is continuously being processed by the multi pose algorithm of PoseNet. A red colored skeleton formed by joining the 17 key points is displayed. The moment the red skeleton turns green, the user is said to achieve his desired pose.
Under the hood the 17 key points obtained from both the image selected and the webcam is compared using our comparison algorithm. The moment the pose matches each other , the red skeleton turns green.
COMPARISON ALGORITHM:
One of the possible ways you could think of comparing is by superimposing the two skeletons. But you need to transform one of the skeletons such that it overlaps the other. You need to keep in mind that every person has a unique physical structure. Hence length of the arm or legs of each person will differ, hence mapping the skeletons can be a difficult task. Hence the solution to this problem is that
instead of mapping the skeletons length, the need is to compare the angles between their joints. The angles are independent of the length of the skeleton hence; they are constant for the maximum time.
Figure 1, refers the angles which are taken into consideration for comparing the pose. As shown in the figure, we consider the elbow and the shoulder angle for comparing the arm position mentioned in Figure 2.
Figure 2: Angles considered in a pose
Along with these angles one more factor is needed.
There is a possibility that these angles are same for both arms which is to be calculated.
Figure 3 illustrates the calculation of the angle.
As shown above to distinguish between both the positions of the arm, the abdominal area becomes the base. The system compares the shoulder coordinates and the wrist coordinates of an arm to locate if the arm is inward, outward, upward or downward. Similarly, for the leg comparison the system uses Hip angle and Knee angle (Figure 2) and for the position of the leg it compare the hip coordinates with the ankle co ordinates.
a
Angle_B=Math.acos
((Math.pow (len (b, c), 2) +
Math.pow (len (a, b),2) –
Math.pow (len (a, c),2))/ (2*len
(b, c) *len (a, b)))*180/Math.PI
a
Angle_B=Math.acos
((Math.pow (len (b, c), 2) +
Math.pow (len (a, b),2) –
Math.pow (len (a, c),2))/ (2*len
(b, c) *len (a, b)))*180/Math.PI
Figure 4(a) Arm inwards
Figure4(b) Arm outwards
Figure 4(a) Arm inwards
Figure4(b) Arm outwards
Figure 4: Different Arm position with equal angles
After obtaining the right elbow angle of both the position in figure 4(a) and 4(b) a situation may be encountered wherein both the angles may be same yet the positions are so different. Hence the extra factor that was discussed earlier is needed, which means , a need to compare the right shoulder coordinates of posture in 4(a) and 4(b) with its right wrist position. If the hand position is inwards then the wrist x co ordinate on a Cartesian plane would be less than the x coordinate of the shoulder. Hence, System is able to distinguish between different poses by calculating the angles of the joints and comparing the coordinates of the keypoints.
CONCLUSION:
Matchpose is successfully able to compare poses. This system can be used for learning new yoga and dance poses. Although this system is working with PoseNet, it can be integrated with another pose estimation algorithm. The main component is the position that is obtained from the estimation algorithm. Currently this system is only comparing the 17 key points mentioned above.
Hence, the dance or yoga poses involving finger positions cannot be compared. That can be included in the system in future as it will follow similar concept as that for arms and legs.
b
b
c
c
REFERENCES:
Figure 3: Formula for calculating the Angle between joints
For calculating the angles, the system uses the cosine triangle rule as mentioned in Figure 3.
Suppose we are calculating the elbow angle; the co ordinates of shoulder, elbow and wrist forms an imaginary triangle with the elbow point as the middle angle (refer triangle ABC in figure 3).

X. Qian, Y. Fu, T. Xiang, W. Wang, J.Qiu, Y. Wu, Y.G. Jiang, and X. Xue,Posenormalized image generation for person reidentification, in ECCV, 2018.

A. Bansal, S. Ma, D. Ramanan, and Y.Sheikh, Recyclegan: Unsupervisedvideo retargeting, in ECCV, 2018.

L. Gui, K. Zhang, Y.X. Wang, X. Liang, J. M. Moura, and
M. M.Veloso, Teaching robots to predict human motion, in IROS, 2018.

P. Panteleris, I. Oikonomidis, and A. Argyros, Using a single rgb frame for real time 3d hand pose estimation in the wild, in IEEE WACV. IEEE, 2018, pp. 436445.

H. Joo, T. Simon, and Y. Sheikh, Total capture: A 3d deformation model for tracking faces, hands, and bodies, in CVPR, 2018.

D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin, M. Shafiei, H.
P. Seidel, W. Xu, D. Casas, and C. Theobalt, Vnect: Realtime 3d human pose estimation with a single rgb camera, in ACM TOG, 2017.

G. Hidalgo, Z. Cao, T. Simon, S.E. Wei, H. Joo, and Y. Sheikh, OpenPose library, https://github.com/CMUPerceptual
ComputingLab/openpose

Zhe Cao, Gines Hidalgo, Tomas Simon, ShihEn Wei, and Yaser Sheikh, OpenPose: Realtime MultiPerson 2D Pose Estimation using Part Affinity Fields

Posenet Library, https://github.com/tensorflow/tfjs models/tree/master/posenet

Tensorflow Blog, Realtime Human Pose Estimation in the Browser with TensorFlow.js, https://medium.com/tensorflow/realtime humanpose estimationinthebrowserwith tensorflowjs7dd0bc881cd5

Description of 17 keypoints detected by PoseNet, Tensorflow Blog