Human activity recognition with openpose and Long Short-Term Memory on real time images

EasyChair Preprint 2297

Abstract

Human Activity Recognition(HAR) is a broad field of study aims to classify time series activities. These activities can include normal body movements such as standing up, sitting down, jumping, walking etc. Whenever activity is performed by a person, it usually few seconds. Image classification algorithms fails to classify such stream of images into activity class. Existing approaches rely on the sensor data recorded by accelerometer, smart phones, or some harnessing devices to recognize the movements. Such data is challenging and expensive to collect and requires multiple sensors, custom hardware and software. In this paper, I have described a systematic method to recognize human activities in real time using Openpose and Long short-term memory networks. This approach is based on the images that are captured in real time by connecting the camera and fetching the timed screenshots. Openpose is an open source, real time project to jointly detect human body with hands, face, facial expressions, legs on single image. Output of body features are split into sub-sequence called window using sliding window approach. Long Short Term Memory(LSTM) is a Recurrent Neural Network(RNN) different from feed forward Neural Networks. It has feedback connections so it can process single data points as well as sequence of data. LSTM is suitable for this scenario and provides improved results. It efficiently learns the key point features and returns an activity class. This system detects Real time human activities such as waving 1 hand, waving 2 hands, jumping_jacks, boxing, jumping, clapping.

Keyphrases: Convolutional Neural Networks, LSTM, OpenPose, image processing, sliding-window approach

@booklet{EasyChair:2297,
  author    = {Chinmay Sawant},
  title     = {Human activity recognition with openpose and Long Short-Term Memory on real time images},
  howpublished = {EasyChair Preprint 2297},
  year      = {EasyChair, 2020}}