Simultaneous Segmentation and Recognition: Towards more accurate Ego Gesture Recognition
File Type:
PDFItem Type:
Conference PaperDate:
2019Access:
openAccessCitation:
Chalasani, T. & Smolic, A., Simultaneous Segmentation and Recognition: Towards more accurate Ego Gesture Recognition, 2019Download Item:
Abstract:
Ego hand gestures can be used as an interface in AR and
VR environments. While the context of an image is impor-
tant for tasks like scene understanding, object recognition,
image caption generation and activity recognition, it plays a
minimal role in ego hand gesture recognition. An ego hand
gesture used for AR and VR environments conveys the same
information regardless of the background. With this idea
in mind, we present our work on ego hand gesture recogni-
tion that produces embeddings from RBG images with ego
hands, which are simultaneously used for ego hand seg-
mentation and ego gesture recognition. To this extent, we
achieved better recognition accuracy (96.9%) compared to
the state of the art (92.2%) on the biggest ego hand gesture
dataset available publicly. We present a gesture recognition
deep neural network which recognises ego hand gestures
from videos (videos containing a single gesture) by gener-
ating and recognising embeddings of ego hands from image
sequences of varying length. We introduce the concept of
simultaneous segmentation and recognition applied to ego
hand gestures, present the network architecture, the train-
ing procedure and the results compared to the state of the
art on the EgoGesture dataset [31].
Sponsor
Grant Number
SFI stipend
15/RP/277
Author's Homepage:
http://people.tcd.ie/smolicaDescription:
PUBLISHED
Author: Smolic, Aljosa; Chalasani, Tejo
Sponsor:
SFI stipendType of material:
Conference PaperCollections
Availability:
Full text availableKeywords:
Hand gestures, Virtual reality, Augmented realitySubject (TCD):
Creative Technologies , Multimedia & CreativityMetadata
Show full item recordLicences: