Estudio de bases de datos para el reconocimiento automático de lenguas de signos

  1. Darío Tilves Santiago 1
  2. Carmén García Mateo 1
  3. Soledad Torres Guijarro 1
  4. Laura Docío Fernández 1
  5. José Luis Alba Castro 1
  1. 1 Universidade de Vigo
    info

    Universidade de Vigo

    Vigo, España

    ROR https://ror.org/05rdf8595

Revista:
Hesperia: Anuario de filología hispánica

ISSN: 1139-3181

Ano de publicación: 2019

Número: 22

Páxinas: 145-160

Tipo: Artigo

DOI: 10.35869/HAFH.V23I0.1658 DIALNET GOOGLE SCHOLAR lock_openDialnet editor

Outras publicacións en: Hesperia: Anuario de filología hispánica

Resumo

Automatic sign language recognition (ASLR) is quite a complex task, not only for the difficulty of dealing with very dynamic video information, but also because almost every sign language (SL) can be considered as an under-resourced language when it comes to language technology. Spanish sign language (LSE) is one of those under-resourced languages. Developing technology for SSL implies a number of technical challenges that must be tackled down in a structured and sequential manner. In this paper, some problems of machine-learning- based ASLR are addressed. A review of publicly available datasets is given and a new one is presented. It is also discussed the current annotations methods and annotation programs. In our review of existing datasets, our main conclusion is that there is a need for more with high-quality data and annotations.

Referencias bibliográficas

  • Athitsos, Vassilis; Neidle, Carol; Sclaroff, Stan; Nash, Joan; Stefan, Alexandra; Yuan,Quan; y Thangali, Ashwin. (2008). “American Sign Language Lexicon Video Dataset(ASLLVD)”. Workshop on Human Communicative Behaviour Analysis.
  • Cabeza, Carmen y García-Miguel, José María (2019). “iSignos: Interfaz de datos de Lengua de Signos Española (versión 1.0)” Universidade de Vigo. http://isignos.uvigo.es
  • Cabeza, Carmen y García-Miguel, José María; García-Mateo, Carmen y Alba-Castro, Jose Luis. (2016). “CORILSE: a Spanish Sign Language Repository for Linguistic Analysis” 10th conference on International Language Resources and Evaluation (LREC’16), EuropeanLanguage Resources Association (ELRA)
  • Chételat-Pelé, Emilie y Braffort, Annelies . (2008). “Sign Language Corpus Annotation: toward a new Methodology”. LREC.
  • Cihan Camgoz, Necali; Hadfield, Simon; Koller, Oscar y Bowden, Richard. (2017). “SubUNets: End-to-end Hand Shape and Continuous Sign Language Recognition”.2017 IEEE International Conference on Computer Vision (ICCV).
  • Cihan Camgoz, Nicati; Hadfield, Simon; Koller, Oscar; Ney, Hermann y Bowden, Richard. (2018). “Neural Sign Language Translation”.IEEE conference on Computer Visionand Pattern Recognition (CVPR) 2018.
  • Clark, A. (2012). “How to Write American Sign”. ASL write.
  • Crasborn, Onno; Hulsbosch, Micha; Sloetjes, Han; Schmer, Trude y Harmsen, Hessel.(2011). “Sign LinC: Linking lexical databases and annotated corpora of signed languages”.Centre for Language Studies, Radboud University Nijmegen; Max Planck Insititute for Psycho-linguistics; Dutch Sign Centre.
  • Donahue, Jeff; Hendricks, Lisa Anne; Rohrbach, Marcus; Venugopalan, Subhashini; Guadarrama, Sergio; Saenko, Kate y Darrel Trevor (2016). “Long-Term Recurrent Convolutional Networks for Visual Recognition and Description”.Computer Vision and Pattern Recognition (CVPR 2015) of IEEE.
  • Dreuw, Philippe y Ney, Hermann. (2008). “Towards Automatic Sign Language Annotation forthe ELAN Tool”.In Procs. of Int. Conf. LREC Wkshp: Representation and Processing of Sign Languages. Marrakech, Morocco.
  • Dreuw, Philippe; Rybach, David; Deselaers, Thomas; Zahedi, Morteza y Ney, Hermann.(2007). “Speech Recognition Techniques for a Sign Language Recognition System”.Interspeech. Fenlon, Jordan; Cormier, Kearsy; Rentelis, Ramas; Schembri, Adam; Rowley, Katherine; Adam, Robert y Woll, Bencie (2014).“BSL SignBank: A lexical database of British Sign Language (First Edition)”. London: Deafness, Cognition and Language Research Centre, University College London. https://bslsignbank.ucl.ac.uk/
  • Filhol, Michael y McDonald, John (2018). “Extending the A Zee Paula Shortcuts to Enable Natural Proform Synthesis”. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
  • Filhol, Michael; Hadjadj, Mohamed y Choisier, Annick. (2014). “Non-manual features: the rightto indifference”.Reykjavik, Iceland: Representation and Processing of Sign Languages: Be-yond the manual channel, Language resource and evaluation conference (LREC).
  • Filhol, Michael; McDonald, John y Wolfe, Rosalee. (2017). “Synthesizing Sign Language by Connecting Linguistically Structured Descriptions to a Multi-track Animation System”. UniAccess in Human–Computer Interaction. Designing Novel Interactions. pp 27-40.
  • Forster, Jean; Schmidt, Cristoph; Hoyoux, Thomas; Koller, Oscar; Zelle, Uwe; Piater, Justus y Ney, Hermann. (2012). “RWTH-PHOENIX-Weather: A Large Vocabulary Sign Language Recognition and Translation Corpus”.Computer Vision and Image Understanding.
  • García Mateo, C. (2019). “LSE_LEX40_UVIGO: una base de datos específicamente diseñada para el desarrollo de tecnología de reconocimiento automático de LSE”. CNLSE 2019. https://www.youtube.com/watch?v=zCDOg5LGkWQ .
  • García, Brigitte y Sallandre, Marie-Anne. (2013). “Transcription systems for sign languages: asketch of the different graphical representations of sign language and their characteristics”. Handbook ”Body-Language-Communication”,Mouton De Gruyter.pp.1125-1338.
  • Guo, Dan; Zhou, Wengang; Wang, Meng y Li Houqiang. (2016). “Sign Language Recognition Based on Adaptative Hmms with Data Augmentation”.IEEE.
  • Hanke, T. (2002). “iLex - A Tool for Sign Language Lexicography and Corpus Analysis”. Proceedings of the Third International Conference on Language Resources and Evaluation. https://www.sign-lang.uni-hamburg.de/ilex/ .
  • Hanke, T. (2004). “Ham No Sys-representing sign”. LREC.Vol. 4. 1–6
  • Hochgesang, Julie A.; Crasborn, Onno y Lillo-Martin, Diane. (2017). “ASL SignBank”. New Haven, CT: Haskins Lab, Yale University. https://aslsignbank.haskins.yale.edu/
  • Isaacs, Jason y Foo, Simon. (2004). “Hand Pose Estimation for American Sign Language Recognition”. Thirty-Sixth Southeastern Symposium on System Theory of IEEE. Atlanta, GA, USA.
  • Johnston, Trevor; Allen, Julia; Banna, Karin; Cresdee, Donovan; De Beuzeville, Louise; Ferrara, Lindsay; Fried, Dani; Goswell, Della; Gray, Michael; Hatchard, Ben; Hodge, Gabrielle; Schembri, Adam; Shearim, Gerry; Van Roekel, Jane y Whynot, Lori . (2008). “Auslan Signbank”. http://www.auslan.org.au/
  • Jongejan, B. (2016). Anvil Facetracker.Universidad de Copenhague. https://github.com/kuhumcst/Anvil-Facetracker .
  • Kipp, M. (2012). “Multimedia Annotation, Querying and Analysis in ANVIL”. Multimedia Information Extraction: Advances in Video, Audio, and Imagery Analysis for Search, Data Mining, Surveillance and Authoring. Publisher: Wiley, Editors: M. Maybury, pp.351-368.
  • Kipp, Michael; Martin, Jean-Claude; Paggio, Patrizia y Heylen, Dirk. (2009). “From Models of Natural Interaction to Systems and Applications”.
  • Koller, Oscar; Ney, Hermann y Bowden, Richard. (2016). “Automatic Alignment of HamNoSys Subunits for Continuous Sign Language Recognition”.LREC Workshop on the Representation and Processing of Sign Languages: Corpus Mining. Portoroz, Slovenia, pp. 121-128.
  • Koller, Oscar; Ney, Hermann y Bowden, Richard. (2016). “Deep Hand: How to Train a CNNon 1 Million Hand Images When Your Data Is Continuous and Weakly Labelled”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas, NV, USA, June2016, páginas 3793-3802.
  • Kristoffersen, Jette H. y Troelsgard, Thomas. (2010). “Danish Sign Language Dictionary”. Proceedings of the 14th EURALEX International Congress. http://www.tegnsprog.dk/
  • Lydell, Thomas y European Sign Language Center. (2006). “Spread The Sign”. https://www.spreadthesign.com Retrieved 10 11, 2019
  • Memo, Alvise; Minto, Ludovico y Zanttigh, Pietro. (2015). “Exploiting Silhouette Descriptors and Synthetic Data for Hand Gesture Recognition”.Eurographics Italian Chapter Conference.
  • Neidle, Carol; Sclaroff, Stan y Athitsos, Vassilis. (2001). “Sign Stream: A tool for linguistic and computer vision research on visual-gestural language data”. Boston University, Boston, Massachusetts, Behavior Research Methods, Instruments, Computers. https://www.bu.edu/asllrp/SignStream/3/
  • Nunnari, Fabrizio; Filhol, Michael; y Héloir, Alexis. (2018). “Animating A Zee Descriptions Using Off-the-Shelf IK Solvers”.Proceedings of the 8th LREC Workshop on the Representation and Processing of Sign Languages. Miyazaki, Japan.
  • Oorfanidou, Eleni; Woll, Bencie y Morgan, Gary. (2015). “Research Methods in Sign Language Studies: A Practical Guide”.
  • Ronchetti, Franco; Quiroga, Facundo; Estrebou, Cesar; Lanzarini, Laura y Rosete, Alejandro. (2016). “LSA64: An Argentinian Sign Language Dataset”. Congreso Argentino de Ciencias de la Computación.
  • Schiel, F.(2009). “BAS Validation Report for the SIGNUM Database”. BAS Bayerisches Archiv für Sprachsignale, Institut für Phonetik, Universität München.
  • Schmidt, Thomas; Elenius, Kjell y Trilsbeek Paul. (2010). “Multimedia Corpora (Media encoding and annotation)”.Interoperability and Standards.CLARIN-D5C-3. Ed.: Erhard Hinrichs, Iris Vogel. CLARIN - Common Language Resources and Technology Infrastructure
  • Shi, Xingjian; Chen, Zhourong; Wang, Hao; Yeung, Dit-Yan; Wong, Wai-kin y Woo, Wang-Chun. (2015). “Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting”. Neural Information Processing Systems (NIPS).
  • Sun, Xiao; Wei, Yichen; Liang, Shuang; Tang, Xiaoou y Sun, Jian. (2015). “Cascaded Hand Pose Regression”.CVPR2015. IEEE.
  • Sutton, V. (2000). Sign Writing. Deaf Action Committee (DAC) for Sign Writing.
  • Tilves, Darío; Benderitter, Ian y García-Mateo, Carmen. (2018).“Experimental Frame-work Design for Sign Language Automatic Recognition”.Proc. Iber SPEECH 2018.72-76,DOI: https://doi.org/10.21437/IberSPEECH.2018-16
  • Tsironi, Eleni; Barros, Pablo; Weber, Cornelius y Wermter, Stefan. (2016). “An Analysis of Convolutional Long Short-Term Memory Recurrent Neural Networks for Gesture Recognition”. Neurocomputing.
  • Zahedi, Morteza; Keysers, Daniel; Deselaers, Thomas y Ney, Hermann. (2005). “Combination of Tangent Distance and an Image Distortion Model for Appearance -Based Sign Language Recognition”.Springer Verlag
Fonte de datos: Dialnet