EASIER

30 July 2020

EASIER is a Horizon 2020 project that aims to design, develop, and validate a complete multilingual machine translation system that will act as a framework for barrier-free communication among deaf and hearing individuals, as well as provide a platform to support sign language content creation.

The project foresees a strong involvement of end-users in design, development, and evaluation of the EASIER ecosystem, where users of various European sign and spoken languages will be able to use their preferred language to interact with each other.

The project concept is based on a unique combination of technological sign language resources and sign language linguistics expertise. This allows for exploitation of a robust data-driven sign language recognition engine, the incorporation of a signing avatar that integrates sign language grammar and prosody features to perform the most advanced synthetic signing currently available, as well as the state-of-the-art machine translation technology that consumes both annotated and unannotated data to deal with a wide range of use scenarios.

Sign Language NLP

Marc Schulder

Research Associate in Computational Linguistics

My research interests include sign languages, natural language processing, and open science.

Publications

Statistical data from video processing other European SL datasets

Abstract

EASIER has established a pipeline for processing datasets from European sign languages beyond those covered by the consortium, with the aim of providing pose analysis as an important entry point for data science on sign language resources. While discussions with third parties on using this service took place and are still ongoing, no data processing happened before the end of the project. Accordingly, there are no statistical data to report. Instead, the report analyses why no such uses have taken place.

Thomas Hanke, Marc Schulder

Extended Interlingual Index for the Project's Core Sign Languages and Languages covered in Work Package 9

Abstract

The purpose of the interlingual index is to link the lexical resources of the sign languages of the project. This is the final project release of the index, which covers both the core sign languages of the project as well as additional sign languages.

Sam Bigeard, Maria Kopf, Marc Schulder, Thomas Hanke, Kiriaki Vasilaki, Anna Vacalopoulou, Theodor Goulas, Athanasia-Lida Dimou, Stavroula-Evita Fotinea, Eleni Efthimiou, Neil Fox, Onno Crasborn, Lianne Westenberg, Sarah Ebling, Laure Wawrinka, Johanna Mesch, Thomas Björkstrand, Anna Kuder, Joanna Wójcicka

Workshop and Training for Experts in Under-Resourced Languages – Session 2

Abstract

This report describes the EASIER Autumn School, a workshop held on 25–26 September 2023 in Hamburg. Titled “Sign language data meets data science – data science meets sign linguistics”, the workshop had the goal of generating expertise for under-resourced sign languages (SLs) to extend the scope of EASIER to more European SLs. The idea was to not only support people who already work with SLs and train them in technological approaches, but also to train people from data science, language technologies, etc. in the handling of SLs. Two parallel tracks introduced both groups to relevant topics of the other area, followed by a joint track addressing language technologies of the EASIER pipeline and ethical considerations.

Maria Kopf, Thomas Hanke, Marc Schulder, Sam Bigeard, Tsubasa Uchida, Giacomo Inches, Eleni Pechlivanidou

Interlingual index for the project's core sign languages

Abstract

The purpose of the interlingual index is to link the lexical resources of all sign languages of the project. This is the second version of the index, which covers all the languages of the project.

EASIER Notation – A Proposal for a Gloss-based Scripting Language for Sign Language Generation Based on Lexical Data

This article is ©2023 IEEE. I recommend using the accepted version of the paper, which is open access. For the closed access published version follow the URL in the citation below, but note that most hyperlinks have been broken in that version. The accepted version does not have this problem.

Abstract

We introduce EASIER Notation, a gloss-based scripting language to describe sign language content to be signed by an avatar and describe the functionality a lexical database for a sign language needs to provide in order to fully support the notation approach. In addition, we present the prototype of a text editor supporting EASIER Notation for human post-editing of machine translation output as well as pre-scribing signed utterances from scratch.

Thomas Hanke, Lutz König, Reiner Konrad, Maria Kopf, Marc Schulder, Rosalee Wolfe

The Sign Language Interchange Format: Harmonising Sign Language Datasets for Computational Processing

Abstract

We introduce the Sign Language Interchange Format, a new format for representing annotations and lexical inventories of sign language datasets. The format is designed as an intermediate step in data preparation for language technologies, unifying the annotation conventions of different corpora for further use. Complex gloss notations and implicit relations between tiers are made explicit through a hierarchy of machine-readable container structures. Sample implementations for converting to and from the new format are provided.

Marc Schulder, Sam Bigeard, Thomas Hanke, Maria Kopf

Tools for harmonizing available annotations to a common format

This deliverable provides tools for harmonizing available annotations to a common interchange format.

Marc Schulder, Sam Bigeard, Maria Kopf, Thomas Hanke

Facilitating the Spread of New Sign Language Technologies across Europe

Abstract

For developing sign language technologies like automatic translation, huge amounts of training data are required. Even the larger corpora available for some sign languages are tiny compared to the amounts of data used for corresponding spoken language technologies. The overarching goal of the European project EASIER is to develop a framework for bidirectional automatic translation between sign and spoken languages and between sign languages. One part of this multi-dimensional project is that it will pool available language resources from European sign languages into a larger dataset to address the data scarcity problem. This approach promises to open the floor for lower-resourced sign languages in Europe. This article focusses on efforts in the EASIER project to allow for new languages to make use of such technologies in the future. What are the characteristics of sign language resources needed to train recognition, translation, and synthesis algorithms, and how can other countries including those without any sign resources follow along with these developments? The efforts undertaken in EASIER include creating workflow documents and organizing training sessions in online workshops. They reflect the current state of the art, and will likely need to be updated in the coming decade.

Hope E. Morgan, Onno Crasborn, Maria Kopf, Marc Schulder, Thomas Hanke

Introducing Sign Languages to a Multilingual Wordnet: Bootstrapping Corpora and Lexical Resources of Greek Sign Language and German Sign Language

Abstract

Wordnets have been a popular lexical resource type for many years. Their sense-based representation of lexical items and numerous relation structures have been used for a variety of computational and linguistic applications. The inclusion of different wordnets into multilingual wordnet networks has further extended their use into the realm of cross-lingual research. Wordnets have been released for many spoken languages. Research has also been carried out into the creation of wordnets for several sign languages, but none have yet resulted in publicly available datasets. This article presents our own efforts towards an inclusion of sign languages in a multilingual wordnet, starting with Greek Sign Language (GSL) and German Sign Language (DGS). Based on differences in available language resources between GSL and DGS, we trial two workflows with different coverage priorities. We also explore how synergies between both workflows can be leveraged and how future work on additional sign languages could profit from building on existing sign language wordnet data. The results of our work are made publicly available.

Sam Bigeard, Marc Schulder, Maria Kopf, Thomas Hanke, Kyriaki Vasilaki, Anna Vacalopoulou, Theodore Goulas, Athanasia-Lida Dimou, Stavroula-Evita Fotinea, Eleni Efthimiou

The Sign Language Dataset Compendium: Creating an Overview of Digital Linguistic Resources

Abstract

One of the challenges that sign language researchers face is the identification of suitable language datasets, particularly for cross-lingual studies. There is no single source of information on what sign language corpora and lexical resources exist or how they compare. Instead, they have to be found through extensive literature review or word-of-mouth. The amount of information available on individual datasets can also vary widely and may be distributed across different publications, data repositories and (potentially defunct) project websites. This article introduces the Sign Language Dataset Compendium, an extensive overview of linguistic resources for sign languages. It covers existing corpora and lexical resources, as well as commonly used data collection tasks. Special attention is paid to covering resources for many different languages from around the globe. All information is provided in a standardised format to make entries comparable, but kept flexible enough to allow for differences in content. The compendium is intended as a growing resource that will be updated regularly.

Maria Kopf, Marc Schulder, Thomas Hanke

Initial Interlingual Index for DGS and GSL

Abstract

The purpose of the interlingual index is to link the lexical resources of all sign languages of the project. D6.3 is the first version of this index, covering DGS and GSL.

Sam Bigeard, Marc Schulder, Maria Kopf, Thomas Hanke, Kiriaki Vasilaki, Anna Vacalopoulou, Theodor Goulas, Athanasia-Lida Dimou, Stavroula-Evita Fotinea, Eleni Efthimiou

Specification for the Harmonization of Sign Language Annotations

Versions

Latest version:
Version 1:

Abstract

This document compiles information required for the joint use of several specific sign language corpora. It compares the annotations of 17 different corpora (some covering multiple languages) that include data for 17 signed languages. The comparison addresses annotation standards for manual and non-manual signs, describing their basic annotation format, formats for various specific phenomena and handshape coding approaches. This information is then used to formulate a strategy for the harmonization of these annotation standards that will result in a corpus-independent interchange format for the representation of all considered corpora.

Maria Kopf, Marc Schulder, Thomas Hanke, Sam Bigeard

Overview of Datasets for the Sign Languages of Europe

Versions

Latest version:
Version 1:

Abstract

This document identifies linguistic corpora that can be explored as high-quality training data for automatic translation within EASIER (as opposed to loosely aligned broadcast data). For each data set, the document lists what parts of the data are available under what access conditions. It also lists the elicitation formats used in several corpora in order to identify those parts of the available corpora that could be explored to build multilingual resources.