Mark has served as the Executive Director of the European Union of the Deaf (EUD) since 2007. He promotes the Deaf community across Europe and attends all European-level meetings related to deaf/disability issues. He is also responsible for building and strengthening the EUD networking with the EU institutions, as well as the communication with EUD member organizations. Mark managed several H2020 projects, such as SignSpeak and NEXES. He is currently managing numerous projects, namely AiD, SignON, SHAPES, JustiSigns2, Deaf Museums, DESEAL, and Access for All (European Parliament). He also contributed to an article about sign language and machine learning published in the Economist and other publications. He also co-authored several articles on sign languages (e.g., Comparative analysis of real-time multi-view reconstruction of a sign language interpreter; Multi-view sensing and real-time 3D reconstruction of sign language interpreter).
The ever-growing use of mobile applications continues to improve the daily lives of European citizens. To what extend does the Deaf community benefit from the ongoing digital transformation of society? What are the challenges?
Currently, most mobile applications that involve sign languages enable sign language lexicon acquisition for hearing adults, i.e., show signs for individual spoken-language words. Another set of sign language applications focuses on fingerspelling as a subsystem of sign languages, basically teaching users to understand fingerspelling. Applications that perform translation from spoken language to sign language are mere dictionary lookup tools that substitute each spoken language word with a sign. Unfortunately, this is irrespective of the differences in word order between spoken languages and sign languages, thereby instantiating the outdated paradigm of word-by-word machine translation. Applications that offer bidirectional translation (from and into sign language) for multiple languages are distinctly rare.
How does EASIER plan to address these challenges? What role will human-centric solutions, like EASIER, play in facilitating communication between deaf and hearing persons?
Firstly, we need to consider the limited performance of mobile phone hardware and the bandwidth requirements for the bi-directional transfer of multimedia data. EASIER will balance between the performance and the costs of phones. The newest phones with the most up-to-date operating systems and high bandwidth capabilities (4G/5G) can surely provide high performance and fulfill the needs of a system but these devices are expensive and therefore not available to all. EASIER will choose a combination phone/OS that provides just enough performance and the needed version of the OS not to leave anyone behind. The EASIER application will be one of the first applications offering fully automatic translation on a sentence level for multiple European spoken/sign language pairs, with the possibility of extending support to additional language pairs. The application will be available in the Play Store free of charge.
Secondly, we plan to target the existing challenges by validating them with the following user groups:
- Deaf citizens, providing them with a robust translation system capable of supporting their daily interaction needs,
- Broadcasters, providing them with tools that accelerate robust production of high-quality sign language content,
- Sign language professionals, boosting their efficiency in accomplishing their job tasks.
What are the project objectives?
EASIER’s goal is to create a framework for barrier-free communication among deaf and hearing citizens across the European Union by enabling users of sign languages to use their preferred language to interact with hearing persons. Similarly, users of spoken languages will be able to use their preferred language to communicate with deaf individuals. The objectives of our project are defined around three main pillars:
- The exploitation of a robust data-driven sign language recognition engine,
- The utilization of a signing avatar engine that not only produces easy to comprehend signing but also integrates information on affective expressions and coherent prosody,
- The incorporation of state-of-the-art machine translation technology capable of dealing with a wide range of languages and communication scenarios.
EASIER targets the following languages:
- Spoken: English, French, Italian, German, Dutch, and Greek.
- Sign: BSL (British), LSF (French), LIS (Italian), DGS (German), DSGS (Swiss-German), NGT (Dutch), and ΕΝΓ (Greek).
EASIER has a large consortium – what areas of expertise do various members bring to the project?
EASIER is in a unique position with regard to human resources. To ensure a wide range of skills and technical knowledge needed to pursue the objectives of our project, we formed a consortium of 14 partners from eight European countries. Our consortium exhibits a strong combination of expertise in the sectors of sign language linguistics, sign language resources creation, machine translation, signing avatar technology, sign language video processing, and affect recognition. Our team includes representatives from the industry, higher education and research institutions, non-governmental organizations, and SMEs. Most of our partners have already worked with different European Deaf communities to develop various resources, tools, and services. It is also worth mentioning that we have access to the three largest sign language corpora in Europe and huge amounts of unannotated data through the involvement of a partner who broadcasts programs in three spoken and three sign languages. Last but not least, our project involves deaf personnel in the entire research cycle, not just the evaluation studies.
Stakeholders’ involvement is crucial for the success of any project. What is your project’s approach? Are members of the Deaf community involved in EASIER?
EASIER targets a large community of potential users across the European Union. The involvement of various groups of citizens is therefore crucial to our project. We intend to bring together not only the people with and without hearing impairments but also the stakeholders from the human sciences and innovative technologies domains, representing both the industry and the research sectors. Effective collaboration of these key players is necessary to ensure the design of an effective, user-driven, inclusive, human-centric, and open-source solution aligned with the Next Generation Internet initiative values. To maximize the impact of our work and project results, we will engage the Deaf community around the planned use cases and the players involved in their implementation. The various communities to which our partners already belong are the starting point for the extended and growing EASIER community. To ensure the best possible adoption of provided solutions, the end users who are the members of the Deaf community will remain at the center of our activities.
What benefits can be expected from EASIER and what is the timeline for the deployment of your application?
Among the benefits expected from EASIER are:
1. An improved innovation capacity
Innovation capacity is a company’s ability to identify new ideas and transform them into new or improved products, services, or processes that benefit the firm. EASIER will be able to provide an efficient and low-cost addition to a firm’s capability to enlarge its customer list by addressing the customer population with hearing impairments. It will also be able to facilitate the recruitment process hence opening more companies to highly qualified deaf employees. EASIER will foster this idea by generating an ecosystem encouraging SMEs to use applications built on the project framework. EASIER consortium members will be among the first end users. One of our partners who is active in sign language content creation and accessibility services plans to incorporate EASIER in their offer to grow their customer base. Another EASIER partner who is active in sign language interpreting and translation services plans to incorporate EASIER into their procedures targeting the adoption of innovation in their business processes.
2. Socioeconomic impact
A growing number of studies and reports show that the majority of deaf people who work in a hearing environment face daily barriers and challenges related to communication. The services provided by EASIER will open new possibilities for deaf persons who interact in predominantly hearing settings, such as workplaces. By facilitating communication at the workplace for deaf people throughout Europe, EASIER will have an intermediate and long-term socioeconomic impact on this group.
3. Impact on individual lives
Although we do not know a precise number of people who live with a hearing loss, WHO estimates that there are 466 million persons in the world with a disabling hearing loss (6.1% of the world’s population). At the same time, EFHOH estimates that 51 million EU citizens (9% of the population) live with a hearing loss. On average, deaf sign language users make up about 0.1% of the whole population in any given country. This does not include people learning sign languages, such as the children of deaf parents (CODAs). It is estimated that there are 750,000 deaf sign language users in the European Union.
Within this context, EASIER will facilitate the communication between sign language and spoken language communities across the European Union. Our technology will have a direct impact on deaf and hearing individuals’ lives and their everyday interactions. Since communication barriers will decrease, people will be able to expand their social and professional interactions and culture transfer. EASIER will also function as a tool bringing the language cultures of the deaf Europeans to light while attracting new sign language learners from the hearing population.