A silent challenge

The Language Council and NorwAI

Is AI a tool for sign language?

The Language Council of Norway is concerned with Norwegian sign language - a minority language with official status in Norway used by 16 500 people. Digitalization of sign language represents some peculiar challenges. 

Sign languages vary from country to country. For example, it is not possible for a Norwegian to understand Swedish sign language without further ado. 

This makes it difficult to build good tools for understanding and generating Norwegian sign language. Another problem is that it is very demanding to create annotated datasets for sign language. 

NorwAI was contacted by The Language Council. We have discussed language models including whether the models can be applied to special needs, says NorwAI Center director, professor Jon Atle Gulla. 

When you transcribe speech, you get a linear text that is easy to use further. When transcribing sign language, one must use what is called a score transcription, i.e., there are many simultaneous movements / patterns that must be transcribed, he explains. 

This is both difficult and very time consuming. There are few computer resources in Norwegian for sign language. There is a small environment at NTNU Dragvoll, but their datasets are hardly precise enough for use in machine learning according to professor Gulla. 

A European research project, SignOn, however, works with sign language recognition. The intention of this project is to use AI / machine learning to recognize sign language. The Language Council had then been in contact with SignOn and was willing to mediate contact with the sign language communities in Dublin and Ghent.

A project will have to include both techniques for automatic transcription and automatic sign language recognition. Both NorwAI and the Language Council will investigate in more detail how we can establish a suitable data set for Norwegian sign language. It is still early days, and we have not decided whether we will try to start a research project on the topic yet, says Jon Atle Gulla.