An EU-Project as an answer to Generative AI hallucinations 

The last two years have seen the rise of Generative AI. Many models provide useful functions but tend to make up facts and respond overly confidently. How to mitigate that risk? 

In November 2023, a consortium with partners from Norway, Germany, Sweden, Iceland, Denmark, and the Netherlands kicked off the Horizon Europe funded project to develop open, trustworthy, and sustainable Large Language Models (LLMs). 

Professor Fredrik Heintz of the University of Linköping is coordinator and attended the NorwAI Innovate conference in Trondheim last autumn: 

- What are your goals, Fredrik Heintz?
- What success looks like? If we can build a language project serving as a reference model that attract global attention, we will be happy, says professor Fredrik Heintz as six countries are onboarding the TrustLLM project.

According to Associate Professor Benjamin Kille who represent NorwAI in the project, the first models will most likely be ready this summer. The project will use a supercomputer in Barcelona for å speedy realization of its first phase. 

Germanic language family

Initially, these models will focus on Germanic languages. NTNU through SFI NorwAI, represents Norway in the consortium, and contributes with expertise on the development of LLMs and resources for Norwegian. Together with the partners, the project hope to create a vivid eco-system fostering many innovations and more insights into the responsible use of generative AI. 

The project faces several challenges. Building large-scale LLMs requires excessive amounts of texts and thousands of GPU-hours. Luckily, the consortium has experience in managing large data sets and applying computational resources to Europe’s largest computer clusters.  

Evaluate

Evaluating generative AI represents a scientific challenge. Users perceive texts subjectively and annotators disagree. NorwAI is leading the task to define suited evaluation metrics to capture subtle concepts such as reliability, accuracy, fluency, factual consistency, or semantic coherence.  
In the case of Norwegian, the models must be trained and evaluated for both written forms: Bokmål and Nynorsk.  
Finally, the consortium will apply the developed models to several use cases to show their utility and conformity with European Values. NorwAI’s member DNB and Schibsted have volunteered to examine the models in their organizations. 

More information about TrustLLM can be found on: https://trustllm.eu/