Large language models have huge potential for value creation - but there is a strong need to address issues of control and risk mitigation.  

We are now moving towards a huge change in intellectual value creation, powered by the weird and surprisingly sophisticated mimicry of intelligence powered by large language models (LLMs). 

These models have unleashed a wave of creativity. They, and their model cousins that can process, transform and generate sound, images and any digitizable data,  have enabled previously impossible products and services along with a torrent of hype. 

Due to the enormous amounts of data, compute and brain power required, these important platforms are now mostly developed and controlled by a few very large private technology companies in the US. This is problematic, because along with all the interesting new functionality, large language models also suffer from serious and complicated challenges such as bias, hallucinations and toxicity. Private companies will invariably balance mitigating these issues with the need for profit. They are likely to do the bare minimum required to avoid regulatory retribution and public relations backlash.  

Closed to scrutiny

Most private company models are black boxes, closed to scrutiny from independent researchers - so research on explainability, bias and other issues happen from the outside. Ideally the models would be open and transparent, enabling society to more thoroughly evaluate them for ethics compliance, bias and explainability. Openness also makes fine-tuning existing models for specific purposes easier. Finally, openness will help advancing researchers  working on developing smaller, faster, cheaper and more performant models. 

Single points of failure

Concentrated ownership also means that we are faced with single points of failure, both politically and commercially. As we move towards an economy where people, services and business processes increasingly depend on language models, this is a serious concern.

The Open Source software movement has shown that it is possible to create large, enduring, open alternative platforms (e.g. Linux).

This in turn ensures resilience and choice, even if open alternatives at any given time may be less advanced than the commercial ones. 

There is also the issue of cultural and national sovereignty. Most current language models are based on available data, mostly English, with other languages treated as an afterthought. Some models perform adequately on less resource-rich languages , but their performance in them is inferior to English and other major languages. GPT-4 has been tested for 26 out of the approximately 7000 languages in use in the world now.  We don’t know yet how much better language models specifically trained to accommodate smaller cultures and languages could perform. This should be investigated.  

The case for European models

An additional argument for openly available language models in Europe is privacy compliance. Chatbots are already being used as personal assistants, receiving detailed, information-rich prompts from users.  Large US-based commercial providers may wish to use these prompts and the data they contain for model re-training and finetuning. This may present European companies with unacceptable levels of privacy and GDPR risk. Open models, running in controlled environments, mitigate this risk. 

Some people will argue that the computing resources and data demands for large language models is so high that it is virtually impossible to compete with the likes of Google, Microsoft, Apple and Facebook. While cost is indeed a significant factor, the collective behind the Stable Diffusion models and Stanford’s Alpaca LLM have shown that it is possible to create and make accessible high quality models outside the research labs of the tech giants. 

Increased use

In Schibsted we are already heavy users of language models, and expect to increase the use in the future. Therefore we are following advancements in this space with great interest. We are seeing ongoing initiatives in Norway (NorwAI, UiO, the Norwegian National Library), AI Sweden, and the international HPLT project, aiming at creating Norwegian, Nordic and European alternative language models. Cooperation between industry and academia and across countries can enable meaningful, high performance language models. 

As of the moment of writing this article, we don’t know what the future holds for these projects - but we are actively contributing where we can.

We are hoping for a future where a wide variety of responsible, open and publicly available useful AI models exist, as an alternative to closed commercial ones.