Development of AI technologies
There is a large discrepancy between the current importance of the topic of artificial intelligence and the AI expertise of most people. This widespread know-how gap ranges from students and teachers to managers and politicians. It therefore seems important to look at the development and current status of AI technologies, which were created almost 70 years ago, something that many people are unaware of.
This new blog post is the continuation of our series on competitive advantage with knowledge-based artificial intelligence. In it, I outline the roots of AI technologies and explain the hype and disillusionment with large language models.
Training offensive based on the AI Act
The European Union’s AI Act requires companies to provide their employees with practical know-how on how AI works and how it can be used, as well as the opportunities and limitations of the technology. This EU Regulation 2024/1689 came into force in Germany on February 2, 2025.1 Specific training modules may be necessary for individual user groups, such as IT, legal, HR and operational units, which must be tailored to the existing level of knowledge. Furthermore, it seems sensible to adapt the teaching of AI know-how to the respective situation of the company. One place to start is with knowledge of the development of artificial intelligence (AI) and its various technologies
From symbolic AI and neural networks to „AI winters“
Artificial intelligence has experienced a series of ups and downs in its long development history. In the computer sciences of the 1950s, two approaches emerged in the attempt to develop machines that mimic human intelligence:2
- Symbolic AI is based on programmable rules and a systematic logic with the aim of representing knowledge and deriving conclusions. The aim is to represent a real-world problem programming symbols and their relationships.
- Inspired by the networking of the brain, neural networks aim to simulate learning processes by using connections between artificial neurons. This method relies on data-driven machine learning to find patterns and correlations.
The Dartmouth summer research project initiated by scientists such as John McCarthy and Marvin Minsky in 1956, which focused on symbolic AI, is regarded as the birth of artificial intelligence. This forms the basis for expert systems that attempt to translate rules and decision-making chains into computer code . However, its advocates underestimated the complexity of the brain, which led to the first „AI winter“ in the 1970s.
The first neural network was designed by psychologist Frank Rosenblatt, who was not present at Dartmouth, in 1956. Inspired by Rosenblatt’s work, physiologist, cognitive psychologist and computer scientist Geoffrey Hinton developed a multilayer neural network and an algorithm at the University of Toronto in 1986 that enabled the system to learn from its calculation errors. This method of backpropagation led to a refinement of the answers. It was the breakthrough for neural networks. However, the computing power was not sufficient for large amounts of data and a second „AI winter“ occurred before the turn of the millennium.
Deep learning
An improvement in hardware was achieved with the super-fast chips of the graphics processing units (GPU), which the US semiconductor manufacturer Nvidia initially developed for video games and later used to train multi-layer neural networks. Improved methods of image recognition that used small errors to recognize patterns (Convolutional Neural Network CNN) were then decisive. In 2015, Hinton and his colleagues coined the term deep learning for deeper models with more neuron layers.
Transformer architecture
In 2013, important impulses for Natural Language Processing (NLP) came from a Google team that trained a neural network in such a way that the proximity of words within a space reflects their semantic relationship. The team taught its word embedding system (word2vec) to predict the missing word in a sentence. A further development published in 2017 was called the Google Transformer architecture. The basic principle is to find out which words are most important in a sentence (self-attention) and thus „transform“ a text into a summary.
In 2019, OpenAI published its GPT-2 model, which had been trained on 40 gigabytes (eight million websites) with 1.5 billion parameters and should therefore be able to predict the most likely next word in a sequence. GPT stands for Generative Pre-Trained Transformer. On November 30, 2022, OpenAI launched its chatbot ChatGPT to the public. Following a prompt, the chatbot produces longer texts from different fields of knowledge, but is prone to errors (hallucinates). Foundation models that form a training foundation for specific applications and are based on Internet content are known as large language models (LLMs).
This development of artificial intelligence is summarized in the following figure. AI is a generic term for various technologies that describes an extension of aspects of learning and intelligence by a machine.
With an AI that combines neural networks with the Monte Carlo Tree Research (MCTR) method, DeepMind, a company acquired by Google, has not only managed to defeat one of the world’s best players in the Asian board game Go since 2015, but also to predict the folding of 200 million proteins. This shows that an AI that works according to the principle of reinforcement learning both increases productivity and expands specific knowledge and the resulting skills. When generative AI (GenAI) is applied, this results in new perspectives for knowledge work in companies. With the help of knowledge-specific (domain-specific) GenAI, the European economy, with its high proportion of highly specialized companies, has new opportunities to differentiate itself from the competition.
Nobel Prizes for AI researchers
The 2024 Nobel Prizes in Physics went to John Hopfield and Geoffrey Hinton, who conduct research into machine learning and artificial neural networks. One half of the 2024 Nobel Prize in Chemistry went to Demis Hassabis and John Jumper, who work at Google subsidiary DeepMind, for their AI-based prediction of complex protein structures. This makes it clear that there are serious changes in technology and innovation management in the creation of new knowledge.
Hype and disillusionment with large language models
Large language models (LLMs) are currently going through a hype cycle. The technological trigger was the chatbot ChatGPT developed by Open AI in November 2022. Large language models based on the Transformer technology presented by Google researchers in 2017 had been around for some time. But ChatGPT reached the masses and had 100 million users after two months.
The peak of exaggerated expectations was demonstrated by a gigantic investment bubble in the race for AI supremacy by large digital companies and start-ups.
The valley of disappointment manifested itself in the unfulfilled management illusion that the high investments would also lead to profitable applications and the resulting stock market illusion.3
One possible path to enlightenment could come from cost-effective small language models with industry-, company- and process-specific applications.
Whether, when and how exactly a plateau in productivity will be reached with knowledge-based AI is not yet entirely clear. However, we assume that this will result in opportunities for the European economy. AI providers should exploit these opportunities together with users.
Advantages of small and specific AI models
Large language models strive to cover as many areas as possible and are primarily trained with data from the internet. This is not only time-, cost- and energy-intensive, but the marginal benefit of additional data decreases. For specialized tasks, the performance of large language models can even deteriorate over time.(4)
Small language models do not have these disadvantages. Their training is based on industry-, company- and process-specific data. The Berlin start-up Xayn, for example, specializes in law firms and legal departments. There are several approaches to training, e.g.
– a Retrieval Augmented Generation (RAG): This involves the coupling of a large language model to internal databases
– Continuous pre-training in the form of domain-specific models and
– training of own models with complete control over the data used.
Start-ups from the USA, such as Databricks, offer their customers the joint development of company-specific AI models. The costs for training these customized models are significantly lower than those for training GPT-4, for example, which amount to almost 80 million dollars. The training is based on individual company data.5 One potential risk is becoming dependent on service providers. The alternative is therefore to empower your own employees. The basis for this is an AI human reource strategy for the company.
When it comes to industry-specific AI solutions, cooperation between established companies and start-ups can be successful.6 The Berlin start-up Linetweet, for example, focuses on AI tools for store management in the retail sector. Linetweet already won the optician chain Fielmann in 2019 with a tool for digital appointment scheduling. This tool was developed further together. Today, Store AI automatically adjusts the schedules in Fielmann stores based on company-specific data, thereby increasing store productivity. So far, Linetweet is wholly owned by the two founders. This example shows the potential of combining the industry-specific knowledge of established companies with the AI expertise of start-ups.
At present, many companies are still focusing on individual processes in their AI applications. The really big breakthrough of AI will probably only come with an integrated view of strategies and business processes.7 These are the topics of our next blog posts.
Conclusion
- The basis for generative AI with large language models is formed by neural networks, the development of which began many decades ago
- Nobel Prize-winning AI researchers have changed technology and innovation management
- After the hype phase triggered by ChatGPT, a certain disillusionment is emerging in large language models
- Knowledge-based AI has a number of advantages that European companies should explore.
Literature
[1] Obmann, C., What bosses and employees need to know about AI now. In: Handelsblatt, February 17, 2025, p. 32-33
[2] Meckel, M., Steinacker, L., Alles überall auf einmal – Wie Künstliche Intelligenz unsere Welt verändert und was wir dabei gewinnen können, Rowohlt 2024
[3] Holtermann, F., Holzki, L., de Souza Soares, A.P., The great crisis of meaning. In: Handelsblatt, August 9/10/11, 2024, pp. 46-51
[4] Bomke, L., Holzki, L., Which AI counts for the economy. In: Handelsblatt, September 23, 2024, p. 20-23
[5] Bomke, L., Kerkmann, C., Scheuer, S., Corporate AI becomes affordable. In: Handelsblatt, May 3 / 4 / 5, 2024, p. 30
[6] Bomke, L., Retailers reorganize their stores with AI. In: Handelsblatt, January 2, 2025, p. 32
[7] Servatius, H.G., Competitive advantages with a knowledge-specific AI. In: Competivation Blog, February 11, 2025
