AI: open-source models imperil profits of big tech’s contenders

In the 1960s, an MIT scientist created a natural language processing program that could mimic human conversation. It’s called ELIZA, and it’s an early iteration of the chatbot that’s been taking the tech industry by storm this year. ELIZA is not a profitable endeavor. Neither is the current version.

Generative AI has clear transformational possibilities. Chatbots developed using large language models (LLMs) enable seamless communication between humans and machines.

The question for investors is whether proprietary LLMs can reliably make money for big tech companies. For businesses developing custom applications, an open source LLM may be a less expensive option.

LLM has no formal definition. They are described as programs trained on large amounts of data available online to be able to predict the next word in a sentence.

With the increase in computing power, artificial intelligence has been able to perform unsupervised learning from unstructured data. They gave some answers that surprised even their creators.

LLM has taken a leap in complexity. In 2020, OpenAI released its Generative Pretrained Transformer 3, or GPT-3. This LLM has 175 billion parameters.

The more parameters, the more data the LLM can process and generate. Google PaLM, which powers its Bard chatbot, has $540 billion. OpenAI’s latest version of LLM is GPT-4. The company did not specify the number of parameters. Authorities believe that 100tn is an accurate figure.

The processing power required for such an LLM is enormous. The rule of thumb is that the larger the dataset used, the better the performance. In theory, this limits LLM to a small number of well-funded firms.

But niche applications can run with smaller datasets. BloombergGPT, designed to help analyze information from Bloomberg data terminals, has 50 billion parameters. Toronto-based startup Cohere AI’s underlying model, the LLM, has 52 billion parameters.

Companies like Google are more concerned with the open source LL.M. Meta gave up its system, LLaMA, as open source software that anyone can copy and use. Smaller LLMs can be built on top of it.

Assuming enterprise users see little difference between proprietary and open source LLMs when developing their own AI services, Google and OpenAI will lose their first-mover advantage before they have a chance to break even.