Thoughts on OpenAI models released in September 2024

Cornelis Cort, The Last Judgment Triptych, engraving on laid paper, plate: 33.5 × 49.2 cm (13 3/16 × 19 3/8 in.)  sheet: 37.5 × 52.1 cm (14 3/4 × 20 1/2 in.), 1570 1560, https://www.nga.gov/collection/art-object-page.52407.html. Courtesy National Gallery of Art, Washington

OpenAI have just released two new models, 'o1-preview' and 'o1-mini'—both of which are in preview, in spite of the name of the later model—and I think there are a few important points to be made.

OpenAI claim that the new models use 'reasoning', which is anthropomorphising—to attribute human intentions or emotions to non-human entities—and just marketing rubbish. From an Ars Technica article¹:

Just after the OpenAI o1 announcement, Hugging Face CEO Clement Delangue wrote, "Once again, an AI system is not 'thinking,' it's 'processing,' 'running predictions,'... just like Google or computers do. Giving the false impression that technology systems are human is just cheap snake oil and marketing to fool you into thinking it's more clever than it is."

It's easy to believe there's deus ex machina in what OpenAI and other companies do. However, what is it that they do?

As Watumull, Chomsky, and Roberts have concluded²:

However useful these programs may be in some narrow domains (they can be helpful in computer programming, for example, or in suggesting rhymes for light verse), we know from the science of linguistics and the philosophy of knowledge that they differ profoundly from how humans reason and use language. These differences place significant limitations on what these programs can do, encoding them with ineradicable defects.

Chomsky later elaborated on the collaborative article³, where he not only explained differences between science and engineering but mainly spoke about the unquestionable limitations of all AI systems. AI is, at its best, a probability device that delivers possibly-erroneous answers, while often sounding extremely confident⁴.

A chart by OpenAI⁶.

One thing about this chart: it's made by OpenAI, who have previously published exaggerated and erroneous benchmark results⁷; this is in line with their brand: fiction above fact.

As the chart image in this post shows, the OpenAI model 'o1-preview' is better than some other OpenAI models at some things, and not very good at other things, like 'personal writing', whatever that may be, according to OpenAI.

Morgan Stanley recently concluded that datacenters across the globe will produce three times more greenhouse gases than if generative AI hadn't existed⁵.

LLM benchmark detail by Kagi⁸.

The two new OpenAI models require far more power than most other powerful and commercially available models from other companies. Just take a look at the table in one of the attached images in this post, which is from Kagi's LLM Benchmarks page: the new OpenAI models require around five times as many tokens to produce calculations and the monetary costs are even higher; you'll pay nearly 17 (!) times more to use 'o1-preview' compared with 'gpt-4o' (both from the same company), partly to see more finely-tuned responses in some fields of search results and partly to see the responses wrapped up in so-called reasoning. All of this at a—very generalised, according to the Kagi page—accuracy rate of 52%.

Interestingly enough, if you'd ask me questions and find out I lied to you half of the time, and I'd say wait, I also delivered you 'reasoning' behind my answers, you'd stop listening to me. I keep this thought at the back of my mind when speaking to people who believe AI is a kind of oracle and evolution of search engines (which is just not true; one should not use AI chatbots as search engines). Deus ex machina, indeed.

AI is surely here to stay, even though recklessness will be held to account, for example, as in the USA, where government has forced some major companies to divulge their models⁹. And, as some of the articles above clearly state, AI is very good for use in some very explicit and testable ways, as with programming.

However, as Chomsky proves, language acquisition is something that comes natural to any four-year-old human being, and will never be understood by AI.

At the end of the day, if 'reason' means to further hurl humanity toward climate catastrophe, I won't swallow OpenAI terminology, which is just reckless marketing designed to sell a product that is, at its best, wrong about half of the time.