OpenAI has become ClosedAI
Once known for being transparent and open source, being committed to provide access to the newest advancements in AI, OpenAI has made a 180° turn from where they come from. The release of GPT-4 showed that the influence of Microsoft has now kicked in. Instead of educating the community about what innovations they come up wth to improve the performance even more they decided to make some Apple like marketing release notes about how great the model performs on hand picked datasets.
The AI community wants to see technical details not marketing slides
As a machine learning engineer myself that is working on natural language processing (nlp) related topics the most interesting part of a new (open source) model release are the innovations being integrated. Are there new layer types? A new loss function? Have they added some extra finetuning steps that were not done before, e.g. Reinforcement Learning with Human Feedback (RLHF) to get a human touch into the outputs like they did with GPT-3 (and GPT-4?). Why are these new techniques work better than before?
In general, the purpose of a release paper of a new model is to educate the community with scientific proven findings about new ideas. So what have we received from OpenAI with the “Technical Report” of GPT-4? A marketing show at best! Usually new models that are not specialised for certain tasks (LLMs in general) are evaluated on many different official datasets against different SOTA models in different configurations. So what do we got from OpenAI? Handpicked results where GPT-4 performed best. They look impressive to be honest!
But, for example, it is common practise to use these models with a Zero-shot approach on new data, because smaller companies or freelancers do not have the ressources to finetune the model to their needs or pay the extra fee to OpenAI to do so. Similar to how everyone is using Chat-GPT right now. Everyone is using the same model for various tasks on different kinds of data, e.g. finance, medicine, recipes, math, … So what can we learn from a 25-shot results on A12 reasoning challenge? Not much, because nobody would do that. So where is the comparison on the Zero Shot results against SOTA? Is SOTA performing better? We will never know.
Okay, I get it, the general public expects the new model to perform much better at everything, so only keep the tests where it performed the best … But what about technical details for the geeks ? Well, there are no information whatsoever how the model was trained, what the architecture is about, changes to the data, what data was used in specific? If you find information about that at LinkedIn, Twitter, in the news, etc. … you can be sure they are fake, because OpenAI is now ClosedAI.
GPT 4 is now pay-only
Okay, so what about another pillar of OpenAI — providing access to the newest advancements in AI research to the general public / developpers — because other companies like Google, Meta, … used to keep it secret. Well, you have to pay now to use the model. No free credits. And yeah I get it, it is very expensive to host these models, but they got a lot of money from Microsoft to pay the bills. In addition, other institutions like HuggingFace also manage to provide open access to models of the same size for testing it out in their hosted space (at least BLOOM vs. GPT 3.5, for GPT-4 we do not know). Oh and what are the competitors doing? Meta just opened access to their newest model “LLama” to researchers so that they can benefit from their findings.
OpenAI missed the opportunity to educate the public
What is most annoying to me since OpenAI has released Chat-GPT is the lack of educational content provided by them to showcase the limitations and capabilities of the model to the non technical public. They could have used their public attention to explain in high level terms how the model works and what consequences that has to something like factual correctness etc. From my point of view it would have been their responsibilty to take care that people can use their amazing tool in a responsible manner. But instead, they choose to abuse the inexperience of the general public to ride the hype wave with their newest model to earn a lot of money. To put the cherry on top, they intentionally refused to give any information about the new model whatsoever. That is totally fair to do, but than please change the name to ClosedAI, because the real OpenAi is now HuggingFace …
