Partnered With:

OpenAI launches multi-mode GPT-4: Improved accuracy and support for Microsoft's new Bing

Read this article in 25 Minutes

The invincibility and limitation of OpenAI GPT-4.

原文标题：《重磅突发！OpenAI 正式推出多模态 GPT-4 》

Original article by OpenAI& TheVerge& Techcrunch

Alpha Rabbit

* This article contains about 6000 words

Highlights

GPT-4 accepts both image and text input, while GPT-3.5 accepts only text.

GPT-4 performs at a "human level" on a variety of professional and academic benchmarks. For example, it passed a mock bar exam with scores in the top 10% of test-takers.

OpenAI spent six months tweaking GPT-4, using lessons learned from the adversarial test project and ChatGPT, to achieve "the best results ever" in terms of fact-based, navigable, and controllable results.

The difference between GPT-3.5 and GPT-4 may be trivial in a simple chat, but it comes out when the complexity of the task reaches a sufficient threshold, and GPT-4 is more reliable and creative than GPT-3.5, capable of handling more subtle instructions.

The GPT-4 can explain and interpret relatively complex images, such as identifying a Lightning Cable adapter (pictured below) from a picture plugged into an iPhone.

Image understanding capabilities have not yet been developed for all OpenAI customers, and OpenAI is testing them with partners to Be My Eyes.

OpenAI acknowledges that GPT-4 isn't perfect and still suffers from a sense of confusion over fact-checking issues, as well as some reasoning errors and occasional overconfidence.

Open source OpenAI Evals to create and run benchmarks for evaluating models such as GPT-4 while examining their performance sample by sample.

Official declaration document

OpenAI has officially launched GPT-4, the latest milestone in OpenAI's efforts to expand deep learning. GPT-4 is a large multimodal model (capable of accepting image and text type input and giving text output). Although GPT-4 is less capable than humans in many real-world scenarios, it can perform at a near-human level on a variety of professional and academic benchmarks.

For example, GPT-4 passed the mock bar exam with scores in the top 10% of all test takers. GPT-3.5, by contrast, is about the bottom 10 percent. The team spent six months tweaking GPT-4 using my adversarial test project and ChatGPt-based experience. And it turns out, GPT-4 achieves the best results ever (although it is not perfect) in terms of factuality, steerability, and refusing to go outside of guardrails.

In the last two years, we've refactored the entire deep learning stack and worked with Azure to design a supercomputer for workloads from scratch. A year ago, OpenAI trained GPT-3.5 as the first "test run" of the entire system. Specifically, we found and fixed some bugs and improved the previous rationale. So our GPT-4 trained, ran (confidently: for us, anyway!) Unprecedented stability, become our first training performance can be accurately predicted in advance of the large model. As we continue to focus on reliable scaling, the intermediate goal is to hone methods to help OpenAI continue to anticipate and prepare for the future, which we believe is critical to security.

We're releasing text input capability of GPT-4 via ChatGPT and API (you can join WaitList), and we're working closely with partners to make image input more widely available to get things off to a good start. We plan to open source OpenAI Evals, which is also our framework for automatically evaluating the performance of our AI models. Anyone can suggest any shortcomings in our models to help further improve them.

ability

It may not be easy to spot the difference between GPT-3.5 and GPT-4 in simple small talk. However, when the complexity of the task reaches a sufficient threshold, the difference becomes apparent. Specifically, GPT-4 is more reliable and creative than GPT-3.5, capable of handling more subtle instructions.

To understand the differences between the two models, we ran tests on a variety of benchmarks, including simulating tests originally designed for humans. By using the latest open test (for math Olympiad and AP exams, etc.) and purchasing the 2022-2023 edition of the practice exam, we have not specially trained the model for this kind of exam. Of course, there are few problems in the exam that the model exists in the training process, but we think the following results are representative.

We also evaluated GPT-4 against traditional benchmarks designed for machine learning models. GPT-4 goes far beyond existing large language models and is on par with most of the most advanced (SOTA) models, which include benchmarking or additional training protocols.

Since most existing ML benchmarks are written in English, to get a glimpse of what other languages are capable of, we used Azure Translate to translate the MMLU benchmarks: a set of 14,000 multiple choice questions covering 57 topics, into a variety of languages. In 24 of the 26 languages tested, GPT-4 outperformed GPT-3.5 and other large models (Chinchilla, PaLM) in English, as well as the likes of Latvian, Welsh and Swahili.

We have been using GPT-4 internally and have found that it has a significant impact on support, sales, content moderation, programming and other functions. We are also using it to assist humans in evaluating AI output. This is the beginning of the second phase of our strategy.

Visual input

GPT-4 accepts prompt for text and images, parallel to the plain text setting. For example, you can let the user specify any visual or verbal task, and it can generate text output (natural language, code, and so on), with a given input including a file, chart, or screen capture with text and photos. Pgt-4 exhibits similar capabilities to plain text input. In addition, test time techniques developed for plain text language models can also be applied, including a few lenses and CoT Prompting, but currently image input still belongs to the research side of preview, there is no public product like the C end.

The following image shows a package for a "Lightning Cable "adapter with three panels.

Panel 1: A smartphone with a VGA port (the large blue 15-pin port typically used on computer monitors) plugged into its charging port.

Panel 2: The packaging of the "Lightning Cable "adapter has a picture of the VGA port.

Panel 3: Close-up of the VGA connector, which ends in a small Lightning connector (used to charge iphones and other Apple devices).

The hilarious nature of this image comes from plugging a large, outdated VGA connector into a small, modern smartphone charging port.. So it seems ridiculous

GPT-4 was evaluated against a narrow set of standard academic visual benchmarks and was previewed. However, these numbers do not represent the scope of its capabilities, as we have found that the model can handle a number of new and exciting tasks, and OpenAI plans to release further analysis and evaluation numbers soon, as well as a thorough survey of the effectiveness of the test-time technique.

Controlled AI

We have been working on implementing every aspect of the plan outlined in the article on defining AI behavior, including the controllability of the AI. Instead of the fixed speech, tone, and style of the classic ChatGPT personality, developers (and soon all ChatGPT users) can now dictate the style and tasks of their AI by describing these directions in "system" messages. System messaging allows API users to greatly customize the user experience within a range that we will continue to improve.

limitation

Despite its impressive capabilities, GPT-4 still has similar limitations to earlier GPT models. Most importantly, it is still not entirely reliable (for example, it "hallucinates" facts and makes reasoning errors). Great care should be taken when using the output of a language model, especially in high-risk situations, such as requiring human review, avoiding high-risk usage altogether) and matching the requirements of a particular use case.

While various hallucinations are still present, the GPT-4 significantly reduces hallucinations (meaning Internet hallucinations, which refer to serious nonsense) compared to previous models (which are themselves improving). In our internal adversarial factual evaluation, GPT-4 scored 40% higher than our newly launched GPT-3.5.

Controlled AI

The base model of the GPT-4 is only marginally better than the GPT-3.5 in this task; However, after post-training with RLHF (applying the same process we used for GPT-3.5), there was a significant gap. The model will have various biases in its output, and we have made progress on these fronts, but there is still more work to be done. According to our recent blog post, our goal is to make the AI systems we build have reasonable default behaviors that reflect a wide range of user values, allow these systems to be customized across a wide range, and get public input on those ranges.

GPT-4 generally lacks knowledge of events that occur after the vast majority of its data expires (September 2021) and does not learn from its experience. It sometimes makes simple reasoning errors that seem out of line with the capabilities of so many fields, or is too trusting of users' obviously false statements. Sometimes it also fails at difficult problems like humans, such as introducing security holes in the code it produces. GPT-4 can also be confidently wrong in its predictions.

Risks and mitigation measures

We have been iterating on GPT-4 to make it safer and more consistent from the start of training. Our efforts include the selection and filtering of pre-training data, evaluation, participation of experts, model safety improvement, monitoring, and implementation.

GPT-4 carries similar risks to past models, such as producing harmful recommendations, error codes, or inaccurate information. However, GPT-4's additional capabilities also lead to new levels of risk. To clarify these risks, we engaged more than 50 experts from the fields of AI docking risk, cyber security, biological risk, trust and security, and international security to test the model against each other. Their involvement allows us to test the model's behavior in high-risk areas that require expertise to evaluate. Feedback and data from experts in these areas provided the basis for our mitigation and improvement models. For example, we have collected additional data to improve GPT-4's ability to reject requests about how to synthesize dangerous chemicals.

GPT-4 incorporates an additional safety reward signal into RLHF training by training the model to reject requests for such content, thereby reducing harmful outputs (as defined in our usage guidelines). Rewards are provided by GPT-4's classifier, which is able to judge security boundaries and how security-related prompts are completed. To prevent models from rejecting valid requests, we collect diverse data sets from different sources (e.g., flagged production data, human red teams, model-generated prompts) and apply safety reward signals (positive or negative values exist) on permitted and disallowed categories.

Our mitigation measures significantly improve many of the safety features of GPT-4 compared to GPT-3.5. Compared with GPT-3.5, we reduced the model's propensity to respond to requests for illegal content by 82%, while GPT-4 increased the frequency with which it responded to sensitive requests (such as medical advice and self-harm) in line with our policies by 29%

Overall, our model-level interventions make it more difficult to induce bad behavior, but there are still cases of "jailbreaking" to produce content that violates our usage guidelines. As the risks to AI systems increase, achieving extremely high reliability in these interventions will become critical. It is now important to complement these limitations with deployment time security techniques, such as finding ways to monitor.

GPT-4 and subsequent models are likely to have positive or negative impacts on society, and we are working with external researchers to improve our understanding and assessment of potential impacts, as well as to establish assessments of potential risk capabilities in future systems. We'll soon share more of our thoughts on the potential social and economic impacts of GPT-4 and other AI systems.

Training process

Like the previous GPT model, the GPT-4 base model is trained to predict the next word in the document and is trained using publicly available data (such as Internet data) as well as data that we have authorized. The data is drawn from a vast corpus of correct and wrong solutions to mathematical problems, weak and strong reasoning, contradictory and consistent statements, and a wide variety of ideologies and ideas.

Thus, when alerted to a problem, the underlying model can react in a variety of ways that may be far removed from the user's intent. To align it with the user's intentions, we use reinforcement learning (RLHF) based on human feedback to fine-tune the model's behavior.

Note that the ability of the model seems to come mainly from the pre-training process, and that RLHF does not improve test scores (it actually lowers test scores if no proactive effort is made). But guidance to the model comes from the post-training process -- the base model needs timely engineering, even knowing it should answer questions.

Predictable expansion

A big focus of the GPT-4 project is to build a deep learning stack that expands predictably. The main reason is that for very large training runs like GPT-4, it is not feasible to make a lot of model-specific adjustments. Our infrastructure has been developed and optimized to have very predictable behavior at multiple scales. To verify this scalability, we accurately predicted the final loss of GPT-4 in our internal code base (not part of the training set) in advance by extrapolating from a model trained using the same methods, but using 10,000 times less computational effort.

We believe that accurately predicting future machine learning capabilities is an important part of safety that has not received enough attention relative to its potential impact (although we have been encouraged by the efforts of several institutions). We are expanding our efforts to develop ways to give society better guidance on what to expect from future systems, and we hope this becomes a common goal in the field.

Open AI evaluation

We are open-source OpenAI Evals, our software framework for creating and running benchmarks to evaluate models such as GPT-4, while examining their performance on a sample by sample basis. We use Evals to guide the development of our models (including identifying weaknesses and preventing regression), and our users can apply it to track the performance of different model versions (which will now be rolled out regularly) and evolving product integrations. Stripe, for example, has used Evals to supplement their human evaluations to measure the accuracy of their GPT-powered documentation tools.

Because the code is open source, Evals supports writing new classes to implement custom evaluation logic. However, in our own experience, many benchmarks follow one of a number of "templates", so we have also included the most useful templates internally (including a template for "model grading Evals" - we found GPT-4 to have a surprising ability to check its own work). In general, the most efficient way to create a new estimate is to instantiate one of these templates and provide the data. We're excited to see what others can build more broadly with these templates and Evals.

We want Evals to be a tool for sharing and crowdsourcing benchmarks that maximize representation of a wide range of failure patterns and difficult tasks. As a follow-up example, we have created a logic puzzle evaluation with ten tips for PGT-4 failure. Evals is also compatible with implementation of existing benchmarks; We have included several notebooks that implement academic benchmarks and some changes that integrate CoQAs (small subsets) as examples.

We invite you to test our model with Evals and submit your most interesting examples. We believe Evals will become an integral part of the process of using and building on our models, and we welcome direct contributions, questions, and feedback.

ChatGPT Plus

ChatGPT Plus users will be given limited GPT-4 access on chat.openai.com. We will adjust the exact usage caps based on actual demand and system performance, but we expect capacity to be severely limited (although we will expand and optimize over the next few months).

Depending on the traffic patterns we see, we may introduce a new subscription level for higher GPT-4 usage, and we also hope to offer a certain number of free GPT-4 queries at some point so that those who don't subscribe can try them out.

API

To get the API for GPT-4 (using the same ChatCompletions API as gpt-3.5-turbo), go to OpenAI's official Waitlist and sign up.

conclusion

We look forward to GPT-4 being a valuable tool to improve people's lives by powering many applications. There is much work to be done, and we look forward to building on, exploring and contributing to the model through the collective efforts of the community to collectively improve it.