Prime Intellect rewrites Verifiers, Agent training evaluation can be assembled like building blocks

According to Voyant Beating monitoring, the AI training platform Prime Intellect has released verifiers 0.2.0, which includes a preview of the architecture for the next generation Verifiers v1. Verifiers is an open-source framework for presenting questions to AI agents, running tasks, and scoring, which can be used for skill assessment and reinforcement learning training.

Prime Intellect has also open-sourced the model training framework prime-rl. In essence, Verifiers define tasks, tools, and scoring rules, while prime-rl trains models based on task outcomes. Developers can freely download and deploy these two sets of tools.

Prime Intellect also operates the Environments Hub and Lab. The former is used to share and download ready-made training environments, while the latter provides hosted training services. Developers can deploy the entire suite of tools themselves or directly utilize Prime Intellect's environments and computing power platform.

The previous version of Verifiers tightly coupled tasks with how the agent operates. In v1, this has been divided into three parts: Taskset specifies what to do, what tools to provide, and how to score; Harness determines how the agent accomplishes tasks; Runtime decides whether the task runs locally, in Docker, or in a remote sandbox.

As a result, the same set of tasks can now switch between agents such as Codex, Kimi Code, Terminus 2, and can be executed locally, in Docker, or in a remote sandbox. Developers no longer need to rewrite tasks and scoring rules each time they change agents or execution environments.

V1 can also track sub-agent invocations, context compression, and other branching processes, saving the Token ID and log probability required for training. The new version is more suitable for tasks lasting several hundred rounds and can directly use the agent's execution trajectory for reinforcement learning. The upcoming 1.0.0 version is planning to introduce a multi-agent environment and enhance support for environment frameworks like OpenEnv, NeMo Gym, and OpenReward.