Large models begin to 'draft' on their own: NCP incorporates predicting subsequent text into the model's internals.

Beating AI News Flash: Shanghai AI Lab and Shanghai Jiao Tong University release an 8.9 billion parameter model, NCP-ArchPreview. Ordinary large models mainly train to "guess the next token," while NCP also predicts the internal concept signals corresponding to a short segment that follows, then uses them to assist token generation. The final output is still token by token, but the model internally already anticipates the following text in advance.

This set of signals can also be used to speed up speculative decoding. Schemes such as DFlash and DSpark have a small model guess the following text first, then hand it to the large model for unified verification. The NCP team also gives the internal concept signals to this small model, which is equivalent to first giving it a hint about "roughly how to write what comes next," and then letting it guess 16 tokens in parallel at once.

In this way, the small model gets more guesses right. In four tests, each time the large model verifies, the average number of tokens that pass rises from 5.933 to 6.180, an increase of 4.17%; on HumanEval, the increase is 7.59%. Connecting this set of signals adds only about 40,000 parameters to the draft model.

Also with 8.9 billion parameters, NCP can reduce training error to roughly the same level using about 85% of the training compute of an ordinary Transformer.