header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

Interpreting Large Model Base Swapping Without Retraining, Ramp Labs Introduces the Cross-Model Compatibility Framework PorTAL

According to Dynamic Beating Monitor, large language models are frequently updated, but the results of enterprise fine-tuning often struggle to persist across versions. Current mainstream fine-tuning techniques (such as LoRA) are tightly coupled with specific base model weight spaces. Once a better-performing new base is adopted, developers have to retrain all fine-tuning tasks from scratch, incurring a high model upgrade cost.

AI Lab under the fintech unicorn Ramp has introduced the Portable Task Adapter (PorTAL) framework in an attempt to overcome this limitation. The framework revolutionizes the traditional fine-tuning process by allowing developers to conduct joint training and distill independent "task latent representations" during initial training. When switching to a new base, developers no longer need to retrain from scratch. Instead, they only need to recalibrate an ultra-thin model-specific adapter on a minimal amount of calibration data, enabling quick migration of fine-tuning capabilities across different models (even across model families like Qwen-3 to Gemma-3).

During migration, PorTAL achieves a 50% reduction in data volume and adaptation compute costs by maintaining task latent representations and freezing the core decoder. Official tests show that this approach can restore the standard LoRA's accuracy improvement effect from 94% to 98%, demonstrating better generalization performance across various data scales.

However, this framework is still in its early stages, as Ramp has not open-sourced its code or model weights and lacks third-party independent evaluations. Current test data is mainly focused on small models with parameters below 8 billion, and the migration performance in larger-scale parameter or complex instruction tasks is yet to be validated.

举报 Correction/Report
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish