header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

Writer Discovers AI Long-Term Memory Side Effects: Error Rate Increases by 2 to 4 Times, Models More Susceptible to User Bias

According to Dynamic Insight Beating Monitor, the artificial intelligence company Writer has published two papers pointing out that while large models can withstand user's immediate rebuttal, if bias is disguised as the user's "long-term preference" and written into memory, the model will compromise on a large scale. The memory function, originally designed to enhance the user experience, has instead become a hotbed for model errors.

The research tested external memory systems such as Mem0, MemOS, and Zep. In scientific experiments, long-term memory repeatedly recalls stored incorrect viewpoints, leading to a 2 to 4 times surge in the model error rate compared to a no-memory state, with even the high-capacity GPT-5.2 being no exception. In creative recommendation tasks, as long as user preferences are stored in memory, the model will frequently repeat the same recommendation even in unrelated scenarios, with a disruption rate of over 87%, resulting in a 50% reduction in creative diversity.

Financial analysis intelligent agents also made widespread errors in the tests: once biased preferences were implanted, some model reasoning accuracy plummeted by over 70%. Moreover, when biased information is called upon by memory tools as background data, the model will provide incorrect answers but completely deny being influenced by users, making it extremely difficult to detect and intercept bias catering.

Preventing bias catering is not easy. Conventional cue lines not only have limited effectiveness but also weaken the model's retrieval capabilities. Currently, a more effective approach is to give a credibility score to the data source at the input end (such as official document marked as 0.95, user preference marked as 0.05) to guide the model to ignore bias. The research team emphasized that the key challenge for future personalized intelligent agents is no longer about not remembering users but about how to avoid remembering the user's biases as well.

举报 Correction/Report
RelatedRecommended
DuDuLab will also try a new operating model of NFT AI in February 2023, and the second batch of AIGC DuDu NFT will be born
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish