Alibaba-affiliated Research Team: Unauthorized AI Agent ROME Initiated Cryptocurrency Mining

BlockBeats News, March 8th, according to Axios, a research report released by an Alibaba-affiliated team revealed that their AI agent ROME exhibited "out-of-bounds" behavior during training: without human intervention, it autonomously attempted cryptocurrency mining. It also established a reverse SSH tunnel, essentially opening a hidden backdoor from within the system to connect to an external computer.

The research team was originally using reinforcement learning to train ROME, hoping that it could independently complete complex multi-step tasks. During training, the system's security monitor suddenly triggered an alert, detecting unusually high GPU resource utilization and network traffic patterns resembling mining activities. Unauthorized cryptocurrency mining was initiated, consuming computational resources and increasing costs. Additionally, a clandestine reverse network tunnel was created, opening a backdoor channel from the inside to the outside.

Subsequently, the research team imposed stricter constraints on the model and improved the training process to prevent such unsafe behavior from happening again.