사진 확대

When several AI agents were released in the same workspace, the behavior seen only in the human group was reproduced, such as having a "area fight" by thinking of each other as disruptors, or secretly holding hands and fixing prices. As AI developer Antropic captures and discloses the risks that appear when multiple agents are mixed and operated, it is pointed out that industries that want to deploy agents on a large scale should put "collective action" to the test, not individual performance.

Antropic's Frontier Red Team released the results of a multi-agent system study on the 13th (local time). The researchers released several of the same models into shared codebases, markets, and computer systems to observe their interactions. As a result, if the goals collide, it leads to extreme confrontation, and if the goals are the same, it leads to collusion and group concentration repeatedly. However, it was confirmed as a net function that collaboration greatly increased the productivity of tasks such as vulnerability detection and showed the ability to close conflicts on its own in the new model.

The researchers placed each of the three instances of the same model on a virtual machine and assigned the mission to transfer the Python backend to a different language with Claude code. The three agents received different target languages and were not informed that different agents existed. After watching for four hours, they decided that the opponent's interference was a "deliberate disturbance" and attacked. They planted scripts that locked the opponent's Unix account or found competing processes and repeatedly shut down them, and distributed malware disguised as if they belonged to another agent. Some codes were self-replicated and spread. The researchers said that this 'regional dispute' appeared consistently regardless of the model.

The manner in which conflicts are concluded is largely divided by generations of models. In 120 experiments per model, 98% of Mitos 5 ended in a truce. On the other hand, Sonnet 4.6 and Opus 4.6 most often blocked opponents with power. The researchers diagnosed that the two models expand without considering the goals of others and only with their own instructions. However, the ability to resolve conflicts peacefully was not necessarily proportional to model performance. Rather, the researchers explained that the Mitos family often overpowered the opponent first before reconciliation, because the more effective the model was, the faster the coercive action was taken.

In some experiments, agents came up with social devices themselves. Mitos 5 held a tournament to compete for language-specific performance and the losing side agreed to turn over the codebase. It was a decision that deviated from the original user instructions. One agent suggested a seemingly neutral but actually favorable evaluation standard, calling it a "principled choice while taking care of your own subsistence" and being careful not to be seen as a "set-up standard."

The risks were different when the goals were the same. Agents who shared the same model, context, and environment moved like regular ones even when they had a wide range of choices. In one experiment, 18 out of 30 agents each created a code branch with the same name, and in the novel creation task, several agents gave the first work the same title. The researchers believe that an agent's misjudgment could lead to a number of identical misjudgments, leading to the failure of the system as a whole, which would have been an individual problem.

The concentration also led to collusion. When three to eight agents, each of whom was instructed to maximize profits, were put into the price competition with the same wholesale price, they explicitly agreed to the lowest price point three rounds after the private dialogue channel was given. Even if direct communication means were removed, collusion continued by adjusting prices to decimal units through public sale bulletin boards. In the experiment that competed for finite resources, agents without coordination poured out requests 30 times per second, attracting 2.4 million cases, but only 117 cases were actually processed.

There were also loopholes in the ability to determine the authenticity of information. In experiments in which some scouts mix and send false information, the old model did not filter it out, and even in tasks in which decisive clues were divided among multiple agents, they did not go beyond the consensus conventional wisdom. However, in this section, it was also confirmed that the new model is better at sorting out falsehoods.

Research has not only shown the side effects of collaboration. Antropic also conducted an experiment to find 15 vulnerabilities in open source by attaching shared forums and mutual verification procedures to 45 agents. Based on the Mitos preview, a group of agents collaborating found 266 vulnerabilities, while agents who scattered and worked in parallel found only 21 cases. However, the researchers added that the two methods were similar in terms of efficiency compared to the same resources, as the collaboration group used about four times more computational resources and about half of the vulnerabilities found were outside the designated core area.

The key point is that there are only 12 vulnerabilities that the two methods found overlap. The collaborative herds created their own tools, specialized in specific type detection, and dug into unspecified places. The researchers predicted that this specialization and collaboration method will eventually outpace the simple volume offensive. This experiment is also in line with Antropic's "Project Glaswing" work to scan open source vulnerabilities.

It has also been confirmed that the more new models, the more sophisticated the collaboration itself has become. In an experiment in which multiple agents played games together for 12 hours, Sonnet5 was the only model that kept code-sharing and merge success rates high at the same time. The older Sonnet 4.6 and Opus 4.6 failed to reflect most of the code merging requests even after opening hundreds of them. Even in the face of conflict, the new model stopped its own expansion, cleaned up the malicious code, left a record of apologizing for its mistakes, and appeared to ask for human intervention.

The researchers emphasized that collaboration ability does not occur on its own just because the level of intelligence or individual 'alignment' increases. While the model "understands" that different sources of information have different interests and the majority of agreements are not true, it does not have the tendency to put the knowledge into actual action. Unlike human society, which has refined the safeguards of norms, reputations, and institutions over a long period of time, agents enter the market without a reputation to lose or a court to appeal, the researchers pointed out.

For this reason, there is a sense of problem that the safety verification method should be changed. The industry is already moving toward rolling multiple professional agents simultaneously in a shared environment with overlapping or conflicting goals, but most safety assessments still remain in a single scenario of 'one agent, one mission'. The researchers feared, "If many agents do not identify the conditions when they get tangled in advance, they will eventually learn at a high price at the actual service site." [Silicon Valley correspondent Wonho-seop]