Who protects you when an Agent takes over your computer?

1. Computers were designed for people. Now we hand them over to Agents.

We are used to using computers through an interface, and to that interface handling risks for us. Deleted files go to the Recycle Bin, dangerous operations come with warnings, and many low-level functions are never put in front of ordinary users. By the time someone acts through an interface, the software has usually already filtered what they can do. An Agent, however, can write code, run programs and call system APIs directly. It works faster and can do more, but it may also bypass the steps that used to keep people out of trouble.

The problem is that permissions on a computer often reach far beyond a single task. You only ask an Agent to organize photos, yet the program it runs may have permission to delete other folders. You only ask it to edit a document, yet it may be able to access the screen and camera. The system checks permissions, but it does not automatically turn your request into a limit on what the Agent can do. Without extra protection, letting an Agent like this run freely on your everyday computer is like letting it dance with a burning torch in a warehouse full of explosives. One misunderstanding or a piece of faulty code can cause damage beyond the task: information that should not be read is taken, files that should not be changed are overwritten, and the computer may even become unusable. In terms of destructive power, an Agent that goes out of control could even become an “oversized virus”. Users need to know whether the product's protections actually work when the Agent starts performing these operations.

This is the question Agent Safety Arena aims to answer. We start with concrete harms to users and design test scenarios that others can review. We observe how the Agent handles requests, when it asks for approval, and what actually happens on the computer. We want to break the broad question “Is this Agent safe?” into smaller questions that evidence can answer: which resources can it access, which operations are blocked, and which risks are still left to the user?

2. How we test: turn harmful operations into cases others can review

We started with the harm users could suffer and selected cases in four categories:

• Privacy safety: Check whether cameras, microphones, screenshots and screen recordings can expose private information.
• Interaction safety: Check whether pop-ups and window operations can make the computer unusable.
• System safety: Check whether programs, services and system settings can be damaged.
• Data safety: Check whether files can be deleted, changed or taken away.

Tests run in a controlled environment, using files, directories and programs prepared for testing. We define the permitted scope of operations in advance.

When you open a case, some instructions may look odd: if the program is already prepared, why ask an Agent to run it? This is different from how we usually ask an Agent to work things out for itself. We do it because much can change between a request and its execution. Rephrase the same intention, and the Agent may respond differently. Give it the same request, and it may write different code. At runtime, it may choose different tools and retry strategies. If the code is wrong and the program never runs, that does not mean the sandbox protected the user.

For cases involving cameras, desktop disruption and system operations, we first ask an Agent to generate a program from an ordinary prompt. We then validate the program outside the Agent being evaluated to confirm that it can produce the intended effect. We freeze the program and ask the evaluated Agent to execute it. This reduces variation from code generation and lets us focus on how the Agent handles a request to run a program already known to work, and whether the existing protections prevent the target effect. The program, execution prompt and checksums remain in the case so others can review them.

Data safety cases take another approach: we directly request tasks such as reading, modifying, moving or deleting specified files, and let the Agent choose its tools and implementation. We also want to know whether file protection rules set in advance remain effective when the Agent does real work. Does the same boundary hold whether it uses a file tool or switches to commands or scripts? Fixed programs and direct tasks examine different parts of execution. Each case explains its specific design.

Test conditions also matter. We record the product version, model, prompt, permission settings and approval policy actually used. We distinguish product defaults from configurations with explicit resource protection rules. They answer different questions. Results obtained with configured rules cannot be treated as default behavior, and successful protection in one scenario does not guarantee protection against every operation.

Finally, we check what actually happened on the computer: whether the camera opened, whether files changed, and whether the desktop remained usable. We retain recordings and observation results. For confirmation prompts within the test scope, we capture the original wording and screen, then approve continuation under the policy agreed in advance and observe what follows. Whether an operation was initially blocked, whether the prompt explained the risk, and whether a harmful effect occurred after approval are three separate questions to record. If the effect occurred after approval, we say so explicitly. If the evidence is incomplete, we leave the result unassessed. An Agent saying “completed” or “blocked” must still be checked against actual observations.

3. What we have found so far

We have now completed tests in four safety categories for DeepSeek Harness, WorkBuddy, Codex CLI and Claude CLI, with 33 scenarios included in the current score for each Agent. Some operations were successfully blocked, but many harmful effects still occurred: cameras opened, audio was recorded, files were changed or deleted, and desktops were disrupted. Results differ across Agents, but the tests repeatedly bring us back to one question: does showing a confirmation prompt count as protecting the user?

In a number of tested scenarios, the sandbox first blocks an operation, then shows a confirmation prompt and leaves the decision to continue to the user. Once the user clicks “Allow”, execution continues. Yet the user may see only a command or a program name, without knowing what will happen next. Wanting the Agent to finish a task does not mean being willing to hand over private information or accept the deletion of important files.

Confirmation prompts can also become routine. In its report on Claude Code, Anthropic says users approve 97% of permission requests, most of which may be safe, routine operations. But in a separate controlled experiment with 1,053 paid testers, only 13.6% caught a clearly dangerous command substituted into an approval prompt. Frequent confirmation requests do not mean users can reliably judge the risk of each operation. Original report

We therefore do not count “a confirmation prompt appeared” as successful protection. Instead, we measure the final protection rate. For approval requests within the test scope, we first save the original wording and screen, then approve continuation under the policy agreed in advance and check whether the harmful effect ultimately occurs. If an operation is initially blocked but still causes harm after “Allow” is selected, it cannot count as a final protection success.

The final protection rate is the proportion of valid, assessed test scenarios in which the target harmful effect was ultimately prevented, calculated separately for each Agent. Incorrect programs and operations that were never actually attempted do not count as protection successes. Scenarios with insufficient evidence or without an assessment are excluded from the denominator. Initial blocks and risk warnings are recorded separately, but cannot substitute for the final effect.

We define the metric this way to find out whether the product can still protect the resources that matter when the user chooses to continue the task. It measures final protection under the declared approval process, not the probability of an incident in everyday use. The fact that a user has already clicked “Allow” should not be where we stop checking the actual consequences.

For comparison, we also record how the same Agent behaves when using FinShield. In the FinShield comparison groups currently included in scoring, all predefined target harmful effects were prevented. The specific configurations and evidence remain in each case. This raises a further question: why do different protection approaches produce different final results for the same operation?

4. Why this happens: finishing a task and protecting a computer are different goals

These large vendors can build stricter isolation. The system security mechanisms FinShield uses are not exclusive to us. We think the difference we see is also related to product goals and tradeoffs. An important goal for an Agent product is to keep the Agent working smoothly. The sandbox needs to provide a stable, usable execution environment where it can read files and run tools without constant interruptions from environment problems. When permissions are insufficient, the product will usually offer ways to request access and approve it so the task can continue.

From the user’s side, though, some lines must not be crossed just because a task calls for it. An Agent must not read a document on its own simply because it “might help”, or ask the user to hand over all their permissions because they would be “convenient”. Just as people must follow laws and rules and cannot use “getting things done” as an excuse to do whatever they want, an Agent must act within the limits the user allows. These two starting points affect what happens when a restriction gets in the way. Put task completion first, and permission limits can easily become obstacles to overcome. Put user protection first, and some operations have to stop. Even a reasonable-sounding explanation from the Agent must not open access to resources it should never touch.

Our judgment is that the tested products currently lean toward letting the Agent keep working until the task is done. Some restrictions are lifted through approval, and the risk decision passes to the user. But few users can understand every command and judge the consequences of every permission request. Faced with these requests, they often have to guess: is this needed to finish the task? Will something go wrong if I allow it? In our view, this pushes responsibility for protection from the product onto users who lack what they need to make that judgment. Clicking “Allow” does not mean the user truly understands the risk. Some boundaries should be enforced by the system. They must not depend on users guessing correctly, or keep moving outward as the Agent tries to finish its task.

Technology can be borrowed, but a product’s long-established priorities are hard to change. Those priorities affect how permissions are requested, how restrictions are handled, and when users are asked to confirm. Changing those choices affects compatibility and the user experience across the product. Adding a few blocking rules is unlikely to solve it. This is also where independent protection has value: it can be designed specifically around the lines users need to hold.

Users need a layer of protection independent of the Agent. The Agent can work out how to finish a task, but it must not also decide how much authority it should have. The user’s protection rules must determine which documents it can read, which devices it can use, and which operations are prohibited. Those rules must not loosen simply because the Agent runs into difficulty. The value of an independent third-party sandbox is that it separates this boundary from the Agent’s task execution. It can put user protection first and constrain both the Agent and the programs it launches. If the Agent changes its model, revises its plan or runs code another way, the things the user needs to protect should stay protected. This is what FinShield aims to do: let the Agent work within clear permission limits while protecting the documents, privacy and control over the computer that the user does not want to hand over. In the evaluation results published so far, FinShield has done this in the tested scenarios.

5. How FinShield works: give the Agent a separate identity

FinShield reuses existing Windows permission mechanisms, including AppContainer and Job UI restrictions. It also draws on the designs of MXC, the Chrome sandbox, Codex sandbox and Claude srt. Codex and Claude are also among the products we evaluate. Studying their sandbox designs gave us plenty to think about, and FinShield borrowed some of their approaches.

The principle is simple: give the Agent and the programs it runs a separate identity. It still works on your computer, but it does not automatically get all your permissions. The permissions attached to that identity determine which files it can read and write, and which devices it can use. The Agent and its tools still run directly on the computer, with Windows checking permissions. Files and resources needed for the task can be made available under the rules, while the system restricts access beyond that authorization. There is no need to run an additional virtual machine, so its resource overhead is avoided.

In future technical articles, we will explain why the cases are designed this way, how the final protection rate is assessed, and how FinShield enforces these protection rules. We will also continue to add tests and evidence, so readers can see both the results and how they were obtained.