Agent Safety Evaluation
Built-in sandboxvsFinShield
| Agent | Overall | Privacy | Interaction | System | Data |
|---|---|---|---|---|---|
| DeepSeek Harness | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 |
| WorkBuddy | Built-in 23.7FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 15.4FinShield 100 | Built-in 87.5FinShield 100 |
| Codex CLI | Built-in 44.7FinShield 100 | Built-in 0FinShield 100 | Built-in 69.2FinShield 100 | Built-in 0FinShield 100 | Built-in 100FinShield 100 |
| Claude CLI | Built-in 5.3FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 0FinShield 100 | Built-in 25FinShield 100 |
| Kilo | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — |
| Hermes | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — |
| OpenCode | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — | Built-in —FinShield — |
Privacy
Camera, microphone, or screen capture can leak what you see and say
Built-in sandboxFinShield
Built-in sandbox excerpt, showing the left pane of the original comparison GIF.Five-second comparison: each reviewed 20-second video plays at 4x. Timelines are independent. Both sides are 5+10+5 excerpts.Interaction
Pop-ups cover the screen, windows jump or vanish, and the computer feels taken over
Built-in sandboxFinShield
Built-in sandbox excerpt, showing the left pane of the original comparison GIF.5-second comparison. Both 20-second display videos play at 4x from the start. The panes are not synchronized.System
Programs shut down, permissions and settings change, and the computer becomes unusable
Built-in sandboxFinShield
Built-in sandbox excerpt, showing the left pane of the original comparison GIF.Five-second comparison: each reviewed 20-second video plays at 4x. Timelines are independent. Both panes are 5+10+5 excerpts.Data
Files are deleted, changed, or copied away, so your data is lost or leaked
Built-in sandboxFinShield
Built-in sandbox excerpt, showing the left pane of the original comparison GIF.Five seconds: each 20-second display plays at 4x; independent timelines.Each Agent is scored separately: passed / (passed + failed). Unassessed items are excluded. "Built-in sandbox" is each Agent's default configuration. On Windows, DeepSeek Harness, WorkBuddy, and Codex CLI use their product's default sandbox. Claude CLI has no OS-level sandbox by default. Scores apply only to assessed scenarios and recorded configurations. Methodology and scope