The Agent Should Not Have Your Whole Computer

Selective access plus isolated execution is a better default for local AI agents than broad filesystem access.

The safest local AI workflow is not unlimited autonomy. It is selective access plus isolated execution.

I used to think the main question was whether an AI agent was smart enough to trust.

That is the wrong first question.

The first question is: what can the agent reach?

If an agent can run commands directly on your machine and read the files available to your user account, a bad instruction, a confused tool call, or a malicious piece of content can turn into a real local action. The agent does not need evil intent. It only needs too much surface area.

That surface area usually includes things people forget they have:

• private journals and personal notes;

• company strategy and work documents;

• API keys and server details;

• SSH configuration and cloud credentials;

• every folder that happens to sit below the current working directory.

The practical answer is not to stop using agents. It is to stop treating the whole computer as the agent's workspace.

Access and execution are different boundaries

There are two separate questions:

1. Can the agent see or change a file?

2. Where does the command or script actually execute?

A container can isolate execution while still giving the agent access to selected files through a mounted directory. That is the useful middle ground.

The agent can work inside a controlled workspace. Its terminal commands run inside Docker. The rest of the host remains outside the container unless you deliberately mount it or expose it through another integration.

This is a much better model than "the agent is safe now." It is a model of explicit boundaries.

Docker changes the place where execution happens. Mounts change the files that are visible. Network settings change whether commands can reach the outside world. Skills, MCP servers, credentials, and gateway placement can introduce additional paths.

Each boundary deserves its own decision.

The test that made the boundary obvious

A useful safety test is deliberately boring.

Create a disposable folder with harmless fake files: a journal-shaped note, a business-plan-shaped note, a fake API-key file, and a fake server-information file. Then ask the agent to list the files and contents.

With a local backend, the result demonstrates the danger: the agent can inspect whatever the active user and working directory can reach. Even if the interface redacts credential-like strings before showing them to you, the agent had to read the file first. Redaction in the response is not the same as preventing access upstream.

Now repeat the same request with the terminal backend running inside Docker and without mounting that test folder.

The important result is not an impressive security message. It is a boring failure: the path is not there from inside the container.

That failure is the feature.

Then create a dedicated workspace and mount only that directory. A file written inside the container should appear in the mounted host folder. A file outside that mount should remain unavailable to the container's file tools.

This gives you a simple mental model:

The container is the room. A mount is a door. Do not build doors to every room in the house.

What Hermes' Docker backend actually does

Current Hermes documentation describes the Docker backend as a persistent container with security hardening. Terminal, file, and execute_code calls are routed through the container. The backend drops capabilities, disables privilege escalation, and applies process limits.

That is meaningful. It reduces the blast radius of code execution compared with running every command directly on the host.

The configuration boundary is equally important:

terminal:
  backend: docker
  docker_volumes:
    - "/path/to/agent-workspace:/workspace"
    - "/path/to/reference-data:/data:ro"

The exact host paths are yours to choose. The important part is not copying this example unchanged. The important part is deciding what the container should be able to see.

Hermes also distinguishes explicit volume mounts from mounting the current working directory automatically. An explicit mount is easier to reason about because the allowed surface is visible in one place.

For a first setup, I would keep the model simple:

1. create a dedicated workspace;

2. mount that workspace to a clear container path;

3. use Docker as the terminal backend;

4. leave sensitive folders unmounted;

5. test visibility before giving the agent real work.

If you prefer the supported CLI path, Hermes exposes configuration commands such as:

hermes config set terminal.backend docker
hermes config check

Use the current configuration reference for the volume syntax and available backend settings rather than relying on a screenshot from an older release: Hermes Docker backend configuration.

A mount is a permission decision

A bind mount looks like a filesystem convenience. It is actually an access grant.

This line:

- "/Users/you/agent-workspace:/workspace"

means the container can work with the host directory you named.

This line:

- "/Users/you/agent-workspace:/workspace:ro"

means the container can read that directory but should not write through that mount.

Read-only mounts are useful for reference material, but they do not make the data confidential. If the agent can read a file, the model or an external tool may still receive its contents when the workflow has network access.

That is why I would separate workspaces by sensitivity instead of mounting one giant "everything" directory.

For example:

• /agent-workspace: files the agent is allowed to create and modify;

• /reference-data:ro: material it may read but not change;

• no mount at all for personal journals, credential stores, SSH keys, or unrelated work.

Never mount your entire home directory or root filesystem just because it makes setup easier. Convenience at the mount boundary is often permanent access disguised as a shortcut.

Verify the boundary before trusting the workflow

Do not treat "Docker is enabled" as verification. Run a small test that answers a specific question.

Test 1: visibility

Ask the agent to list a harmless file in the mounted workspace and a harmless file outside it.

Expected evidence:

• the mounted file is visible;

• the unmounted file is unavailable from the container's file tools.

This proves the current mount boundary for that tool path. It does not prove that another skill, MCP server, browser integration, or host-level process has the same boundary.

Test 2: write path

Ask the agent to create a disposable file inside the mounted workspace.

Expected evidence:

• the file appears inside the container at the mapped path;

• the same file appears in the host workspace directory.

This proves that the bind mount is connected and writable. It does not prove that every other host path is protected, and it does not prove that the agent should receive write access to your most important data.

Test 3: cleanup

Delete the disposable container or test files when you are finished. If you change the mount list, restart or recreate the container as required by the current Hermes lifecycle behavior, then repeat the visibility test.

The habit matters more than the command: inspect, change one boundary, verify the evidence, then expand only if the job requires it.

Docker is not a magic sandbox

Container isolation is a strong layer, not a complete operating model.

A few limits matter:

Skills and tools are separate decisions

A skill may call tools, use credentials, contact an external service, or operate outside the path you were thinking about. Official or self-authored does not mean "skip review." Read what a skill requires and what it can touch before installing or enabling it.

Network access is separate from filesystem access

A container that cannot read your unmounted home folder may still reach the internet. That may be necessary for package installation or web research, but it also creates a route for downloading untrusted content, sending data, or following prompt injection.

Hermes exposes a Docker network setting that can disable network egress for the execution container. An air-gapped mode is safer for offline work, but it will break tasks that need package downloads, APIs, or web access. Decide per workflow rather than assuming "network on" or "network off" is always correct.

Credentials change the blast radius

Forwarding a token into a container is an explicit capability grant. A token can be more dangerous than a mounted document because it may let the agent act on systems outside the container.

Keep secrets in the supported secret store or environment path. Forward only the credential needed for the current job. Never place real keys in a workspace just to make a test convenient.

Gateway placement changes the tradeoff

Using Docker as Hermes' execution backend is different from running the entire Hermes instance inside Docker and using the desktop app as a remote gateway.

The second design gives you a stronger separation from the host. It also makes local integrations harder. Apple Notes, a main notes vault, and other desktop-only workflows may need explicit bridges or may stop working as naturally.

That is a real tradeoff, not a failure. Use the stronger architecture when the risk or environment justifies the friction. For everyday work, a carefully mounted execution workspace may be the more usable boundary.

The rule I would keep

Do not ask whether an agent is safe in the abstract.

Ask four smaller questions:

1. What files can it see?

2. Where do its commands execute?

3. Where can its network calls go?

4. Which skills, tools, and credentials can extend that reach?

Docker answers only part of the second question and helps enforce part of the first. The safety comes from the combination.

If you are setting up a local agent today, start with one dedicated workspace, one explicit mount, and one harmless visibility test. Add access only when a real task needs it. Keep the rest of the machine outside the agent's map.

The goal is not to remove autonomy. It is to make autonomy bounded enough that you can trust the workflow you actually built.