Phi Agent: Bringing Skills into the Browser

Phi Browser is built around the idea that a browser should not just show pages. It should understand the work happening across those pages, help you repeat common workflows, and eventually act on your behalf when you ask it to.

That is easy to say. It is much harder to design safely.

A normal AI assistant usually works with text. A browser agent works inside a live environment: active tabs, logged-in sessions, private page content, forms, cookies, scripts, and websites that may contain messy or hostile instructions. Once the agent can do more than talk, the design problem changes completely.

The hardest part of building a browser agent is not making it smarter. It is making sure that, after it becomes smarter, it still knows what it must not do.

This is the problem Phi Agent Skills is designed to solve.

Skills give Phi Agent a way to learn reusable workflows without stuffing every instruction, example, and edge case into the prompt. They also give us a structure for deciding what the agent is allowed to do, when it is allowed to do it, and where those permissions should stop. Apparently “just let the model figure it out” is not a security model. Shocking development.

Building on the standard Skills model

Anthropic’s Skills model introduced a clean and useful abstraction: a directory, a SKILL.md file, and optional scripts, references, and resources. The model sees the Skill’s name and description first, then loads the full instructions only when the Skill becomes relevant.

That is the right shape. It solves a practical problem in agent systems: the context window cannot grow forever. You cannot place every workflow, every instruction, and every supporting file in front of the model all the time and expect stable behaviour. A better approach is progressive disclosure: show summaries by default, then expand only when needed.

Phi Agent Skills follows this model. A Skill is centered on SKILL.md. The name and description are used for discovery. The body contains the workflow. Attached files can provide references or more advanced capabilities.

We are not creating a private Phi-only Skill format. That matters. If Skills become a proprietary plugin format from the beginning, things may move faster for a while, but authors, tooling, tests, and migration paths eventually become locked in. The standard Skills file model is simple, open, and portable. Phi should build on it, not bypass it because we felt like inventing another format for the graveyard.

The part we narrow is the runtime assumption.

In a general assistant environment, code may run inside a sandbox, against local files, or in a controlled workspace. In a browser, scripts may run against pages where the user is already logged in. They may read private content from the DOM. They may interact with page state shaped by cookies, sessions, and previous user actions. They may also be influenced by webpage text that was never meant to be trusted.

So Phi Agent’s position is simple:

Keep the file model as open as possible. Keep the browser runtime as constrained as necessary.

Prompt-only by default

For user-defined Skills, prompt-only is the default.

That does not mean prompt-only Skills are weak. For many writing, analysis, formatting, research, and workflow-guidance tasks, a well-written SKILL.md is enough. It can teach the agent how the user wants something done, what format to follow, what checks to apply, and what domain context matters.

But prompts can only guide reasoning. They cannot reliably change the environment. They cannot safely read the DOM, capture the state of the current tab, access trusted browser worlds, or turn a complicated web export flow into a deterministic pipeline.

The other extreme would be to let every Skill carry arbitrary scripts or allow the model to generate scripts and execute them directly. That is powerful, but inside a browser it is not acceptable. A user-defined Skill with free script execution is no longer just a reusable workflow. It becomes a low-friction code execution entry point sitting next to logged-in sessions and private page content. A tiny convenience grenade, lovingly wrapped in Markdown.

Phi Agent takes a third path:

Prompt-only by default. Trusted code execution only for reviewed built-in Skills. Tool permissions declared in the Skill manifest, but resolved and enforced by the runtime.

This distinction matters because browser-agent errors have a different risk profile. A bad prompt-only Skill may produce a poor answer. A bad browser script may read the wrong page, act in the wrong tab, upload the wrong data, or return private content to the model. These are not the same class of failure.

Code execution belongs to reviewed Skills, not model improvisation

Some operations are better handled by code than by model output. Sorting, parsing, extracting structured data, converting formats, and checking page state are often more reliable when performed by scripts.

Phi Agent supports that, but with strict boundaries.

The model cannot invent JavaScript and ask the browser to execute it. Executable scripts must already exist inside the Skill’s scripts/ directory. They must be explicitly listed in the Skill metadata. They must belong to the active Skill. They must pass server-side checks. Only then can they be dispatched to the browser.

The point is not simply to allow JavaScript execution. The point is this:

The model cannot invent an execution entry point.

The model’s role is limited. It can decide whether a Skill is relevant, read the Skill instructions, and request the execution of a documented script.

The server checks whether the Skill is active, whether the script name is valid, whether the script is allowlisted, whether the file exists, whether it stays within size limits, and whether the returned result is acceptable.

The browser extension then runs the specified trusted code in the specified tab.

This separation is critical. Once the model can both decide what should be done and generate the code that will do it, security review becomes forensic archaeology. You are no longer reviewing a system. You are reviewing whatever the model improvised five seconds ago while staring into the haunted soup of the internet.

Phi Agent avoids that by keeping executable behaviour inside reviewed Skills and runtime validation.

allowed-tools must become a real permission system

In a browser agent, allowed-tools cannot remain a decorative documentation field. It has to become part of the runtime capability system.

If a Skill declares that it needs a tool, the system must turn that declaration into an enforceable permission. It is not enough to tell the model, “you may use this tool.” A model’s self-restraint is not a security boundary. It is barely a personality trait.

Phi Agent recalculates available tools on every turn. The inputs include the active Skill, automatically triggered candidate Skills, installed and enabled Skills, the current hostname, site restrictions in the Skill manifest, declared tool requirements, and the current execution context.

The model only sees the tools that survive this filtering process.

This means permissions are not global defaults. They are derived from the current turn, the current page, and the Skills that are actually allowed to participate.

We prefer mechanical boundaries wherever possible. They cannot solve every semantic risk, but they have one major advantage:

They are explainable, testable, and reproducible.

The first boundary is the current page

For ordinary Skills, the primary boundary is often the filesystem directory. For browser Skills, the first boundary is the current page.

That is why Phi Agent Skills support site-related metadata. A Skill can declare which hostnames it applies to, and trusted scripts can be restricted further using URL patterns.

These restrictions are used in two places: deciding whether the model can see the Skill, and deciding whether the Skill can still run after explicit user selection.

That is much stronger than writing “do not use this on other websites” in the instructions and hoping the model experiences moral growth.

Many browser-agent failures are not dramatic attacks. They are ordinary context mismatches. The user thinks the agent is working on the current YouTube page, but the agent reads another tab. The user asks it to export the current conversation, but the page has navigated elsewhere. The domain is correct, but the path is unsupported. The page is still loading, but the script already runs.

Site and URL gates block a large class of errors before they reach the script. Inside the script itself, task-specific checks still need to happen.

A correct hostname does not mean the page state is correct. A correct URL pattern does not mean the DOM is ready. A logged-in session does not mean the current content is the intended target.

This is why trusted browser scripts need structured failure modes. Instead of vague apologies, they should return stable error codes such as not_on_target_page, state_lost, no_target_tab, or unsupported_page_state.

The model can recover from structured errors. It cannot reliably recover from a script shrugging in natural language.

Why user-defined Skills stay prompt-first

A fair question is: why not let user-defined Skills carry scripts and declare their own tool permissions?

The answer is that writing a reusable workflow should not quietly become installing a browser extension by another name.

User-defined Skills are valuable because they let users encode preferred workflows, output formats, checklists, terminology, and domain knowledge. They can guide how Phi Agent works. They can teach the model what the user expects. But they should not grant new runtime powers by themselves.

Permissions should come from built-in system capabilities, reviewed built-in Skills, and the server-side capability registry, not from a Markdown file written by a user or generated by a model.

This does not mean user-defined Skills must remain limited forever. In the future, more powerful Skills may be possible with signatures, reviews, sandboxes, permission prompts, organization-level publishing, versioning, and test records. A Skill marketplace is not hard to imagine.

But until those mechanisms exist, user-defined Skills should stay prompt-first. Capabilities can be added later. Overbroad permissions are much harder to take back.

Testing the boundaries

For Phi Agent Skills, the main question is not whether the model behaves nicely when everything goes well. That is demo logic, and demo logic is how civilization ended up with smart fridges.

The more important question is whether the harness preserves boundaries when things go wrong: when the page is noisy, when context is misaligned, when the wrong tab is active, when a Skill is disabled, when a script returns too much data, or when a user-defined Skill tries to declare powers it should not have.

The tests that matter are boundary tests:

• Calling a script from an inactive Skill must fail.
• Scripts not included in the allowlist must fail.
• A site mismatch must prevent automatic triggering and block execution after explicit selection.
• A user-defined Skill must not be able to grant itself new tool permissions.
• Oversized script results must be truncated or rejected.
• A multi-stage script that loses its run ID must fail closed.
• Overwriting a user-defined Skill must require the expected version.
• Disabled or uninstalled Skills must not enter the current capability set.

These tests are less flashy than a polished demo. They do not make good launch videos. They are, however, the difference between a browser agent that works in controlled examples and one that can survive contact with the real internet.

Demos prove capability. Regression tests prove boundaries.

The trade-off

The design of Phi Agent Skills comes down to one trade-off:

Be as compatible as possible with the standard Skills file model. Be more conservative when Skills enter the browser runtime.

That means we adopt SKILL.md instead of inventing a private plugin format. We use progressive disclosure instead of stuffing every workflow into the prompt. We support trusted scripts for built-in Skills, but we do not let the model generate and execute scripts on the fly. We turn allowed-tools into runtime capabilities instead of treating it as documentation. We keep user-defined Skills prompt-first until stronger permission systems exist. We let the current page, active tab, manifest, and tool registry jointly decide what is possible.

This costs speed. A powerful built-in Skill requires scripts, manifests, runtime validation, and tests. It cannot be thrown together as a prompt in five minutes and then trusted with the user’s logged-in browser session. Tragic, really. We almost had chaos as a service.

But browser agents need this structure. An agent that only talks is limited. An agent that can do anything is untrustworthy. A browser agent built for the long term needs enforceable boundaries around its capabilities.

Anthropic’s Skills model gives us an excellent starting point: a simple, open, and portable file model. Phi Agent Skills extends that idea into the browser, where live pages, logged-in sessions, and trusted scripts require stricter runtime rules.

This is not just a cleaner abstraction. It is part of the foundation Phi Browser needs before browser agents can move from impressive demos to durable systems people can actually rely on.