Release governance: guardrails for agents at scale

Sierra Blog ·

A source describing release governance features for AI agents built on the Sierra platform: formalized safety processes for large-scale deployments, automated pre-deployment validation via Agent Checks, and incremental deployment using split traffic releases. Read 3 viewpoints with supporting evidence and source links.

Understand this piece

3 key points

Synthesis

  1. Release safety at scale requires formalized governance

    For large companies building agents on Sierra—with hundreds of people collaborating across hundreds of customer journeys and serving millions—informal checks or individual memory are insufficient for safe agent releases.

    Supporting evidence 1

    Original excerpt

    Some of the world’s largest companies build their agents on Sierra: hundreds of people working inside a single agent, across hundreds of journeys, serving millions of customers. A small change to the agent can instantly reshape how it behaves with every customer. At that scale, releasing an agent safely can’t rely on an informal check or on someone remembering to double-check a change.

    Sachi Shah · Paragraph 1

    Read in source context →
  2. Agent Checks surface high-impact configuration issues before deployment

    Agent Checks acts as a linter that proactively identifies costly, easy-to-miss issues—such as referenced but unavailable tools, conflicting instructions, role confusion between lookup and action tools, response failures in voice contexts, or insufficient authentication for sensitive data lookups—and prioritizes them by severity, with inline Ghostwriter fixes.

    Supporting evidence 1

    Original excerpt

    It catches the issues that are easy to miss and expensive to ship: a tool your prompt references but never made available, conflicting instructions to the agent, a lookup tool doing an action tool’s job, a response that works on screen but falls apart on a call, or a sensitive-data lookup with insufficient authentication. Checks are prioritized by severity, helping teams distinguish issues likely to impact customers from lower-priority quality improvements, and most come with a suggested fix from Ghostwriter that can be applied in place.

    Sachi Shah · Paragraph 5

    Read in source context →
  3. Split traffic releases enable canary-style rollouts for agent changes

    Split traffic releases let organizations incrementally deploy agent changes to a subset of customer traffic before full rollout—enabling verification of model upgrades, authentication redesigns, or behavioral changes—and are already used in production by a large airline, a travel marketplace, and a fintech company.

    Supporting evidence 1

    Original excerpt

    Our new split traffic releases let organizations incrementally roll out a release to a portion of customer traffic before expanding it to everyone — the same canary strategy software teams have relied on for years. If you’ve upgraded your model, redesigned authentication, or made a sweeping behavioral change, you can verify the release behaves as expected before rolling it out more broadly. A large airline, travel marketplace, and fintech company are already using split traffic to control how rollouts reach their customers.

    Sachi Shah · Paragraph 13

    Read in source context →

    Continue exploring

    incremental deployment →

Key passages3

Attributed passages with the context to verify them. Open the original text to check the source.

release governance for AI agents

Release safety at scale requires formalized governance

Original excerpt

Some of the world’s largest companies build their agents on Sierra: hundreds of people working inside a single agent, across hundreds of journeys, serving millions of customers. A small change to the agent can instantly reshape how it behaves with every customer. At that scale, releasing an agent safely can’t rely on an informal check or on someone remembering to double-check a change.
incremental deployment

Split traffic releases enable canary-style rollouts for agent changes

Original excerpt

Our new split traffic releases let organizations incrementally roll out a release to a portion of customer traffic before expanding it to everyone — the same canary strategy software teams have relied on for years. If you’ve upgraded your model, redesigned authentication, or made a sweeping behavioral change, you can verify the release behaves as expected before rolling it out more broadly. A large airline, travel marketplace, and fintech company are already using split traffic to control how rollouts reach their customers.
automated agent validation

Agent Checks surface high-impact configuration issues before deployment

Original excerpt

It catches the issues that are easy to miss and expensive to ship: a tool your prompt references but never made available, conflicting instructions to the agent, a lookup tool doing an action tool’s job, a response that works on screen but falls apart on a call, or a sensitive-data lookup with insufficient authentication. Checks are prioritized by severity, helping teams distinguish issues likely to impact customers from lower-priority quality improvements, and most come with a suggested fix from Ghostwriter that can be applied in place.

Source & methodology

These viewpoints are linked to their original sources. Paraphrases are labeled and are not verbatim quotes.

Open transcript or source material (opens in a new tab)Report an issue

Explore these viewpoints by person