Model content policies¶

Editions: OSS, Cloud, Enterprise. Unless stated otherwise, everything on this page ships in OSS.

For detection limits, storage modes, reference-only verification and complete patient/payment examples, see Sensitive-data handling.

Model I/O rules extend the existing policy engine so instance policies can inspect model prompts and completions. Actions are the same as tools (allow, deny, and require_approval) plus notify, which is only available on model I/O rules (see Notify-only rules).

This is policy on Preloop-governed traffic. It is not a standalone guardrail product.

Targets¶

Use these stable names:

model.request: evaluated before the provider is called
model.response: evaluated after the provider returns, before bytes reach the client

When no model I/O rules exist, traffic is allowed. That matches tool evaluation when no access rule matches (No access rules defined / No rules matched (default allow) in backend/preloop/services/policy_evaluator.py).

Console¶

/console/policies is the instance-wide authoring page (sidebar item next to Tools), available by default to users with view_policies permission. Operators can hide the page with PRELOOP_POLICIES_CONSOLE=false; instance admins retain access and backend permission checks remain in force. The per-tool approval widget on Tools links here through Manage in Policies. In the OSS edition, /console/governance redirects here. With the Enterprise weekly review enabled, that route opens the report under Audit > Weekly review.

Create, edit, disable, and delete rules from the Rules list. The primary action is Describe a change: the account default model proposes an edited YAML document. The UI shows a unified YAML diff against the current export. Save applies it; Discard does not. Generation never auto-applies.

The guided Add rule form includes:

target (model.request or model.response)
action (allow, deny, require_approval, notify; tool rules do not offer notify)
the existing approval-workflow picker (not needed for notify)
a condition field
detector toggles for PII, injection, and moderation

YAML import and export on the same page round-trip these rules. Omitting model_io (or setting it to null) preserves the existing rules. An explicit model_io list replaces the section; use model_io: [] to remove all Model I/O rules. The preview reflects these import semantics, so omitted rules are not listed as removals. To remove only one rule through YAML, supply the complete list you want to keep.

Condition attributes¶

Canonical text fields:

request.text: concatenated message contents (and Responses input when present)
response.text: assembled assistant text plus returned reasoning/thinking text (including DeepSeek reasoning content, Anthropic thinking, and Responses reasoning summaries). Opaque encrypted reasoning and signatures are not text.

Also available:

model.id, model.provider, model.name
session.id when a runtime session is present
pii.found (bool), pii.types_found (list), pii.count (int)
injection.score (0-1), injection.matched_patterns (list)
moderation.flagged (bool), moderation.categories (list)

Simple and CEL expressions work the same way as tool conditions. Example: pii.found == true, injection.score > 0.7.

Injection scoring is best-effort. It reuses the deterministic security_screen prompt-injection heuristics. It is not a guarantee.

Detectors¶

Detectors run only when a rule enables them, or when a condition references their attributes (pii., injection., moderation.).

Detector	Default	Result attributes
PII	email, phone, credit-card (Luhn)	`pii.found`, `pii.types_found`, `pii.count`
Injection	`security_screen` regex	`injection.score`, `injection.matched_patterns`
Moderation	local keyword ruleset (`local`)	`moderation.flagged`, `moderation.categories`

Sensitive data types¶

pii.types accepts every built-in type and the name of a custom entry declared under sensitive_data.detectors. A rule that lists no types scans the account default (sensitive_data.detectors.types), and without one the original three.

Type	What it matches
`email`	mailbox addresses
`phone`	E.164 plus US and European national formats
`credit_card`	13 to 19 digits that pass the Luhn check
`iban`	IBANs with a valid mod-97 check
`ip_address`	IPv4 and IPv6 addresses that parse
`date_of_birth`	the nearest date to a birth keyword (DOB, born, Geburtsdatum, date de naissance, ...)
`national_id`	US SSN shape, DE tax id (ISO 7064 check), UK NINO, FR NIR (key), NL BSN (11-proef, keyword anchored); narrow with `locales: [us, de, uk, fr, nl]`
`medical_record_number`	an identifier after MRN or patient-id keywords; `medical_record_number_pattern` replaces the identifier shape
`person_name`	honorific or label anchored names (`Dr. X`, `patient: X`); low recall by design

Account-defined entries:

sensitive_data:
  detectors:
    types: [email, iban, employee_id]      # default scan set (optional)
    locales: [de, nl]                      # national_id locales (optional)
    custom_patterns:
      - name: employee_id
        regex: 'EMP-\d{6}'
        flags: [i]
    keywords:
      - name: codenames
        terms: ["Project Phoenix"]
        case_sensitive: false

An account may configure at most 50 custom patterns and 50 keyword lists. Custom regexes are capped at 512 characters, must not nest quantifiers ((a+)+) or use backreferences, and each pattern has a 0.25 second match timeout. All account patterns on one text share a one-second budget; exceeding either limit counts as a detector timeout for the rule. Keyword lists match whole words only.

GET /api/v1/policies/sensitive-data/types lists every selectable type with a label, description, example and locales. POST /api/v1/policies/sensitive-data/test runs the detectors on sample text and returns the match spans; the text is never logged or stored.

Sensitive data rules on tool calls¶

sensitive_data.rules run the same detectors on MCP tool calls and, for model targets, compile into ordinary model I/O rules so each path keeps one evaluator.

sensitive_data:
  rules:
    - id: block-cards-in-tools
      on: [tool.args, tool.result, model.request, model.response]
      scope: {agents: [], tools: [], servers: []}   # empty means all
      types: [credit_card, iban]                    # default: the detectors block types
      action: deny                                  # notify | deny | require_approval
    - id: ask-before-sharing-records
      on: [tool.result]
      scope: {tools: [get_patient_record], servers: [ehr]}
      action: require_approval
      approval_workflow: humans

scope.agents holds managed agent ids, scope.tools client-visible tool names, scope.servers MCP server names (case-insensitive; builtin tools belong to preloop-mcp). Model targets honour agents only.
tool.args runs before the call. deny refuses with the types found (never the values); require_approval creates an approval through the rule's workflow or the account default and fails closed without one; notify records a notice and lets the call through. Tool access-rule conditions can read pii.found, pii.types_found, pii.count and pii.paths from the scan next to args.
tool.result runs after the call. deny replaces the result with a refusal; require_approval holds the result until a human decides.
Audit rows carry the rule id, target, types found, argument paths and a SHA-256 of the scanned text. With no rule in scope no detector runs.

Tool metadata and argument predicates¶

For server allowlists, write-tool patterns and bulk-export checks, a sensitive_data.rules entry may use a CEL condition instead of detector types. It binds args, tool.name and tool.server, runs only on tool.args, and supports notify, deny and require_approval. Evaluation errors skip notify rules; deny and approval rules fail closed. A condition rule cannot use detector types or redact. See the downloadable pack in Governing back-office AI use.

Reference-only logging¶

sensitive_data.reference_only names tools, servers or agents whose calls must leave no payload behind. Every store that would hold the arguments or the result of an in-scope call (audit rows, policy decision rows, approval rows, activity summaries, the tool-call search index) holds a reference record instead: tool, server, principal, rule id, the values named by keep_fields, args_hmac and result_hmac, byte sizes, key names, timing and cost.

sensitive_data:
  reference_only:
    - id: patient-tools
      scope: {agents: [], tools: [get_patient_record], servers: [ehr]}
      keep_fields: ["$.consent_id", "$.call.id"]   # dotted keys, [n], [*]
      approver_view: redacted                       # redacted | original_until_decided

A path that starts with $result instead of $ reads the tool result. For a tool that returns the consent it acted under:

sensitive_data:
  reference_only:
    - id: owner-data-reference-only
      scope: {tools: [read_scope], servers: [lifecare-sandbox-source]}
      keep_fields:
        - "$.scope"              # from the arguments, stored under kept
        - "$result.consent_id"   # from the result, stored under kept_result

The value is read from the result's structuredContent; when that is absent or lacks the path, from the first text content block parsed as JSON (a tool that returns a JSON string arrives that way). A missing path, or a result that is not JSON, keeps nothing. The result itself is never stored: the audit tool_call row gets kept_result (here {"$result.consent_id": "cons-42"}) and result_hmac, the salted fingerprint of the result, next to kept and args_hmac. Kept values, argument or result, go through the redact rules in scope first. $.result.x still means an argument named result.

The tool_call row's rule_matched holds the access rule the policy evaluator matched for the call (the same text as rule_description on the policy_* row with the same correlation_id). The reference-only rule id stays in the record's rule_id.

New fingerprints use scrypt over canonical JSON with a per-account salt stored encrypted; historical HMAC-SHA256 records remain verifiable. The legacy field names args_hmac and result_hmac hold either format. Each record carries its salt_id, so a rotation leaves old records verifiable. POST /api/v1/policies/sensitive-data/hash-check (manage_policies) answers whether a candidate payload matches a stored fingerprint for the caller's account without storing it. The audit chain export lists the account's salt ids and never a salt.

Approvals: with approver_view: redacted the approver sees the reference record. With original_until_decided the raw arguments stay encrypted on the pending row, are returned by GET /api/v1/approval-requests/{id}/original-args (console, decide permission) while pending, and are deleted in the same transaction as the decision. Emails, webhooks and push payloads never carry them. An asynchronous post-approval replay of a reference-only call cannot reconstruct the arguments and reports that; the in-process approval wait keeps the original in memory and is unaffected.

Retention and legal holds: reference records follow the existing retention of their store; a legal hold keeps the records but cannot resurrect payloads that were never stored.

Each rule has detector_timeout_ms (default 500) and on_detector_timeout (default deny, fail closed). Set on_detector_timeout: allow to skip that rule on timeout. A rule whose conditions are all notify never blocks on timeout: the rule is skipped.

Application logs never include full prompts. Audit and approval tickets store the rule id, detector summary, and a SHA-256 of the scanned text. They do not store a raw preview (the preview can be the PII the rule blocked).

Streaming¶

Response rules use buffer-until-assembled: the gateway holds SSE events until response.text is complete, evaluates policy, then either replays the buffered events or returns a deny error. The blocked payload is never sent to the client. When no model.response rules exist, streams pass through without buffering.

Buffering is required for deny and require_approval. Tokens already sent to the client cannot be retracted, so a rolling window cannot enforce those actions (it can miss PII from earlier chunks and cannot unsend them). The tradeoff: time-to-first-token becomes time-to-last-token when any model.response rule is enabled. Issue #313 allowed either buffer-until-assembled or a rolling window; this implementation keeps buffering for correctness of deny.

When every enabled model.response rule is notify-only, the stream is not buffered. Events reach the client as they arrive and the rules are evaluated once the stream ends (or the client disconnects). If any enabled response rule can deny or require approval, the stream is buffered as above and notify conditions are evaluated on the same assembled text.

Deny errors¶

Denied requests return HTTP 403 with:

code: content_policy_denied
message: Blocked by content policy (rule <id>)

OpenAI-compatible clients can surface that message.

Approvals¶

require_approval uses the existing tool-approval workflow (require_approval / approval_service). The hold appears in the existing approvals inbox. The ticket includes rule id, detector summary, and a SHA-256 of the scanned text. It does not store the full prompt or a raw preview. HTTP workers and background optimization jobs run the hold on the application event loop. Callers with no loop fail closed instead of creating a temporary loop. A required approval pauses a background optimization worker for the approval window (default 5 minutes, configurable up to 30 days).

Notify-only rules¶

A condition with action: notify lets the call through exactly like allow: no approval, no buffering, no error to the client. It records that the rule matched and tells the policy owners.

Evaluation order:

A matching notify condition records one hit for its rule and evaluation continues with the next rule. A rule records at most one hit per evaluation, even when several of its notify conditions match.
deny and require_approval still use first match wins, and that terminal match is stored as a hit with its action. allow is not stored. A notify rule listed before the terminal match still records its hit (so a denied call can also produce a notice). Rules after the terminal match are not evaluated.
A notify condition whose expression fails to evaluate is skipped.
When only notify rules matched, the decision is notify, which proceeds like allow.

Each notify, deny, and require_approval match is stored in the policy_notice_hit table with the account, user (when known), target, rule id and description, time, the SHA-256 of the scanned text, an excerpt of at most 280 characters around the match, the action (notify, deny, or require_approval), the model alias, the runtime principal (agent_ref), the runtime session id, and the audit event id. The excerpt goes through the same secret redaction as the gateway logs. If redaction fails, only the hash and rule id are kept. The full text is never stored. redact is reserved on the column. A redact match is audited as counts by type and is not stored as a hit, because an excerpt would keep the value the rule exists to remove.

The core audit service records an audit row for each stored hit in every edition. Notify rows use action notify. Deny and require_approval rows use those actions. The hit's audit_event_id is that audit row's id. The audit service nests tool arguments under details.tool_args and then copies extra_details onto the top level of details, which is where audit_event_id has to sit.

Policy owners are told about a notify hit through:

email, for owners whose notification preferences leave email on
mobile push, for owners with enable_mobile_push on
the rule's approval workflow (or the account default) when that workflow is of type Slack, Mattermost or webhook

Owners are active users who are the account's primary user, a superuser, or hold the manage_policies permission. The message names the rule, includes the redacted excerpt, and has no approve or deny link. Messages are debounced to one per rule, per user, per hour. Hits are still recorded every time.

Where hits show up:

Attention: one "Policy notice" card per rule with hits in the last 7 days. Dismissing it hides it until the rule matches again.
Optimization digest: a "Policy notices" section with each rule's hit count over one window, the last user and the last excerpt. The window is the 7 days ending at now by default, or exactly the start and end the caller passes; both bounds are always applied, and the section reports the window it covered as window_start and window_end. The digest is sent by the EE optimization plugin; without it, nothing is sent.
GET /api/v1/policies/notices/summary?days=7 (requires view_policies). group_by is rule (the default), user, or action. The default rule list is notify hits only, which is what the Attention cards use. group_by=user and group_by=action count deny and require_approval hits as well. The response also includes model and agent breakdowns of every stored action. The rule list, the user list, and each breakdown are capped at 100 rows. Within that cap, the per-user counts add up to the rule totals. Past it the lists are truncated separately, so those two sums can diverge.

Example:

model_io:
  - id: notify-codename
    description: Mentions of the internal codename
    target: model.request
    conditions:
      - expression: "request.text.contains('project-x')"
        action: notify

notify is rejected on tool rules.

YAML example¶

version: "1.0"
metadata:
  name: Content policies
approval_workflows:
  - name: high-risk
    timeout_seconds: 300
model_io:
  - id: deny-pii-in-prompts
    target: model.request
    detectors:
      pii:
        types: [email, phone, credit_card]
    conditions:
      - expression: "pii.found == true"
        action: deny
  - id: approve-flagged-output
    target: model.response
    approval_workflow: high-risk
    detectors:
      moderation: true
    conditions:
      - expression: "moderation.flagged == true"
        action: require_approval
  - id: deny-injection
    target: model.request
    detectors:
      injection: true
    conditions:
      - expression: "injection.score > 0.7"
        action: deny

Optional sensitive_data, access_rules, access_rule_mode, resource_access, and resource_shares sections also preserve their current configuration when omitted or set to null. Explicit values replace the corresponding section; use [] for list sections or {} for configuration maps to reset them. The preview uses these same import rules.

The mcp_servers, approval_workflows, and tools sections instead add or update named entries. Omitted/null sections, empty lists, and entries absent from a supplied list preserve the existing entries. Importing [] does not clear these sections, and the preview does not report those entries as removed.