Agent Governance for Production AI Systems

What is agent governance?

Agent governance is the discipline of controlling, verifying, and auditing the behavior of autonomous AI agents at runtime. Unlike application security (AppSec) which protects static code, agent governance must handle dynamic, LLM-driven behavior that cannot be fully predicted at deployment time.

An agent governance framework provides: identity verification for every agent, a state machine with model-checked transitions, runtime safety gates that intercept unsafe actions before execution, drift detection to catch model degradation, and a circuit breaker that halts runaway behavior automatically.

The MAREF framework implements all of these as a cohesive governance operating system — not as features bolted onto an orchestration tool.

Why do production agents need governance?

Production agent systems face risks that orchestration frameworks alone cannot handle. A LangGraph or CrewAI deployment can orchestrate complex workflows, but without governance, there is no mechanism to enforce safety boundaries at runtime.

The OWASP Agentic Top 10 identifies critical risks including: unauthorized tool access, prompt injection through agent-to-agent communication, denial-of-wallet attacks, excessive agency (agents taking actions beyond their intended scope), and supply chain vulnerabilities in agent capabilities. These risks are not theoretical — real-world incidents have caused data leaks, financial loss, and regulatory penalties.

What makes MAREF's governance different from other frameworks?

Every major agent framework (LangGraph, CrewAI, AutoGen, Anthropic's agent protocol) treats safety as a secondary concern. Their focus is orchestration — getting agents to work together efficiently. MAREF inverts this priority: governance is the foundation, and orchestration is built on top of it.

Specific differentiators:

• TLA+ model checking — the governance FSM is model-checked against TLA+ specs of its constitutional red-line invariants.
• 10-state Gray Code FSM — Hamming distance = 1 ensures every transition changes exactly one dimension of trust state. No ambiguous intermediate states.
• Four-level safety decision tree — Rule → Mode → SafetyGate → User escalation, resolving routine decisions automatically and escalating edge cases to human judgment, with clear audit trails.
• Runtime drift detection — LoRA/ontology dual detection catches model drift before it causes safety failures.
• Circuit Breaker with HALT absorption — Three consecutive failures trigger an irreversible halt that only human override can release.

How does the governance state machine work?

MAREF's governance engine uses a 10-state finite state machine encoded in 4-bit Gray code. Each state represents a distinct trust level, from full autonomy to complete halt. The Gray code encoding ensures adjacent states differ by exactly one bit — eliminating ambiguous transitions.

The state machine includes an absorbing HALT state. When three consecutive safety failures occur within a configurable time window, the system transitions to HALT. No agent, pipeline, or automated process can override this state. Only explicit human authentication can release it.

from maref import GovernanceStateMachine, GovernanceState

sm = GovernanceStateMachine()
sm.transition(GovernanceState.OBSERVE, "start monitoring")

# HALT (state 9) is absorbing — no outgoing edges.
# Leaving it requires an explicit, authenticated human override.

Can I use MAREF with existing frameworks?

Yes. MAREF is designed as a governance overlay — it sits above or alongside your existing orchestration framework. Integration works today at the protocol level via MCP for tool calls and A2A for inter-agent handoff. Native sidecar adapters are rolling out — the AutoGen adapter is shipped; LangGraph's is specified but not yet merged (see docs/quickstart §3.3). The governance layer intercepts agent actions before execution, verifies trust state, applies safety rules, and only then forwards approved actions to your orchestrator.

This complementary architecture means you keep your existing agent workflows and add governance without rewriting anything. pip install maref is all it takes to get started.

What threats does agent governance protect against?

MAREF's governance engine covers all 10 categories of the OWASP Agentic Top 10, including:

• Excessive agency — Agents cannot escalate their own permissions or take actions outside their declared scope.
• Tool access abuse — Every tool call is verified against the agent's permission matrix before execution.
• Agent-to-agent injection — Cross-agent communication is isolated per SubAgent, preventing prompt injection spread.
• Denial of wallet — Gas metering and cost budgets prevent runaway LLM calls from exceeding financial limits.
• Supply chain attacks — Every Skill and capability has a signed manifest with integrity verification.
• Non-human identity abuse — Zero-trust identity per agent with SM2/SM3/SM4-GCM cryptographic binding.

Where can I learn more?

Read the Governance feature page for technical details on the Gray Code FSM and TLA+ verification. See Runtime Safety for operational deployment guides. Check the Recursive Evolution page for autonomous self-improvement capabilities.