← Development & Operations AI

Shared AI assessment methodology

Four separate claims. Evidence for each.

For Development and Operations agents, software generation identifies the implementation release. Capability describes demonstrated work. Autonomy describes designed user involvement. Authority records what governance permits. None substitutes for another.

Primary autonomy benchmark

Levels of Autonomy for AI Agents · Knight First Amendment Institute at Columbia University · 2025-07-28

  1. Level 1 · User as operator

    The user directs the workflow and invokes agent support.

  2. Level 2 · User as collaborator

    The user and agent plan, delegate, and execute through frequent collaboration.

  3. Level 3 · User as consultant

    The agent plans and executes bounded work while the user supplies direction, preferences, and feedback.

  4. Level 4 · User as approver

    The agent handles a bounded workflow and involves the user for blockers or consequential approvals.

  5. Level 5 · User as observer

    The agent operates without normal user involvement; only monitoring and an emergency stop remain.

Read the original Knight framework

Capability evidence dimensions

Practices for Governing Agentic AI Systems · OpenAI · 2023-12-14

Goal complexity

How challenging and varied are the goals the system can demonstrably pursue?

Environmental complexity

Across how many tools, domains, stakeholders, and time horizons can it operate?

Adaptability

How well does it respond to novel or unexpected circumstances?

Independent execution

How reliably can it achieve goals with limited direct supervision?

Qualitative evidence dimensions only. They are not converted into a Beast score or claimed as an OpenAI certification.

Read the original OpenAI publication

Classification method

  • • Classify designed user involvement in the agent's actual bounded operating environment, not model capability or marketing aspiration.
  • • Require reproducible workflow evidence and explicit stop, approval, and escalation behavior.
  • • Record the highest demonstrated level no greater than the behavior evidenced; reassess after a material workflow or authority change.
  • • Keep autonomy separate from capability and canonical authority.

Comparison limits

  • • BeastFusion classifications are internal self-assessments, not Knight Institute certificates.
  • • A level describes designed user involvement for a bounded environment, not intelligence, safety, quality, or general capability.
  • • Comparisons can become stale when models, tools, workflows, or governance change.

Every Beast classification is an environment-bound self-assessment, not certification or a universal industry standard.