Persona Coherence

Persona coherence evaluation tests whether an agent maintains a consistent identity across all dimensions simultaneously. Rather than testing individual aspects, this evaluates the agent as a whole person.

What it tests01

• Does the agent maintain a coherent identity across interactions?

• Are skill usage, personality, communication, and thinking patterns internally consistent?

• Does the agent's behavior match its declared persona?

• Is the identity stable under pressure or unusual inputs?

Scorer: persona_coherence02

The persona_coherence scorer evaluates end-to-end identity consistency across all available dimension scores:

testcase.ScorerConfig{
    Name: "persona_coherence",
    Config: map[string]any{
        "persona_ref":     "senior-engineer",
        "min_consistency": 0.75,
        "dimensions":      []string{"skill", "trait", "communication", "cognition"},
    },
}

Scenario: persona_coherence03

The persona_coherence scenario generator creates test cases that exercise multiple dimensions simultaneously:

Case{
    ScenarioType: testcase.ScenarioPersonaCoherence,
    Input:        "A junior developer asks you to review their first pull request. It has a security issue and some style problems.",
    Context: map[string]any{
        "persona":          "senior-engineer",
        "expected_traits":  []string{"thoroughness", "patience"},
        "expected_skills":  []string{"code-review"},
        "expected_tone":    "professional but supportive",
    },
}

This scenario tests that the agent:

• Uses code review skills correctly (skill dimension)

• Shows thoroughness and patience (trait dimension)

• Communicates professionally but supportively (communication dimension)

• Follows an analytical then reflective cognitive pattern (cognition dimension)

Dimension score04

result.DimensionScores["persona"] // 0.0 to 1.0

The persona score is typically derived from the consistency across all other dimension scores. A high persona coherence score means the agent performs well across all dimensions in a way that forms a coherent identity.

PersonaRef05

Suites can declare a PersonaRef field that references an agent persona by name. When set, persona-aware scorers use this reference to look up expected traits, skills, and behaviors:

s := &suite.Suite{
    Name:       "senior-engineer-eval",
    PersonaRef: "senior-engineer", // references an agent persona
}

Use cases06

• End-to-end evaluation of a fully configured AI agent

• Regression testing after persona configuration changes

• Comparing two different persona configurations for the same role

• Validating that a persona behaves consistently across diverse scenarios