LlamaBrain™

Deterministic NPC interaction pipeline built around authoritative state and a stateless LLM generator.

Current Version: v0.3.0-rc.2 (The Structured Update)

What it is

LlamaBrain is an architecture and implementation pattern for neuro-symbolic AI-driven game dialogue and intent emission where the model is treated as untrusted. Continuity is produced by deterministic state reconstruction: immutable snapshots, bounded prompts, strict validation, and explicit memory mutation.

Core principle

The LLM has no memory, no authority, and no direct world access. It receives only a bounded prompt assembled from authoritative state. Its output is parsed and must pass validation before it can affect memory or the game world.

Architecture (nine components)

  1. Untrusted Observation → Interaction Context
    Trigger reasons include player utterance, zone trigger, time trigger, quest trigger, NPC interaction, world event, and custom triggers.
  2. Determinism Layer (Expectancy Engine)
    Rules evaluate the interaction context and produce constraints (Permission/Prohibition/Requirement with severity). These constraints are used both to shape the prompt and to validate the output.
  3. External Authoritative Memory System
    Structured memory with an authority hierarchy:
    • Canonical Facts (immutable, designer authority)
    • World State (mutable, game-system authority)
    • Episodic Memory (appendable, validated-output authority)
    • Beliefs/Relationships (lowest authority, can be wrong, confidence-filtered)
  4. Authoritative State Snapshot
    An immutable snapshot captures interaction context, retrieved memories, constraints, history, and attempt metadata so retries can be replayed deterministically.
  5. Ephemeral Working Memory
    Builds a bounded, token-efficient prompt from the snapshot with explicit limits and deterministic ordering.
  6. Stateless Inference Core
    Calls the model (via llama.cpp HTTP). Output is treated as untrusted text.
  7. Output Parsing & Validation (Validation Gate)
    Parses dialogue plus proposed mutations and intents. Validates against constraints, canonical facts, knowledge boundaries, and mutation legality. Supports retry with constraint escalation.
  8. Memory Mutation + World Effects
    Executes only approved mutations (append episodic, transform belief/relationship, emit world intent) with strict authority enforcement. Canonical facts cannot be overridden.
  9. Author-Controlled Fallback
    If retries are exhausted, returns a context-aware fallback response with no mutations.

Few-shot prompting

Includes an in-context example system that can inject demonstrations into prompt assembly with deterministic ordering. Fallback lines can be converted into few-shot examples to improve normal generation and failure handling.

What “deterministic” means here

Determinism is claimed at the system level: deterministic snapshot creation, prompt assembly, validation decisions, and memory mutation rules. Model generation itself is treated as stochastic, with a feature to lock entropy using deterministic seeding via InteractionCount.

Proof and engineering discipline

  • RedRoom testing suite and Unity PlayMode integration tests are part of the implementation.
  • Architecture has 2000+ passing tests and ~90% line coverage.

My Role

Creator, architect, and sole implementer of LlamaBrain.

Key Contributions

  • Built a deterministic runtime control plane for Unity that treats LLMs, voice, and tools as untrusted proposers.
  • Defined a deterministic JIT intent compilation pipeline: proposal → structured intent → deterministic IR → validated, scoped execution.
  • Implemented validation gates and capability boundaries so only whitelisted, schema-valid actions can mutate runtime state.
  • Added failure-path design as a first-class feature: explicit rejection reasons, safe fallbacks, and zero-mutation guarantees on invalid output.
  • Produced replayable audit traces for every interaction, enabling explainability, debugging, and deterministic reproduction from captured state.
  • Designed mutations to be reversible via explicit diffs and rollback handles, enabling safe live iteration.
  • Authored determinism proofs and contract-style tests (fixed seeds, golden outputs) to verify correctness rather than assert it.
  • Packaged the system as modular layers (core contracts/policy, execution, Unity host adapters) to avoid monolith drift and keep the control plane engine-facing.

Tech Stack

  • C# for the core runtime and engine adapters
  • Engine-agnostic core design with planned adapters for Unity and potential Unreal/custom engines
  • Local LLM backends (e.g. llama.cpp / compatible HTTP or RPC servers)
  • JSON schemas and contracts for tools, actions, and structured responses

Why This Example Matters

  • LLMs are nondeterministic; games and live systems cannot be.
    This work shows how to safely integrate stochastic generators into real-time systems without surrendering authority or correctness.
  • Most AI demos avoid failure paths.
    This system is built around them. Invalid output is expected, detected, rejected, and proven harmless.
  • Runtime changes are usually opaque and irreversible.
    Here, every mutation is scoped, logged, replayable, and reversible, which makes live debugging and iteration safe instead of risky.
  • “AI-driven” systems usually rely on trust.
    This replaces trust with governance: validation gates, capability boundaries, and test-gated execution.
  • It moves AI from a feature to infrastructure.
    NPCs, quests, voice commands, and tools become clients of a control plane, not special cases with hidden authority.
  • It demonstrates senior-level systems judgment.
    Determinism, auditability, rollback, and proof of correctness are the concerns that surface only in production, not prototypes.

In short: this work shows how to use AI in runtime systems without losing control of the system itself.

How to Use

  • Run a local inference backend (for example, a llama.cpp-compatible HTTP server).
  • Integrate LlamaBrain as a library alongside your game runtime:
    • build an Interaction Context when an NPC needs to respond (player utterance, zone trigger, quest trigger, time trigger)
    • evaluate expectancy rules to produce constraints
    • retrieve memory and build an immutable StateSnapshot
    • assemble a bounded prompt from the snapshot
    • send the prompt to the backend
    • parse and validate the response through the validation gate
    • apply only approved mutations and intents
    • if validation cannot succeed within the attempt limit, return an author-controlled fallback with no mutations

Honest Current State Notes

  • The system guarantees determinism for state reconstruction and decision boundaries (snapshot, prompt assembly, validation/mutation decisions), while the model’s raw text generation remains stochastic unless/ until deterministic seeding is used.

Links