Skip to content

Testing rituals

A ritual is a program, and it deserves the same test any other program gets. vekna.trial runs one with every medium answering from a script, so a test costs no agent call and no wall time.

Testing a ritual with vekna.trial vekna.trial runs a ritual with doubles at the folio outer edge: the mediums still run, each call is answered from a script by glob then queue, nothing defaults, and the recordings are what a test asserts on. Testing a ritual with vekna.trial A ritual runs with every medium answering from a script: no agent call, no wall time. pip install 'vekna[trial]' brings a trial pytest fixture. WHERE THE DOUBLES STAND your test scripts the answers, runs a step or a cast, asserts on what the doubles recorded trial.walk(…) trial.cast(…) the ritual's steps your code, unchanged: branches, loops, budgets, failure routing shell(…) · enqueue(…) coding(…) · decide(…) the mediums' own bodies still run, so they are tested too: • session threading • resume resolution • output-schema validation • exit-code handling a wrong Session.CONTINUE fails its test the folio's outer edge shell double exit code, stdout, stderr coding double the agent's reply, tools, questions decide double the operator's answer queue double a queued cast's result the world real bash a live agent you at a prompt the daemon's queue never reached from inside a trial the script: answers, matched per call the recordings: every call, as the medium built it THE MECHANISMS Two ways in trial.walk(step, payload) one step, no ritual required hands back its transition: the next payload, or Done(result) trial.cast(ritual, components) the whole ritual, across steps returns the result model; trial.steps says which ran, in order walk_async · cast_async for a suite already in an event loop How a call finds its answer a call arrives at its double 1 · a glob claims it by command for shell, prompt for coding and decide, ritual for queue 2 · else the ordered queue used up as it answers; always=True keeps one standing 3 · nothing left: TrialScriptError naming the call and what the script still held nothing defaults: an invented exit 0 would test a branch nobody wrote Assert on what was recorded no assert_called_with: the recording is the surface the doubles shell.commands coding.prompts coding.gated coding.calls decide.prompts queue.enqueued queue.rituals the cast trial.steps trial.deltas trial.events coding.calls[1].resume proves the second call joined the first one's thread a decide answer not among the options offered raises before the ritual sees it WHAT WILL CATCH YOU OUT A tool gate is a decide allow tool 'Bash'? arrives at trial.decide, not at trial.coding: one channel builds both Only inside its with-block outside it, cast and walk raise TrialError rather than fall through to real bash Calls in one TaskGroup arrive in whichever order the scheduler picks; errors come wrapped in BaseExceptionGroup 
pip install 'vekna[trial]'

That brings a trial pytest fixture.

One step

trial.walk(step, payload) runs a single step and hands back its transition. No ritual required — this is the unit of a step test.

def test_a_green_run_finishes(trial):
    trial.shell.replies(exit_code=0)

    transition = trial.walk(fix, Attempt(left=3))

    assert transition.result.outcome == "green"
    assert trial.shell.commands == ["pytest"]

A whole cast

trial.cast(ritual, components) runs the ritual across steps and returns the result model.

def test_it_gives_up_after_the_budget(trial):
    trial.shell.replies(exit_code=1, always=True)
    trial.coding.replies("tried something", always=True)

    result = trial.cast(fix_tests, FixTests(bound=2))

    assert result.outcome == "gave up"
    assert len(trial.coding.prompts) == 2

Scripting the answers

Each double takes answers matched by a glob — the command for shell, the prompt for coding and decide, the ritual name for queue — and falls back to an ordered queue for whatever no pattern claims:

trial.shell.replies(when="mise run lint*", exit_code=0)
trial.coding.replies("wrote a test", uses=["Bash"])
trial.decide.answers(answer=True, when="*proceed*")
trial.queue.returns(Fixed(outcome="pushed"), when="pr_fix")

trial.queue answers await_cast only, returns with the child's result or fails(detail) with CastFailedError; an enqueue needs no script.

always=True keeps one answer standing instead of consuming it. A decide answer is checked against the options the step offered, and one that is not among them raises before the ritual sees it — the real channel's contract.

Nothing defaults. An unscripted call raises TrialScriptError naming the call and what the script still held. An invented exit_code=0 would send the ritual down a branch nobody wrote and report the run as a pass, which is worse than no test at all.

A trial is attended, the way a plain cast is. trial.decide.attended = False is the cast run with --unattended: attended() answers false, and a decide the ritual asks anyway raises UnattendedPromptError, scripted answer or not.

Assert on what the doubles recorded

trial.shell.commands, trial.coding.prompts, trial.coding.gated, trial.decide.prompts, trial.queue.enqueued, trial.queue.rituals. There is no assert_called_with here — the recording is the assertion surface.

The cast records too: trial.steps is the step names in the order they ran, trial.deltas and trial.events the grimoire as it happened, and trial.coding.calls the calls as the medium built them — calls[1].resume is how a test proves the second call joined the first one's thread.

What the doubles do not replace

The doubles stand at the folio's outer edge, where a medium reaches the outside world, so the medium's own body still runs: session threading, resume resolution, output-schema validation and exit-code handling are all exercised.

A ritual that mis-declares session=Session.CONTINUE fails its test. That is the point of doubling at the edge rather than mocking the medium.

Two things that will catch you out

  • A tool gate arrives at trial.decide, not at trial.coding. The folio builds both out of the same channel, so allow tool 'Bash'? is a decide.
  • A Trial answers only inside its with block. The fixture hands you one already entered. Outside it, cast and walk raise TrialError rather than letting shell() fall through to real bash — a test that forgot the block would otherwise run its commands for real and pass.

Two mediums started in one asyncio.TaskGroup arrive in whichever order the scheduler picks, which is what the ordered queue is for. An exception from inside a group arrives wrapped, so a test reaching a grouped step needs pytest.raises(BaseExceptionGroup) and an assertion on .exceptions[0]. That is Python's doing, not the trial's.

trial.cast_async and trial.walk_async are for a suite already inside an event loop; the plain forms own one.