memdoor/examples/workflows/team-solve at main · guregodevo/memdoor
Open Coding agent. Shift + Tab Go Team Mode: a team of small models, frontier results. - guregodevo/memdoor
Open Coding agent. Shift + Tab Go Team Mode: a team of small models, frontier results. - guregodevo/memdoor
A team of small open models on one task, each role a fresh context, the work checked before the run passes. This folder holds the published steps of the GVS5H method (Lee, 2026) as a Memdoor workflow: YAML files, no code, no model named. The built-in that grew out of it, team-solve, runs on any repository.
| benchmark | model | single call | team-solve | per instance |
|---|---|---|---|---|
| SWE-bench Verified, all 500 | DeepSeek V4.1 Flash | 388, 77.6% | 400, 80.0% | $0.022, median 258 s |
| LiveCodeBench Hard, the authors' first ten, five passes | Qwen3.8-27B | 8.0 (Fable 5, one call) | 9.8 | $0.39, median 40 min |
The SWE-bench run, with every patch, log and trajectory and the kit that ran it: guregodevo/swe-bench; the summary: bench/memdoor-swe-bench. The LiveCodeBench runs: bench/results/parity-5pass.
curl -fsSL https://memdoor.ai/install.sh | bash
memdoor workflow try # the team on a sample problem, two minutes to a verdict
cd your-project && memdoor workflow run team-solve "fix the failing test in pkg/auth; go test ./... must pass"
A run as it plays, one minute: memdoor.ai/docs/team-mode. How the published steps work, and what the runs found: below.
The authors' harness on the same model, Qwen3.8-27B, with their prompts, loop and 128K reply cap. The first ten problems of their LiveCodeBench Hard list, five passes each, graded on every hidden test with their corrected grader:
| pass | this workflow | authors' harness | Fable 5, one call |
|---|---|---|---|
| 1 | 10 / 10 | 9 | 7 |
| 2 | 10 / 10 | 10 | 8 |
| 3 | 10 / 10 | 10 | 10 |
| 4 | 10 / 10 | 9 | 8 |
| 5 | 9 / 10 | 10 | 7 |
| mean | 9.8 | 9.6 | 8.0 |
The authors' columns: the regraded result files in their repository, the
numbers in their paper. Ours:
bench/results/parity-5pass/, all 50 runs,
each with every request and reply, the program, the grader's output and the
cost. A median of 40 minutes and $0.39 a run; $35.79 for the 50.
What this does and does not show:
patched_verdicts.json).Parity check (every request byte for byte against their code), hosts, the changes it took, the first run's findings: NOTES.md. The result on one page: memdoor.ai/docs/team-solve.
curl -fsSL https://memdoor.ai/install.sh | bash
memdoor connect openrouter # or anthropic, openai, gemini, chatgpt
# or no key: memdoor connect local (Ollama, llama.cpp, vLLM, LM Studio; Ollama needs a 32K window, not its 4K default)
git clone --depth 1 https://github.com/guregodevo/memdoor
mkdir -p my-problem/.memdoor/workflows/team-solve && cd my-problem
cp -r ../memdoor/examples/workflows/team-solve/{steps,check-samples.sh} .memdoor/workflows/team-solve/ # these steps, not the built-in's
cp -r ../memdoor/examples/workflows/team-solve/sample/* . # task.md + tests/1.in … 5.out
memdoor workflow run team-solve --wait # exit 0 when every sample passes
Your problem: task.md and paired tests/N.in / tests/N.out. Program:
solution.py, stdin in, compared to the samples with diff. Needs Python 3
and bash. /w in memdoor tui draws the run live; /w team-solve runs it from there.
team-solve is also built in, with its own prompts (a task from a prompt,
any project): workflows/team-solve. The
copy in .memdoor/workflows runs these steps instead, the ones measured above.
The steps name no model; the /model pin answers, a local one included
(how to connect one).
A role is its prompt in steps/.
plan ──┬── ideate-a ──┬── solve ── test ── review ── repair ── verify
└── ideate-b ──┘
| step | kind | reads | writes |
|---|---|---|---|
plan |
agent | task.md |
plan.md, notes.md |
ideate-a, ideate-b |
agent, in parallel | the plan | a proposal each: the most direct provable algorithm, and one from another family |
solve |
agent | both proposals | solution.py, the decision in notes.md |
test |
command | tests/ |
test-output.log (never fails the run) |
review |
agent | the test output | repair instructions in notes.md |
repair |
agent | the instructions | a fixed solution.py, or nothing |
verify |
command | tests/ |
the verdict: a nonzero exit fails the run |
Every agent prompt: "fresh context, read the files, do not rely on chat"; the
paper credits that ledger discipline for the gain. One YAML file per step in
steps/, in the format of mario, the
engine underneath. requires: is the DAG.
Texto de github.com tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en github.com · Volver a la noticia de El Autómata