Portfolio project: a multi-agent operating model for program management
tpm-agent-os
Ambiguity in. Structure, risk, and a decision out.
A six-stage Claude pipeline that encodes recurring Staff/Principal TPM competencies as software. Feed it one ambiguous program brief; it frames it, maps risk through two independent lenses plus a judge, runs a build-vs-buy panel, writes a RAG status, makes a redirect-or-kill recommendation and generates a mentoring playbook entry. It recommends only; nothing acts on its output.
See what is measuredView the repoWhat is not built
- 12 tests passing offline (counted locally)
- Latest CI run passed (checked)
- Tests use canned fixtures (TPM_AGENT_MOCK=1)
- No human gate
- No evaluation of live output
01
The problem
Every Staff/Principal TPM role is, underneath the title, asking for the same operating model: take something ambiguous, impose structure on it fast, catch cross-team risk before it becomes an incident, make the build-vs-buy and kill/redirect calls explicitly instead of by default, and teach the next TPM how you did it.
A resume can claim any of these; it can't produce evidence of them. This project turns each competency into a stage whose output is a schema-validated artifact.
A prompt doing three jobs does each one shallower
The first risk-mapper draft asked a single call to map dependencies, analyze PHI/compliance boundaries, and synthesize a matrix at once. Splitting into two lenses plus a judge fixed it without adding a single extra unit of total work.
Optional fields disappear silently
recommended_ownerstarted asOptional[str], so an unowned risk would not mention an owner. It is now a required string that must say "unowned -- needs assignment".One model averages toward the middle
Three lenses each committed to a stance (build, buy, TCO skeptic), judged by a fourth call, surface a named dissenting view instead of an implicit "it depends."
02
How it works
A program brief goes through six stages in a fixed order. Each stage returns a typed artifact that is the schema-validated input to the next. Every run writes its artifact trail to outputs/<program-name>/.
- Program briefmarkdown, ambiguous and underspecified
- Charterscope, stakeholders and the open questions nobody has answered
- Risk matrixdependency / SPOF lens and compliance / PHI lens, then a judge
- Build-vs-buy verdictbuild, buy and TCO-skeptic stances, then a judge names the dissent
- RAG statuschecked against the org's OKRs
- Redirect-or-kill callcontinue, redirect resources, or shut down
- Playbook entrya mentoring case study generated from the run
Stages, one at a time
Stage 1 Frame
Frames the brief into a structured charter: scope, stakeholders, and the open questions nobody has answered yet, surfaced instead of papered over.
Stage 2 Map architectural risk
Two independent lenses, a dependency / SPOF (single point of failure) lens and a compliance / PHI boundary lens, run in parallel and are synthesized by a judge.
Stage 3 Build-vs-buy panel
Three committed lenses (build, buy, and a TCO skeptic) each make their case in separate calls without seeing each other (not a live debate). A judge then decides and names the dissenting view.
These are independent calls plus a judge, not a peer debate.
Stage 4 Status, redirect or kill, playbook
Synthesizes a RAG status and checks it against the org's OKRs, makes the redirect-or-kill call explicitly instead of defaulting to "continue", then generates a mentoring playbook entry from the run.
The redirect-or-kill stage produces a recommendation as JSON; nothing waits for approval or acts on it.
03
What is measured
This project has no benchmark and publishes no quality results. The only checked figure is its offline test suite, which runs against canned fixtures.
Run with TPM_AGENT_MOCK=1 pytest tests/, which short-circuits every Claude call to a fixture so the pipeline, its concurrency and every schema contract are exercised with no API key and no network.
| Test file | Tests |
|---|---|
tests/test_pipeline.py | 3 |
tests/test_tracing.py | 9 |
| Total | 12 |
Caveat These tests check wiring and schema contracts against deterministic fixtures. They say nothing about the quality of live Claude output, and the README publishes no evaluation of it. The per-file counts come from the test files, not from a README figure.
Source: tests/
04
Architecture pattern
Pattern Orchestrator-Worker (a fixed, code-sequenced pipeline). orchestrator.py calls six stages in a hard-coded order, passing each stage's typed output to the next. Two stages fan out internally and then synthesise: agents/risk_mapper_agent.py (two lenses, then a judge) and agents/decision_panel_agent.py (three lenses via asyncio.gather, then a judge). run_portfolio also runs several programs concurrently.
Only the sequencing, schema validation and artifact writing are deterministic. Every stage's content is a single structured Claude call, or a canned fixture when TPM_AGENT_MOCK=1. Judgment-critical steps run on claude-opus-5; parallel lenses and the playbook write-up run on claude-sonnet-5. Each "agent" is one prompt-and-parse call with no tools, memory or loop.
- 01 Framing agentbrief to structured charter
- 02 Risk Mapperdependency / SPOF lens and compliance / PHI lens, then a synthesis judge
- 03 Build-vs-buy panelbuild advocate, buy / borrow advocate, TCO skeptic, then a judge
- 04 Status SynthesizerRAG and OKR alignment
- 05 Redirect / Kill agentcontinue, redirect, shut down; a recommendation only
- 06 Playbook agentmentoring case study
05
Competencies demonstrated
Owning ambiguous, high-risk programs
The Framing agent turns a brief into a charter and surfaces the open questions.
Technical integration
The Risk Mapper surfaces architectural misalignment and cross-domain risk through two lenses and a judge.
Build-vs-buy judgment
Three committed stances in separate calls, then a judge that names the dissenting view.
Roadmap and OKR alignment
The Status Synthesizer produces a RAG status and checks it against the org's OKRs.
Kill and redirect calls
The redirect-or-kill stage makes the call explicitly instead of defaulting to continue.
Mentoring
Every run emits a playbook entry, so onboarding material accumulates.
06
What is not built or not verified yet
Taken from the repository's own README and its known limits. This is part of the pitch: the gaps are known, written down and not rounded away.
- limit
No human gate
The redirect-or-kill stage produces a recommendation as JSON; nothing waits for approval or acts on it.
- limit
Independent calls plus a judge
The three stances and the two risk lenses never see each other, so the debate is independent opinions plus a judge, not a peer swarm or an autonomous agent.
- limit
Each agent is one call
Each "agent" is one prompt-and-parse call with no tools, memory or loop.
- not verified
Quality of live output
The README reports no evaluation of live Claude output, and no run artifacts are committed (
outputs/holds only a placeholder). The offline tests use canned fixtures. - not built
Jira or Linear connector
The Framing agent reads a markdown file, not a real backlog. The schema layer is named as the seam for it.
- not built
Adversarial verify pass
A second instance trying to refute each flagged risk before it is trusted is listed as a next step.
- not built
Memory across runs
The Playbook agent treats every run as the first; a memory store is listed as a next step.
07
Roadmap
- DoneSix-stage pipeline with schema-validated artifacts at every stage
- DoneMock mode as a first-class path, so the full pipeline runs in CI with no key
- DoneConcurrent runs of several programs (
asyncio) - Not doneA real Jira/Linear connector
- Not doneAn adversarial verify pass on the Risk Mapper's judge output
- Not doneA memory store across runs for the Playbook agent