If you've spent any time in the AI engineering space lately, you've probably watched the pendulum swing wildly. First, everything was Python scripts and Jupyter notebooks. Then, as those systems crumbled under production requirements, the call went out: "Rewrite the orchestrator in Rust!"
I fell into this trap myself. I wanted the safety, the performance, and the strict type guarantees. But if you've tried to build a full AI orchestration framework in Rust, you've likely hit the same wall I did: you spend days fighting the borrow checker over JSON schema shapes that change weekly, only to realize the runtime performance gains are completely eclipsed by a two-second wait for an LLM response.
The reality of production AI systems is that they have two fundamentally different needs. Orchestration is heavily I/O-bound, demanding flexibility and rapid iteration. Validation and console control, on the other hand, require strict guarantees and zero-dependency reliability.
The solution isn't to pick one language. It's to build a deliberate seam between them. Here is how I structure the Python/Rust boundary in my production systems.
The Python Orchestrator: Embracing the I/O Wait
In my orchestration framework, the core execution model is simple: FastAPI receives events, Celery queues the tasks, and a workflow runtime walks a directed acyclic graph (DAG) of nodes.
At the end of the day, orchestration is overwhelmingly I/O-bound. Model calls, database queries, and network latency dominate the execution time. The microseconds you save by rewriting a router node in Rust are mathematically meaningless when you are waiting 2,000 milliseconds for Claude to return a completion.
Python stays Python because flexibility wins here. By using pydantic-ai for model interactions and a shared mutable TaskContext ledger for nodes to read and write outputs, I can rapidly adapt to new prompt strategies and tool schemas.
The orchestrator's job is to manage latency and state, not to be a CPU powerhouse.
The Rust Validation Spine
While Python handles the messy reality of LLM outputs, I rely on Rust for the system's structural integrity. I built a standalone Rust validation engine (which I call mev) to serve as the strict validation spine.
Instead of writing brittle Python scripts to check content constraints, mev enforces the rules at build and commit time. It parses Markdown files, validates YAML frontmatter against a strict schema (OKF), ensures that cross-references in the system graph actually exist, and verifies that anchor links in documentation match the rendered output.
Rust shines here. It provides deterministic, high-performance graph traversals and memory-safe parsing. The validation spine guarantees that by the time the Python orchestrator reads configuration or content, it is structurally flawless.
The Two-Surface Architecture for Control Planes
The most critical application of this Python/Rust seam is in the control plane—the CLI tool used to monitor and manage the agents. I built my control plane, Bastion, using a strict "Two-Surface Architecture."
Bastion is a Rust binary that operates on two entirely decoupled tracks:
- Workflow Observability: Bastion acts as a read-only observer of the Python orchestrator. It polls the orchestrator's PostgreSQL
eventstable to reconstruct the live DAG and stream token usage, errors, and node status. It never writes to this database, ensuring the orchestrator's state remains pure. - Process and Session Control: For actually managing the agent sessions (attaching, sending commands, killing processes), Bastion shells out directly to
tmux.
Why decouple them so strictly? Because the session management surface must always work. By keeping the tmux session control entirely free of database dependencies, I guarantee that even if the Python orchestrator crashes or the Postgres pool is exhausted, I can still attach to the agent's terminal and see what went wrong.
Finding Your Seam
When building your own agentic systems, resist the urge to homogenize your stack.
Let Python do what it does best: orchestrating I/O-bound model calls and rapidly iterating on data shapes. Then, draw a hard boundary. Use Rust where you need absolute structural guarantees, fast deterministic graph validation, and robust, dependency-free console tools.
By decoupling workflow observability from session control, and separating execution from validation, you create a system that is both agile enough to adapt to new models and resilient enough to survive them.
Stop rewriting your orchestration in Rust. Build your console and your validation spine in Rust, and let the orchestrator wait on the network in peace.