Rust Webserver with Plugin Architecture

I started this project mainly because I wanted to get deeper into Rust than a few toy programs — and a webserver felt like a good excuse to deal with real concurrency, real I/O, and a design that actually matters. This server is deliberately minimal at its core: it just routes requests and manages plugins. All the actual request-handling logic lives in separate plugin processes that talk to the core over a protocol I designed myself.

Why plugins as separate processes

• Modularity — each plugin is an independent application with its own lifecycle.
• Language-agnostic — a plugin just needs to read from stdin and write to stdout, so it can be written in anything, not just Rust.
• Isolation — plugins run in separate processes, so a crash in one doesn't take the server down.
• Dynamic routing — plugins declare which hosts and paths they handle, with wildcard support.
• Hot reloading — plugins can be restarted without shutting down the whole server.

Architecture

• Webserver core: an HTTP/1.1 server built on top of Hyper that receives and processes requests.
• Plugin manager: discovers, loads, starts, and manages the lifecycle of plugins.
• Plugin communication: handles bidirectional communication with plugin processes via stdin/stdout.
• Request router: matches incoming requests against plugin-declared host/path patterns and picks the right plugin.
• IO abstraction: a file-system abstraction layer used for config loading and for testing without touching the real filesystem.

src/
├── main.rs                          # Server entry point
├── lib.rs                           # Library exports
├── webserver/                       # HTTP server implementation
│   ├── webserver.rs                 # Webserver trait
│   └── http_1_server.rs             # HTTP/1.1 implementation
├── plugin/                          # Plugin management
│   ├── plugin_manager.rs            # Plugin lifecycle and routing
│   ├── plugin_entry.rs              # Plugin configuration
│   ├── plugin_config.rs             # Config parsing
│   └── running_plugin.rs            # Active plugin instance
├── plugin_communication/            # Plugin communication layer
│   ├── plugin_communicator.rs       # Communication interface
│   ├── package_handler.rs           # Binary protocol handler
│   ├── models.rs                    # Data structures
│   ├── protocols/                   # Protocol implementations
│   └── app_starter/                 # Process management
└── io/                               # File system abstraction
    ├── data_storage.rs              # Storage interface
    └── in_memory_storage.rs         # Test implementation

Each plugin ships a pluginConfig.json describing its name, startup command, protocol, timeouts, and which HTTP methods/hosts/paths it handles. When several plugins could match the same request, the router falls back to a specificity algorithm — an exact host beats a wildcard subdomain beats *, and the same logic applies to paths, so /api/users wins over /api/* wins over /*.

The protocol itself is a length-prefixed, binary-framed JSON exchange over stdin/stdout: a 4-byte length prefix followed by the JSON payload, with message types for handshakes, requests, responses, and error notifications. Two example plugins — one serving static files, one handling email subscriptions — exist to prove the protocol actually works end to end.

What went well, what didn't

Designing that protocol was the most interesting part of the whole project. I had to think through framing, handshakes, error handling, and how specific a plugin's routing rules are allowed to be — all before writing a single line of the actual server logic. Getting the specificity rules right took a few iterations, but the result is a routing system I trust.

The hardest part was honestly just Rust itself at that stage: getting the ownership model to cooperate with a design built around long-lived processes and inter-process communication meant a lot of fighting the borrow checker before things clicked. In hindsight, that fight is exactly why I stuck with Rust for this — once it compiles, entire categories of bugs just aren't there anymore.