hydra docs
hydra wraps process-compose. One baseline stack runs in your main checkout. Each git worktree starts only the services it changed and points everything else at the baseline.
It exists for working on several branches at once, often with a coding agent in each worktree. Five copies of a full stack cost five times the memory and fight over the same ports. With hydra, a frontend-only branch runs one extra process.
Install
hydra needs git and process-compose. The Homebrew formula installs process-compose for you.
$ brew tap santiagotoscanini/tap $ brew install --HEAD santiagotoscanini/tap/hydra $ brew upgrade --fetch-HEAD santiagotoscanini/tap/hydra # later, to update
To build from source, clone the repository and run make link. It puts that checkout's build on your PATH as hydra; make unlink removes it.
Quick start
1. Describe your services
Add x-hydra keys to the process-compose.yaml you already have: a block at the top level, and one on each process that a worktree may run. process-compose ignores keys it doesn't know, so the file still works without hydra.
version: "0.5"
x-hydra:
base_ref: main
slot_stride: 200
reserved_ports: [5432]
global_paths: [pnpm-lock.yaml]
processes:
db:
command: docker compose up postgres
x-hydra:
shared: true
paths: ["db/migrations/**"]
deps:
command: pnpm install
x-hydra:
follow: true
api:
command: go run ./services/api
depends_on:
db: { condition: process_healthy }
x-hydra:
port: 8080
port_env: API_PORT
paths: ["services/api/**", "packages/shared/**", "!**/*_test.go"]
worker:
command: go run ./services/worker
depends_on:
db: { condition: process_healthy }
x-hydra:
paths: ["services/worker/**", "packages/shared/**"]
url: "jobs-{slot}"
baseline_url: jobs
upstreams:
API_URL: api
web:
command: pnpm --filter web dev
depends_on:
deps: { condition: process_completed_successfully }
api: { condition: process_started }
x-hydra:
port: 3000
paths: ["apps/web/**", "packages/shared/**"]
upstreams:
API_URL: api
JOBS_QUEUE: worker
NEXT_PUBLIC_SITE_URL: web
Each service has to read its port and its upstreams from the environment variables named here. hydra sets those variables; it doesn't proxy traffic.
2. Start the baseline
In the main checkout, hydra runs the whole file, exactly as process-compose would.
$ cd ~/code/shop $ hydra
3. Start a worktree
$ git worktree add ../shop-checkout -b feat/checkout $ cd ../shop-checkout $ hydra plan # what would run here, and why $ hydra # run it
A fresh worktree has changed nothing, so nothing runs in it yet. Edit a file under apps/web and web starts on port 3200, pointed at the baseline API on 8080.
worktree, slot 1: ~/code/shop-checkout SERVICE RUNS PORT WHY api baseline 8080 unchanged db baseline - shared deps here - needed by web web here 3200 changed: apps/web/app/page.tsx worker baseline - unchanged wiring: web API_URL -> api (baseline) http://127.0.0.1:8080 default: api not running here web JOBS_QUEUE -> worker (baseline) jobs default: worker not running here web NEXT_PUBLIC_SITE_URL -> web (local) http://127.0.0.1:3200 default: web running here
.process-compose/ and .hydra.local.yaml to your .gitignore. If your process-compose.yaml is itself git-ignored, a worktree without one uses the main checkout's.Concepts
| Term | Meaning |
|---|---|
| Baseline | The stack running in the main checkout. It runs every service on its normal port. |
| Worktree | Any other checkout made with git worktree add. It runs only what it changed. |
| Slot | A number per worktree. The baseline is slot 0. Slot N listens on each port plus N × slot_stride. |
| Upstream | An environment variable of one service that says where another service lives. hydra fills it with the local copy's address or the baseline's. |
| Shared | A service that only ever runs in the baseline, such as a database or a tunnel. |
| Follow | A service that runs in a worktree whenever a local service depends on it, such as pnpm install. |
| Stand-in | A placeholder process for a baseline service that a local one depends on. It runs nothing and only waits for the baseline's copy to be ready. |
How it works
hydra runs no services itself. It works out what should run in this checkout, writes an ordinary process-compose file for it, and hands that file to process-compose. In a worktree, hydra does this:
- Lists the files the worktree changed and picks the services to run here.
- Takes a slot for the worktree, which it keeps across runs.
- Writes
.process-compose/hydra.yaml: those services only, on the slot's ports, with each upstream variable set. - Starts process-compose on that file, on a unix socket that belongs to this worktree.
Because of the socket, hydra down, hydra attach and hydra process … in a worktree reach that worktree's instance and never the baseline's.
What counts as changed
A worktree changed a service when a file that differs from where its branch left main matches one of the service's paths. Committed, staged, unstaged and new files all count. Git-ignored files don't.
$ git diff --name-only $(git merge-base HEAD main origin/main) # committed, staged, unstaged $ git ls-files --others --exclude-standard # new files
The remote-tracking branch counts as well as the local one, because a local main is often behind. A branch cut from a freshly fetched origin/main forked there, and the commits merged in between aren't its work.
Watching
The generated file has one extra process, hydra-watch. It starts with the services and stops with them. Every 2 seconds it lists the changed files again. When the set of services to run differs, it rewrites the file and calls process-compose project update, which touches only the processes that differ.
| Transition | What happens |
|---|---|
| Unchanged → changed | The service starts in the worktree. Local services that point at it restart with its address. |
| Changed → unchanged | The service stops. Local services that pointed at it restart on the baseline's address. |
Those are the only two things the watcher does. Further edits to a service that already runs here are left to the service's own reload, such as next dev, air or process-compose's watch.
The watcher polls git instead of watching the file system. Only git can answer "does this differ from main": a revert has to stop the service, and ignored files such as node_modules must not start one. It reads git without taking the index lock, so it doesn't collide with an agent committing in the same worktree.
Stacked branches
Worktrees can be stacked: b1 cut from main, b2 from b1, b3 from b2, each in its own worktree. Run hydra in b3 and it walks the chain down to main.
| Layer | Compared with | Its diff holds | Runs |
|---|---|---|---|
b1 | where it left main | b1's work | api, from b1's worktree |
b2 | where it left b1 | b2's work, not b1's | worker, from b2's worktree |
b3 | where it left b2 | b3's work only | web, from b3's worktree |
- A service runs from the topmost layer that changed it, in that layer's worktree. An agent editing b2's worktree reaches the worker through the worker's own reload.
- Everything runs in b3's process-compose project: one slot, one set of logs. b1 and b2 don't need to run
hydrathemselves. - The watcher polls every worktree in the stack, so an edit in b1's worktree starts or stops services in b3's project.
- Follow services run once per worktree that has a local service, so each one installs into its own
node_modules.
How a parent is found
parent:in the worktree's.hydra.local.yaml. The base branch there means "not stacked".- The branch it was created from, read from its reflog, if that branch is checked out in a worktree and still shares commits with it past
main. This holds after the parent gets new commits. - Otherwise, the deepest checked-out branch whose tip it contains.
Branches that only share history are not parents. Two siblings cut from the same commit look, from ancestry alone, the same as a parent that moved on, so hydra doesn't guess. A branch made with git checkout -b b2 while on b1 records "Created from HEAD"; once b1 gets new commits, give b2 a parent: b1.
hydra plan prints the stack it found and why each service runs where it does.
The generated file
hydra render prints what hydra hands to process-compose. Compared with your file:
- Only the services that run here are kept, with the
x-hydrakeys removed. - Each service's
environmentgains its port variable, its upstream variables, andHYDRA_SERVICE,HYDRA_SLOTandHYDRA_UPSTREAMS. working_dir,log_locationandwatchpaths become absolute, pointing into the worktree the service runs from.- Readiness and liveness probes on the baseline port move to the slot's port. Otherwise a worktree's probe would hit the baseline's copy and pass at once.
- A stand-in replaces every baseline process that a local one depends on. It copies only the readiness probe or ready log line, so
process_healthywaits for the baseline's database instead of starting a second one. Stopping the worktree never runs the baseline's shutdown commands. - A
hydra-prepare-<service>step runs before each service that hasprepare.
hydra relies on the process environment winning over .env files. That holds for Next.js and for godotenv; check it for other loaders.
Slots and ports
Slots live in hydra-slots.json inside the repository's shared .git directory, guarded by a file lock. A worktree keeps its slot across runs, and the entry of a deleted worktree is dropped.
When the config loads, hydra works out every service's port for every slot up to max_slots and fails if two of them meet or if one lands on a reserved port. Pick a slot_stride larger than the distance between your closest base ports, and list the ports of shared infrastructure in reserved_ports.
$ hydra slots # which worktree has which slot $ hydra slots free # free this worktree's slot
Stack options
The top-level x-hydra block. A file without it runs unchanged.
| Key | Default | Description |
|---|---|---|
base_refstring | main | The branch a worktree is compared with. If it tracks a remote branch, that one counts too. |
slot_strideinteger | 1000 | Port offset per slot. Slot N listens on port + N × slot_stride. |
max_slotsinteger | 20 | How many worktrees can hold a slot. Port collisions are checked for all of them. |
reserved_portslist of integers | [] | Ports used by shared services or anything else on the machine. No slot may land on one. |
hoststring | 127.0.0.1 | The host used to build upstream addresses. |
port_envstring | PORT | The variable a service reads its port from, unless the process sets its own. |
global_pathslist of globs | [] | Files that count as a change to every service that declares paths, such as a lockfile. |
Process options
The x-hydra block on a process. A process without one is treated as shared and only runs in the baseline.
| Key | Default | Description |
|---|---|---|
portinteger | none | The port the baseline's copy listens on. Leave it out for services that don't listen, such as workers. |
port_envstring | stack's | The variable this service reads its port from. |
pathslist of globs | [] | Files that belong to the service, relative to the repository root. A change to one makes the service run in the worktree. |
upstreamsmap | {} | Environment variable → the service it points at. Write API_URL: api, or API_PORT: { service: api, value: "{port}" } to change what the variable receives. A service may point at itself. |
urlstring | http://{host}:{port} | What an upstream variable pointing at this service receives. It doesn't have to be a URL. |
baseline_urlstring | url | Replaces url for the baseline's copy, for values that don't come from a port. |
sharedboolean | false | Always use the baseline's copy. A change to its paths is reported as a warning. |
followboolean | false | Run in a worktree whenever a local service depends on it. |
preparestring | none | A shell command run once before the service starts in a worktree, for example to create the queue or namespace named by url. |
Values that aren't URLs
"Where a service lives" is not always a port. A worker that reads jobs from a queue lives at a queue name. Give it a url built from the slot, and whoever enqueues jobs gets the right name:
worker:
x-hydra:
paths: ["services/worker/**"]
url: "jobs-{slot}" # slot 2 reads from jobs-2
baseline_url: jobs # the baseline reads from jobs
prepare: "queue create {value}"
web:
x-hydra:
upstreams:
JOBS_QUEUE: worker # jobs-2 if the worker runs here, else jobs
Placeholders and globs
Placeholders
Available in url, baseline_url, prepare and an upstream's value.
| Placeholder | Becomes |
|---|---|
{host} | The stack's host. |
{port} | The service's port in this slot. |
{slot} | The slot number. 0 for the baseline. |
{value} | In prepare only: the service's own rendered url. |
Globs
Patterns in paths and global_paths match paths relative to the repository root, with / as the separator.
| Pattern | Matches |
|---|---|
* | Anything within one path segment. |
** | Any number of segments. |
? | One character. |
{a,b} | Either alternative. Not nested. |
dir/ | The same as dir/**. |
!pattern | Excludes matching files, wherever it appears in the list. |
Per-worktree settings
A git-ignored .hydra.local.yaml at the root of a worktree adjusts what that worktree runs. The watcher re-reads it, and the stack file, on every tick, so edits apply without a restart.
parent: b1 # the branch this worktree is stacked on
with: [web] # also run these here, changed or not
without: [worker] # never run these here
routes:
web: { api: baseline } # web's upstreams that point at api use the baseline's
What a worktree runs
- Services whose
pathsmatch a changed file, orglobal_pathsfor services that declare paths. - Services named in
with, or withhydra up NAME. - Services with a route override, plus the target of a
localoverride. - Follow services that one of those depends on.
- Minus shared services and anything in
without.
Routing
By default an upstream points at the worktree's copy if that service runs in the worktree, and at the baseline's otherwise. A route overrides that for one service and one target:
$ hydra plan --route web.api=baseline # web here, but against the baseline API $ hydra plan --route web.api=local # also starts api here
The --route flag previews a route with plan and render. To run with one, put it under routes in .hydra.local.yaml.
Two frontends that share a backend each have their own upstream, so one can use the local API while the other stays on the baseline.
Command line
hydra stands in for process-compose. Arguments it doesn't know pass through.
| Command | What it does |
|---|---|
hydra | Starts this checkout with the TUI. Everything in the main checkout, only what changed in a worktree. |
hydra up -D | Starts it in the background. |
hydra up web | Also runs web here, changed or not. The watcher keeps running. |
hydra attach | Opens the TUI of this checkout's running instance. |
hydra down | Stops this checkout's instance only. |
hydra process … | Any other process-compose command, sent to this checkout's instance. |
hydra plan | Shows what this checkout runs, on which ports, wired to what, and why. |
hydra render [-o FILE] | Prints or writes the generated process-compose file. |
hydra slots | Lists worktree slots. |
hydra slots free [PATH] | Frees a worktree's slot. The default is this worktree. |
Flags for plan and render
| Flag | Meaning |
|---|---|
-f FILE | The stack file. The default is process-compose.yaml in the checkout. |
--root DIR | The checkout to plan for. The default is the one containing the current directory. |
--slot N | Plan as if this were slot N. 0 is the baseline. |
--changed PATHS | Comma-separated changed paths, instead of asking git. |
--with SVC | Also run this service here. Repeatable. |
--without SVC | Never run this service here. Repeatable. |
--route S.U=T | Point service S's upstream U at T, which is baseline or local. |
Set HYDRA_DRY_RUN=1 to print the process-compose command instead of running it.
Variables every local service receives
| Variable | Value |
|---|---|
PORT or port_env | The service's port in this slot, if it has a port. |
| Each upstream variable | The address of the local or baseline copy of its target. |
HYDRA_SERVICE | The service's name. |
HYDRA_SLOT | The slot number. |
HYDRA_UPSTREAMS | The names of its upstream variables, comma-separated. |
Limits
- The database is shared. A branch that adds a migration still uses the baseline's database, and hydra warns about it. A database per worktree is planned.
- Scheduled jobs. A worker that sets up its own schedules on startup does so in every worktree it runs in.
- Hard-coded ports. A service has to read its port and its upstreams from the environment. A port written into code or a dev script needs a small patch first.
- Paths are written by hand. They aren't derived from a dependency graph yet, so a shared package has to be listed on each service that imports it.
- Tested with process-compose v1.124 on macOS.
More: the source and the design notes, which cover open problems and decisions.