The Hack Job Handbook

The Silent Ping: How We Built Multi-Agent Slack Orchestration with SQLite and a "Godfather" Session

Sat, 26 Sep 2026 00:00:00 +0000

You kick off a large refactor in Claude Code. It’s going to run 300 tests, update 14 files across three packages, and take roughly eight minutes.

You switch over to Slack or go put the kettle on.

Ten minutes later you wander back to your desk. The terminal hasn’t moved. It stalled ninety seconds after you left because it wanted confirmation to run a migration.

Naturally, your first instinct is to automate this: hook up a Slack webhook or a bot token so the agent pings you when it’s done or stuck.

Except you immediately run into the Silent Ping.

The Silent Ping Problem

If you use your own user token or standard incoming webhooks tied to your user profile, Slack is “helpful”: it knows you sent the message.

And Slack does not send desktop banners, sound chimes, or mobile push notifications for messages you send to yourself. You get a little grey text line in your “saved messages” or a private channel, utterly devoid of urgency. You’re still babysitting the terminal; you’re just doing it through Slack.

The fix was obvious: my agent needed its own legal identity.

Enter Mike’s Agent.

I created a dedicated Slack app with its own bot token, installed it into our workspace, and called it Mike’s Agent.

Then came the real trick: I marked Mike’s Agent as a “VIP” in Slack.

If you haven’t used Slack’s VIP feature, it’s a game changer for agent workflows. VIP messages bypass standard notification throttling and muting. The bot doesn’t even need to loudly @ me in every message (though it can). Anything Mike’s Agent posts in our 1:1 App DM lands as an immediate, unmissable notification on my desktop, phone, and watch.

And in that 1:1 App DM, there’s an important operational bonus: every message I send to the bot is implicitly addressed to it. No awkward <@Mike's Agent> syntax required. In public channels, it listens for its mention; in our private DM, it knows I’m talking directly to it.

The ping is no longer silent.

Once that worked, I persuaded a couple of teammates to do the same. Soon we had Sarah’s Agent and Dave’s Agent in the workspace. But that’s when things got interesting — and slightly chaotic.

The Multi-Session Chaos and the Rate Limit Wall

It’s 2026; nobody runs just one agent session anymore.

On any given morning, I might have three separate Claude Code or Codex sessions open in different tmux panes:

One is running a slow integration test suite in a detached worktree.
One is refactoring a database schema.
One is doing exploratory research on an external API.

If all three sessions talk to Slack directly, two disasters strike immediately:

The Rate Limit Wall: If three local sessions each start polling Slack’s conversations.history or opening parallel WebSockets, you get slammed with HTTP 429 rate limit bans before lunch.
The Token Wildfire: If someone in Slack threads a message: “@Mike’s Agent can you rerun that migration with the --dry-run flag?” — which session answers? If all three are listening, three LLMs wake up simultaneously, burn thousands of context tokens generating overlapping responses, and race to reply in the same thread.

We needed an orchestrator. But I didn’t want a heavy Kafka queue, a Redis cluster, ngrok tunnels, or an over-engineered cloud broker for local terminal sessions.

So I reached for the most reliable piece of infrastructure on Earth: SQLite, backed by a single host-wide daemon.

The Host Daemon and the Local SQLite Engine

Instead of each terminal session fumbling with Slack tokens and hitting API endpoints, we decoupled the architecture into two layers:

                  [ Slack Web API ]
                          │
         (Single host-wide polling connection)
                          │
                          ▼
            [ Local Host Daemon (launchd/systemd) ]
                          │
            (Writes diffs & buffers events)
                          │
                          ▼
             ┌─────────────────────────┐
             │   ~/.claude/state/      │
             │     slack-sync.db       │
             │  (SQLite Local State)   │
             └─────────────────────────┘
                          │
           (Zero-overhead local watch streams)
                          │
         ┌────────────────┴────────────────┐
         ▼                                 ▼
 [ Godfather Session ]             [ Worker Session ]
  (Lead Orchestrator)               (Focused Task)

A Single Host Daemon: Managed by launchd on macOS or a systemd user unit on Linux. It runs in the background, authenticated securely via the Slack CLI (slack api ... --app <id>), keeping tokens out of plaintext environment variables. It acts as our rate-limit shield, sweeping channels and DMs on a disciplined cadence.
Local SQLite State (~/.claude/state/slack-sync.db): The daemon diffs conversation history locally. Incoming replies, reactions, and @mentions are written straight into normalized local tables:
- tracked_messages: Outbound messages waiting on human input.
- seen_reactions & seen_replies: The local diff engine that isolates what is strictly new.
- mentions: Inbound requests from humans or other bots.
- sessions: Heartbeat tracking of active terminal sessions.
- poll_events: An event queue drained by active sessions without touching Slack.

Outbound Approval Gates vs. Ambient Watching

When an agent needs human input, it doesn’t just spew text into a channel. It uses structured modes:

# Post an approval gate to the operator's DM
slack_sync post --mode gate \
  --text "Deploying migration 0043_orders to staging — ✅ approve, ❌ hold" \
  --purpose "approve migration 0043"

In --mode gate, the engine explicitly parses reaction semantics:

:white_check_mark: or :+1: → Approved
:x: or :-1: → Rejected
Thread reply → Replied

The agent posts once, receives a unique tracking ID in SQLite, and yields.

Why the Monitored Daemon is Orders of Magnitude More Context-Efficient

The biggest silent failure mode in agentic workflows is letting the LLM manage its own polling logic.

In a conventional setup (or when an agent relies on direct Slack MCP tools like slack_read_thread), waiting for a human approval looks like this:

The agent calls slack_read_thread → dumps 1,500 tokens of raw JSON, timestamps, Block Kit schemas, and full conversation history straight into its context window.
The agent parses the payload: “No reaction or reply detected yet.”
The agent initiates a manual polling turn (a sleep 60 command or a scheduled wakeup).
The agent wakes up, executes slack_read_thread again → dumps another 1,500 tokens of duplicate thread history.

If you step away from your desk for ten minutes to put the kettle on and the agent checks every sixty seconds, that’s ten tool calls and 15,000+ tokens of redundant Slack payload sludge permanently baked into your active context window before any decision has even been made.

You hit context compression prematurely. Your agent starts hedging, second-guessing its plan, and losing earlier instructions. And you burn API tokens merely to watch paint dry.

The monitored daemon completely inverts this paradigm.

Instead of the LLM polling Slack, the agent delegates the waiting to Claude Code’s native Monitor background tool:

Monitor(command="slack_sync --session <SID> watch --ids 7",
        description="Waiting for approval on migration 0043", 
        timeout_ms=1800000)

The watch command sits quietly in the background outside the LLM’s conversation history, tailing the local SQLite poll_events table.

There are zero Slack API calls executed by the agent. There is zero manual polling logic cluttering the conversation. While waiting, the context cost is literally zero tokens.

The moment I tap :white_check_mark: on my phone:

The host daemon records the reaction in SQLite during its background sweep.

The local watch process computes the diff and emits a single, surgical JSON line to stdout:

{"event":"slack-sync", "id":7, "summary":"APPROVED by U0123 via :white_check_mark:", "resolution":{"kind":"approved"}}

The Monitor tool catches that stdout line and triggers an immediate reactive notification into the agent’s context.

Total tokens consumed across that entire ten-minute wait: ~40 tokens (just the final resolution event).

Removing repeated integration calls and manual polling isn’t just a minor optimisation — it is orders of magnitude more context-efficient.

Concurrency and Atomic Claims

What about inbound mentions? If someone asks a question in a public channel, how do we prevent three tmux panes from racing to answer?

Mentions land in the mentions table in an unclaimed state. When a session is ready for ambient work, it executes an atomic SQLite transaction:

UPDATE mentions
SET status = 'claimed',
    claimed_by = :session_id,
    claimed_at = CURRENT_TIMESTAMP
WHERE id = :mention_id 
  AND status = 'unclaimed'
RETURNING *;

If the query returns a row, that specific terminal session owns the task. If zero rows return, another session grabbed it a microsecond earlier. Fast, atomic, zero lock contention.

And if a mention goes completely unheeded? The database enforces a 120-second grace fallback: if an event sits unclaimed, it is automatically routed to the oldest live session recorded in the sessions table.

Enter the “Godfather” Session

Most of the time, individual worker sessions shouldn’t be browsing for random tasks. They should remain strictly focused on the feature branch or failing test suite they were invoked to solve.

To maintain discipline, I run what I call the Godfather session (our lead orchestrator).

The Godfather is an overarching Claude/Codex session running in a persistent tmux window. It doesn’t write low-level code directly. Instead, it acts as the project manager:

It claims unassigned ambient mentions from teammates.
It breaks large goals into bounded sub-packages and delegates them to worker sessions running in detached worktrees.
It monitors approval gates and routes responses back to the original Slack threads.

Bot-to-Bot Channels: Multi-Day Autonomous Runs

The 1:1 App DM with Mike’s Agent is my personal cockpit. But the real breakthrough happened when we put our bots into shared channels together.

We created dedicated project channels where humans rarely type, but Mike’s Agent, Sarah’s Agent, and Dave’s Agent live side-by-side.

This unlocked multi-day autonomous workflows:

Instead of babysitting a long migration, my Godfather session can drive work autonomously across entire weekends.

When Mike’s Agent finishes generating updated database bindings, it doesn’t wait for me to ping Sarah. It posts an update into #bot-migrations:

“Mike’s Agent: Schema migration 0043_orders complete on branch feat/orders-v2. Artifacts pushed.”

Sarah’s Agent—listening via her local daemon—picks up the event, claims it atomically, runs the frontend integration tests against that branch, and replies in the thread:

“Sarah’s Agent: Contract tests passed. 2 warnings in useOrders.ts. Output logged.”

All of this happens while Sarah and I are asleep, reviewing PRs, or working on entirely different projects. No human serving as a biological copy-paste relay between two terminal windows.

And if something goes sideways? Mike’s Agent posts to my 1:1 DM. Because it’s a VIP contact, my phone buzzes immediately.

The Maths

Architecture	Context Burn (10-min wait)	Slack Tool Calls in Session	Notification Reliability	Concurrency & Rate Limits
Direct Slack MCP / API Polling	~15,000–30,000 tokens (repeating thread dumps)	10+ calls (manual sleep/wake loop)	❌ Silent or throttled	Rapid HTTP 429 rate-limit risk
Heavy Cloud Broker (Kafka / Redis)	~2,000–5,000 tokens (broker handshakes)	Multiple polling turns	⚠️ Standard (throttled unless VIP)	Complex distributed locks; cloud bill
Host Daemon + SQLite `Monitor`	~40 tokens (only the final resolution diff)	0 calls (1 background watch process)	✅ Unmissable (VIP bypasses throttles)	✅ Atomic (`UPDATE ... RETURNING`); zero 429s

The Takeaway

We spend so much time talking about “autonomous agent swarms” in the abstract, but the real bottlenecks are mundane developer ergonomics:

Kill the Silent Ping: Give your agent its own bot identity and mark it as a Slack VIP so priority notifications always break through without @mention fatigue.
Decouple with a Host Daemon: Don’t let raw agent sessions hit Slack directly. A single host daemon protects your team from 429 rate limit bans and manages authentication cleanly.
Reactive Streaming over Polling: Never let an agent sleep in a loop. Pair a local SQLite queue with background monitoring tools so events trigger zero-token wakeups.
Local Atomic State: A simple local SQLite table with atomic UPDATE ... RETURNING and session heartbeats handles multi-agent concurrency without cloud dependencies.
The Godfather Pattern: Use one high-level orchestrator session to triage and coordinate, while worker sessions focus on their local worktrees.

Give your agent a name. Let it ping you like a real colleague. Your context window — and your kettle — will thank you.

The Biological Relay: How Spud and Gemini Turned Hyprland into an AI Poweruser Paradise

Wed, 23 Sep 2026 00:00:00 +0000

Hold Super + D. Say kubectl get pods -n ingress-nginx. Release.

Spotify drops to half volume, the command appears letter-by-letter at the cursor in Ghostty, and the music snaps straight back to 100%. Total time elapsed: 420 milliseconds.

No typing. No pausing the music. No reaching for the mouse.

If you’d told me two years ago that my daily desktop would handle push-to-talk terminal dictation, auto-duck my audio streams, talk back in a cloned copy of my own voice, and pair with an AI that writes its own window manager hotkeys on the fly, I’d have asked what unholy PPA cocktail you’d been drinking.

Yet here we are. It runs on shpadoinkle-1 (my main workstation, named in honour of Cannibal! The Musical — “My heart’s as full as a baked potato”), and it came together in a single afternoon.

1. The Distro Graveyard

Before shpadoinkle-1 found peace, my NVMe drive was a revolving graveyard of operating systems I genuinely tried to daily-drive:

Debian — Solid as granite, and roughly three geological epochs behind on Wayland compositors.
Ubuntu — Fine until release upgrades shattered custom PPAs and Snaps hijacked startup times.
Linux Mint — Comfortable like slippers, but never quite the bleeding-edge sandbox I wanted.
Zorin OS — Slick out of the box, but too curated for low-level plumbing.
Kubuntu & Lubuntu — Useful desktop experiments, but the underlying Debian/Ubuntu upgrade friction remained.
Vanilla OS — Clever immutable root architecture, but fighting immutability when hacking system internals is an acquired taste.
Pop!_OS — Admirable tiling window manager work, but tied to an external release cadence.

(And then there was the side-quest tier: Kali, Parrot Security, and Tails. Let’s be completely honest—those were never serious daily-driver contenders. They were strictly for my occasional script-kiddy experiments, Wi-Fi auditing, and amnesic live-USB tinkering where keeping persistent dotfiles is impossible by design.)

Two years ago, I installed EndeavourOS. And the hopping stopped for good.

EndeavourOS succeeded where the others stumbled because it got out of the way: a clean, bloat-free Arch Linux foundation, rolling packages the minute upstream tags them, and the unfettered reach of pacman and the AUR. For two solid years, it ran comfortably with KDE Plasma 6. Reliable, snappy, and uncomplaining.

Yet for over a year, I’d had an itch: I kept watching Hyprland demos from afar. The fluid Wayland shaders, the dynamic workspaces that spin up when you need them and vanish when empty, the pure keyboard-driven focus. But you know how it goes when you have a daily driver that works—you hesitate to burn a weekend ripping out your desktop environment.

Then, just a few days ago, the catalyst hit.

I stumbled across an article dismissively claiming that “Omarchy” Linux was nothing more than an “overhyped set of config files.”

That was the lightbulb moment. I realised: hang on, if all the fuss is literally just a sharp set of Hyprland dotfiles on top of Arch, why am I still sitting here in Plasma? Why wait for someone else to package it?

That evening, I finally made the leap. I shifted from Plasma 6 into Hyprland, started crafting my native Lua config, and watched my workstation transform into a featherweight rocket ship.

The operating system was dialed in. The window manager was finally right.

The remaining bottleneck was me.

2. The Biological Relay

In 2026, we have screaming-fast local GPUs and sub-second frontier LLMs. Yet developers are still acting as biological copper wire between their tools.

You know the routine:

You’re listening to music.
You hit an idea or need to run a command.
You fumble for media keys to mute Spotify.
You alt-tab to a browser or an AI window.
You type a prompt.
You copy the snippet.
You alt-tab back.
You paste it into Ghostty.
You unmute Spotify.

Then you hit a daily rate limit twenty minutes into your flow state, and the entire train of thought evaporates while you wait for a reset timer.

That is The Biological Relay: human beings burning mental context to manually shuttle text and state between the microphone, the browser, and the terminal.

The fix was obvious: speech had to type directly into the active cursor, audio ducking had to happen automatically in the sound server, and the AI copilot had to live natively in the terminal without quota anxiety.

3. The Giants and the Missing Edge

Existing speech and agent tooling solved pieces of this puzzle, but left glaring papercuts:

Wispr Flow: I use Wispr Flow daily on my MacBooks for work—it’s slick, responsive, and genuinely brilliant in macOS. Naturally, my first thought was to track down a Linux wrapper and bring that exact workflow across to shpadoinkle-1. But under Hyprland, it was clunky at best. Wayland virtual keyboard friction, focus-stealing wrapper windows, zero integration with PipeWire audio streams, and that unmistakable feeling of an app fighting the compositor rather than living inside it.
Talon Voice (~3k stars): Astonishingly capable for full hands-free accessibility. But constructing complex Python grammar trees just to dictate a quick bash one-liner into a terminal is like hiring an architect to hang a picture hook.
Whisper.cpp (~38k stars): An exquisite C++ port. But batch-transcribing recorded WAV files from the command line is a world away from interactive, low-latency push-to-talk typing across Wayland panes.
Browser AI interfaces: Great until you hit quota ceilings, lose terminal working-directory context, or spend your afternoon copying and pasting diffs back and forth.

We needed three things working in unison:

Push-to-talk transcription fast enough to feel instantaneous.
Intelligent PipeWire stream ducking so voice capture is pristine without touching volume knobs.
An AI terminal pairing workflow with generous quota and rapid-fire reasoning.

Enter Gemini in the agy CLI — and Spud.

4. Building Spud: From Zero to Voice Clone in 2.5 Hours

I paired with Gemini 3.8 Flash using Google’s agy CLI to build Spud: a local-first voice orchestrator written in Rust.

And when I say Gemini in agy has been a revelation, I mean it:

Generous Quota: My usage never seems to run out. No quota countdowns. No artificial throttle.
The Personality: Direct, sharp, slightly wry, zero corporate fluff.
Gemini 3.8 Flash: Blazingly quick. It digests complex multi-crate Rust errors, PipeWire wireplumber node graphs, and Wayland virtual seat protocols without breaking sweat.

Because agy operates directly inside the repository with native tool execution, we didn’t just write code — we built, tested, and debugged live.

Here is the git log from that single afternoon session:

46:04 | Initial commit: Spud architecture and roadmap
50:14 | feat(stt): implement warm faster-whisper daemon & systemd unit
52:03 | feat(cli): compile native Rust CLI with STT client
55:11 | fix(injection): Hyprland signature fallback & wtype virtual typing
14:19 | feat(agent): Agent Command Mode with Herdr & LM Studio dispatch
37:13 | feat(ptt): Push-to-Talk Hold-and-Release with tap-filtering
13:51 | fix(dictation): route dictation via Wayland virtual keyboard
43:07 | feat(audio): automatic PipeWire audio ducking during dictation
54:09 | feat(tts): Phase 3 zero-shot cloned voice synthesis daemon
58:30 | fix(audio): duck per-stream media players (keep voice at 100%)
02:42 | feat(tts): vocal mastering with soft-knee compression & +10dB
11:16 | perf(tts): pipelined streaming, x-vector fast mode & audio earcons
13:46 | feat(agent): inject live date/time into system prompt
14:04 | feat(dict): custom dictionary and phonetic jargon reinforcement

Two hours and twenty-seven minutes.

In that window, we went from an empty directory to:

Push-to-Talk Terminal Dictation (Super + D): Captures audio with micro-tap filtering, ducks background media streams to 50% via PipeWire (wpctl), feeds a warm faster-whisper CUDA daemon (/tmp/spud-stt.sock), and types into the focused window using wtype (Wayland virtual keyboard) at sub-200ms latency.
Jargon Reinforcement (src/dict.rs): Pre-conditions Whisper’s vocabulary with terms like shpadoinkle-1, Hyprland, Herdr, wtype, PipeWire, and kubectl, coupled with deterministic phonetic replacement rules so technical jargon is never misheard.
Agent Mode & Cloned Voice Egress (Super + Shift + D): Dispatches commands to local agent panes, queries local LLMs, and synthesizes verbal responses through a warm Qwen3-TTS daemon on CUDA using cached speaker embeddings and soft-knee audio compression.
Self-Improving Desktop Skills: Antigravity skills (like add-shortcut) allow the agent to inspect Hyprland configurations, validate keybindings, and trigger live hyprctl reloads without touching a config file by hand.

5. The Maths

Metric	The Biological Relay	Spud + Gemini (`agy`)
Dictation Latency	3–6s (cloud API / batch STT)	<200ms (`faster-whisper` on CUDA)
Audio Ducking	Manual keyboard faffing	Automatic PipeWire stream attenuation (50%)
Cursor Typing	Clipboard copy-pasting	Native Wayland `wtype` at focused cursor
Tech Jargon Accuracy	Fails on `kubectl`, `wtype`	Deterministic phonetic dictionary
Wayland & Hyprland Fit	Clunky wrappers, focus loss, XWayland hacks	Zero-overhead native Wayland daemon (`wtype`)
Agent Egress	Reading terminal blocks	Zero-shot cloned voice TTS audio debrief
Pairing Quota	Daily exhaustion anxiety	Endless practical quota, zero interruptions
Initial Implementation	2–3 weeks of weekend hacking	2 hours 27 minutes

The Receipt

The biological relay is retired.

Give your terminal a voice. Let your music duck itself. Let your desktop configure its own shortcuts.

# Clone and build Spud
git clone git@github.com:nihilok/spud.git ~/Code/spud
cd ~/Code/spud && cargo build --release

Spud on GitHub

The Discovery Tax

Sat, 19 Sep 2026 00:00:00 +0000

Every Claude Code session starts the same way. You say “run the tests.” The agent says “I’ll look for how to run tests in this project.”

Then it reads your README. Then your Makefile. Then your package.json. Then it tries make test. Wrong. npm test. Wrong project. It cats the Makefile again, picks a different target, runs it with the wrong arguments. Eventually it gets there.

Six tool calls. 2,500 tokens of context. Gone — before any real work begins.

That’s the discovery tax. You pay it every session. The agent forgets everything and starts over. Every time.

What it actually costs

It’s not just the wasted tokens. It’s the compound effect:

Context pressure. On long sessions you’re already fighting the context window. Burning 2,500 tokens on discovery means you hit compression sooner.
Confidence degradation. Each failed attempt leaves error messages in the context. The agent second-guesses itself. It starts hedging. “Let me try another approach…” You’ve seen this — the agent starts confidently, hits two errors, and suddenly every response begins with “I apologise for the confusion.”
Your time. You’re sitting there watching it fumble through files you could have pointed it at in seconds.

The refund

What if the agent already knew your project’s tools before it started?

That’s what MCP is for. Instead of discovering tools by reading files, the agent gets a structured registry: tool names, descriptions, typed parameters, defaults. No reading. No guessing. No retries.

I built a task runner called run with a built-in MCP server. You define your tasks in a Runfile:

# @desc Run the test suite
# @arg filter Optional test name filter
test(filter = "") {
    cargo test --workspace --no-fail-fast $filter
}

# @desc Deploy to an environment
# @arg env Target environment (staging|prod)
deploy(env) ./scripts/deploy.sh $env

Add it to your Claude Code config:

{
  "mcpServers": {
    "runtool": { "command": "run", "args": ["--serve-mcp"] }
  }
}

Now the same interaction looks like this:

You:    run the tests
Agent:  [tool] mcp:runtool test → success

One tool call. ~100 tokens. Same result next session.

The maths

	Without MCP	With MCP
Tool calls to run tests	~6	1
Context consumed	~2,500 tokens	~100 tokens
Next session	Starts from scratch	Same registry
Reliability	Varies by session	Deterministic

Over a day of coding sessions, that’s thousands of tokens and dozens of tool calls you’re not wasting on rediscovery.

The discovery tax is zero.

For the full picture — auto-truncation, security sandboxing, the deterministic skills layer — see the deep dive.

brew install nihilok/tap/runtool

GitHub · Docs

Your Agent Doesn't Need a README

Sun, 08 Mar 2026 00:00:00 +0000

Your AI agent is reading your README right now. It’s on paragraph three of the “Getting Started” section, trying to figure out whether npm run dev or make dev is the current way to start the project. It will get there eventually. That’s the problem.

We write READMEs for humans. Structured headings, prose explanations, code blocks with context. A human reads “To run the tests, use cargo test --workspace” and knows what to do. But an AI agent reads the same sentence and has to:

Find the README
Parse the natural language
Extract the command from the surrounding prose
Hope the README is up to date
Hope there aren’t flags mentioned three paragraphs later that are also required

It works. Eventually. But it’s the wrong interface for the job.

Structured beats unstructured

READMEs are documentation. They’re great at explaining why. They’re terrible at telling a machine what, because the machine has to do natural language parsing on a document designed for humans, and extract structured information from an unstructured format.

What agents actually need is a schema: a tool name, a description, typed parameters, and defaults. That’s what MCP provides.

A Runfile gives your project’s tools exactly that schema:

# @desc Run the test suite
# @arg filter Optional test name filter
test(filter = "") {
    cargo test --workspace --no-fail-fast $filter
}

The agent doesn’t read this file. It receives a structured tool definition over MCP: test is a tool, it takes an optional string parameter called filter, and here’s what it does. No parsing. No ambiguity.

What the agent sees

From a README:

A blob of Markdown. It has to figure out which parts are commands, which parts are commentary, and which parts are outdated. It might get it right. It might try the example from the “Legacy Setup” section that nobody deleted.

From an MCP tool registry:

{
  "name": "test",
  "description": "Run the test suite",
  "parameters": {
    "filter": { "type": "string", "description": "Optional test name filter" }
  }
}

No ambiguity. No staleness risk. The schema is the interface.

What the agent doesn’t see

Here’s the other thing: when the agent reads your README to discover commands, it also reads everything else in there. Internal URLs, architecture decisions, deployment details, service names. All of it goes into the context window.

An MCP tool registry exposes only the tool interface: name, description, parameters. The implementation stays behind the wall. Your agent gets powerful, well-defined tools without getting a map of your internals.

Deterministic beats probabilistic

There’s a deeper point here. When an agent reads a README and extracts a command, the result is probabilistic. It probably gets the right command. It probably passes the right flags. But “probably” compounds badly across a session. Each probably-correct step increases the chance that one of them isn’t.

A Runfile is deterministic. run test runs the test suite, with the right flags, every time. The agent doesn’t interpret instructions, it calls a tool. There’s no gap between what you intended and what executes.

This matters even more when you combine it with Claude Code’s skills. You can write a skill that says “before committing, always run ci” — and because ci is a deterministic MCP tool, not a natural language instruction, the behaviour is predictable and auditable. Skills tell the agent when to act. The Runfile tells it what to do. Keeping those concerns separate makes both more reliable.

The README is still useful

This isn’t an argument against READMEs. Write them for your human teammates. Explain the why, the architecture, the gotchas.

You can even document your Runfile commands in the README. They’re just as useful on the command line as they are via MCP: run test, run deploy staging, run ci. The README explains when and why to use them. The Runfile is the executable source of truth.

But stop expecting your agent to use the README as an API. Give it a real one.

brew install nihilok/tap/runtool

GitHub · Docs

How I Gave Claude Code a Deterministic Toolbox (and Stopped It Guessing How to Run My Project)

Sat, 07 Mar 2026 00:00:00 +0000

If you’ve spent any real time pairing with Claude Code, you’ve seen the dance.

You ask it to run the tests. It reads your README. Maybe it finds a Makefile, maybe a package.json, maybe a scripts/ directory. It tries make test. That fails. It tries npm test. Wrong project. It cats your Makefile, parses the targets, picks one, runs it with the wrong arguments. Three tool calls and 2,000 tokens later, it’s running the right command — until next session, when it’s forgotten everything and does the dance again.

This is the discovery tax. Every Claude Code session pays it, and it compounds: each failed attempt burns context window, each retry eats tokens, and the agent’s confidence degrades as it accumulates error messages. On a long session where you’re already bumping up against context limits, those wasted tokens matter.

I built a tool called run that fixes this. Here’s how.

The problem isn’t intelligence, it’s discovery

Claude Code is remarkably good at executing tasks once it knows what to do. The bottleneck is the gap between “I need to run the tests” and “the command is cargo test --workspace --no-fail-fast”. Bridging that gap currently requires the agent to:

Read files to discover what’s available (README, Makefile, package.json, scripts/)
Infer intent from naming conventions and comments
Guess at arguments — is it make deploy ENV=staging or make deploy-staging or ./scripts/deploy.sh staging?
Fail and retry when the guess is wrong
Forget everything next session and repeat from step 1

Each of these steps consumes tool calls and context. And because there’s no structured metadata — just files full of text — the agent is doing natural language parsing of shell scripts, which is exactly the kind of task where LLMs are most likely to make subtle mistakes.

What if the agent already knew?

MCP (Model Context Protocol) lets you expose tools to AI agents with structured schemas — name, description, typed parameters, defaults. The agent doesn’t discover tools by reading files; it discovers them through a protocol designed for exactly this purpose.

RunTool is a task runner with a built-in MCP server. You define your project’s tasks in a Runfile:

# @desc Run the full test suite
# @arg filter Optional test name filter
test(filter = "") {
    cargo test --workspace --no-fail-fast $filter
}

# @desc Deploy to the specified environment
# @arg environment Target environment (staging|prod)
deploy(environment) {
    ./scripts/deploy.sh $environment
}

# @desc Resize an image using Python
# @arg file Path to the image file
# @arg width Target width in pixels
# @arg height Target height in pixels
resize(file, width: int, height: int) {
    #!/usr/bin/env python
    from PIL import Image
    img = Image.open(file)
    img.resize((width, height)).save(file)
    print(f"Resized to {width}x{height}")
}

Add run as an MCP server in your Claude Code config:

{
  "mcpServers": {
    "runtool": {
      "command": "run",
      "args": ["--serve-mcp"]
    }
  }
}

Now when Claude Code starts a session, it immediately sees a tool registry: test takes an optional string filter; deploy takes a required string environment; resize takes a file string plus width and height integers. No file reading. No guessing. No retry loops. Zero discovery tax.

Before and after

Before (no MCP, typical Claude Code session):

Human: run the tests

Claude Code: I'll look for how to run tests in this project.
  [tool] cat README.md                           → 800 tokens
  [tool] ls scripts/                              → 50 tokens
  [tool] cat Makefile                             → 600 tokens
  [tool] make test                                → fails (wrong target name)
  [tool] make tests                               → fails (missing argument)
  [tool] make tests FILTER=""                     → success

  Total: 6 tool calls, ~2,500 tokens of context consumed
  Next session: starts from scratch

After (RunTool MCP):

Human: run the tests

Claude Code: I'll run the test suite.
  [tool] mcp:runtool test                          → success

  Total: 1 tool call, ~100 tokens of context consumed
  Next session: same tool registry, same result

That’s not a marginal improvement. It’s the difference between an agent that fumbles through your project and one that operates like a team member who already knows the codebase.

Auto-truncation: solving the output problem

There’s a second problem that anyone using MCP tools with Claude Code or Codex has run into: output truncation. Claude Code and Codex both impose limits on tool output — and when your test suite dumps 500 lines of results, the agent gets a chopped-up view with the middle missing, which is often exactly where the useful information is.

RunTool handles this at the MCP server level. By default, tool output is capped at approximately 300 tokens (~1KB) — enough for the agent to see whether something passed or failed and get the key details, without blowing up the context window. Critically, run keeps the tail of the output, not the head. That’s where the useful information almost always is: the test results summary, the final error message, the exit status. When output exceeds the limit, run truncates from the top and saves the full output to a file. The agent gets the actionable ending plus a file path it can read selectively if it needs the full details. The threshold is configurable in the environment, so you can tune it up or down to match your context budget.

Compare this to what happens without it: Claude Code and Codex both impose their own truncation on tool output — Codex chops at 256 lines or 10KB using a head+tail strategy that drops the middle, which is often exactly where the useful information is. RunTool’s approach is smarter because it happens at the source, before the agent’s own limits kick in, it prioritises the part of the output you actually care about, and the full output is always recoverable.

The security argument: sandboxing through metadata

Here’s something that’s easy to overlook but has real implications for how you think about agent access.

When Claude Code reads your Makefile or scripts to discover tasks, it sees everything: implementation details, file paths, secrets files being sourced, internal service URLs, database connection strings referenced in scripts. All of that goes into the context window.

RunTool’s MCP server exposes only the annotations — function name, description, argument signatures. The implementation stays hidden. The agent knows that deploy takes an environment argument of type string; it doesn’t know that internally it sources ~/.secrets/prod.env and shells out to an internal deployment service at deploy.internal.corp:8443.

This is sandboxing through metadata. You’re giving the agent powerful, well-defined tools without giving it a map of your internals. If the agent wants to read the Runfile itself it can (it’s just a file), but that’s an explicit read operation that you could restrict — not something the MCP server hands over automatically.

For teams that are cautious about what their AI agents can see — and you should be — this is a meaningful property.

The deterministic layer: encoding skills in your Runfile

Claude Code has a concept of “skills” — instructions stored in .claude/ that tell the agent how to handle specific tasks. Skills are useful, but they’re prompts: natural language instructions that the agent interprets probabilistically. They can drift, be misinterpreted, or interact unpredictably with other context.

A Runfile is a deterministic skill layer. When you define:

# @desc Run linting with auto-fix
lint() cargo clippy --fix --allow-dirty

# @desc Format all source files
fmt() cargo fmt --all

# @desc Run the full CI pipeline locally
ci() {
    run lint
    run fmt
    run test
}

There’s no interpretation involved. ci runs lint, then fmt, then test, in that order, every time. The agent doesn’t need to figure out your CI pipeline from scattered config files — it has a single, composable tool that does exactly what you’ve defined.

This is particularly powerful when combined with skills. You can write a .claude/ skill that says “before committing, always run the ci tool” — and because ci is a deterministic MCP tool rather than a set of natural language instructions, the behaviour is predictable and auditable.

Skills tell the agent when to act. The Runfile tells it what to do. Keeping those concerns separate makes both more reliable.

Getting started

Install run:

brew install nihilok/tap/runtool
# or
cargo install run

Create a Runfile in your project root with your common tasks. Add @desc and @arg annotations for anything you want the agent to discover. Add run as an MCP server in your Claude Code config.

The documentation covers the full syntax, including polyglot scripting (Python, Node, Ruby, PowerShell in the same file), cross-platform @os variants, and command composition.

The code is on GitHub. If you’re working with Claude Code daily and you’re tired of watching it guess how to run your project, give it a try.

Integration Test Isolation in a Next.js App with Drizzle ORM

Sun, 25 Jan 2026 00:00:00 +0000

1. Introduction: The Complexity of Modern Integration Testing

The evolution of web application architecture has precipitously increased the complexity of ensuring software quality. In the era of the monolithic, server-rendered application—typified by frameworks such as Ruby on Rails or early ASP.NET—testing strategies were relatively straightforward. The application ran in a single process, connected to a single relational database, and served HTML directly. Integration testing in such an environment was a solved problem: wrap the test in a database transaction, execute the logic, and roll back the transaction at the end of the test. This approach, often referred to as the “transactional rollback” strategy, provided a clean slate for every test case, ensuring isolation and determinism with minimal overhead.

However, the modern full-stack landscape, dominated by React meta-frameworks like Next.js and diverse data-access layers such as Drizzle ORM, has fundamentally altered this equation. The shift towards serverless compute models, the introduction of React Server Components (RSC), and the decoupling of the frontend from the backend logic via mechanisms like Server Actions have introduced new boundaries and constraints. In this environment, the database is no longer just a passive store of state; it is a shared mutable resource accessed by highly concurrent, asynchronous processes that may not share the same memory space or execution context as the test runner.

This report provides an exhaustive analysis of the strategies available for achieving rigorous integration test isolation in a Next.js application using Drizzle ORM. It explores the theoretical underpinnings of database isolation, the specific architectural challenges posed by the Next.js App Router, and the practical implementation of four distinct testing methodologies: Transactional Rollbacks, Database Truncation, Containerised Isolation (Testcontainers), and In-Memory WebAssembly Databases (PGlite). By examining the mechanisms, performance characteristics, and trade-offs of each approach, this document aims to equip engineering teams with the knowledge required to architect robust, scalable, and non-flaky test suites for mission-critical applications.

1.1 The Imperative of Isolation

Integration testing sits at the precarious intersection of the testing pyramid. Unlike unit tests, which verify discrete logic in isolation by mocking external dependencies, integration tests must validate the interaction between the application code and its infrastructure—specifically, the database. The fidelity of these tests is paramount. If a test mocks the database driver, it ceases to be an integration test; it merely tests the developer’s assumptions about how the database should behave, rather than how it actually behaves.

The central challenge in integration testing is Shared Mutable State. When a test suite runs, multiple test cases execute against the database. If these tests are not perfectly isolated, they interfere with one another, leading to a class of failures known as “flaky tests.” Flakiness manifests in several ways:

Data Pollution: One test inserts a record (e.g., a user with email test@example.com) that persists after the test finishes. A subsequent test, expecting an empty user table or attempting to create a user with the same unique email, fails.
Phantom Reads: In a concurrent test environment, a query in Test A might inadvertently retrieve data inserted by Test B, leading to assertion failures that are impossible to reproduce when running the test in isolation.
Race Conditions: Tests that modify global configuration or shared singleton resources can create non-deterministic outcomes based on the precise timing of execution by the CPU scheduler.

To mitigate these risks, the testing environment must guarantee that every test runs in a pristine environment. The database state at the beginning of a test must be known, deterministic, and unaffected by any previous or concurrent test execution.

1.2 The “Next.js Effect”: Architectural Constraints

Next.js, particularly with its App Router architecture introduced in version 13, imposes specific constraints that complicate traditional isolation strategies.

Server Actions as Remote Procedure Calls (RPC): Server Actions in Next.js allows functions to be executed on the server, triggered directly from client-side components. From a testing perspective, these are not simple function calls; they are asynchronous operations that often run in a separate context. Testing them requires an environment that can simulate the Next.js server runtime, including access to environment variables, headers, and cookies.
The Singleton Pattern & Module Caching: In a typical Next.js application, the database client (Drizzle instance) is instantiated as a global singleton. This is necessary to prevent connection exhaustion during development (hot reloading) and to manage connection pooling in production. However, this global singleton makes Dependency Injection (DI)—a primary technique for test isolation—significantly more difficult. If a Server Action imports the database client directly from a static module path (e.g., import { db } from '@/db'), the test runner must intervene at the module loading level to inject a test-specific database instance.
Server-Only Boundaries: Next.js enforces strict boundaries between server and client code using the server-only package. Integration tests that import server-side logic must execute in a Node.js-compatible environment (like vitest with a node environment) rather than a browser-like environment (like jsdom), or they will trigger build-time errors. This constrains the choice of test runners and mocking strategies.

1.3 Drizzle ORM: A TypeScript-First Approach

Drizzle ORM represents a modern approach to database interaction, favoring type safety and SQL-like syntax over the heavy abstraction layers of traditional ORMs. It separates the query builder (the TypeScript API) from the driver (the mechanism that talks to the database). This separation is crucial for testing because it allows the underlying driver to be swapped—for example, replacing a network-based Postgres client with an in-memory PGlite client—without changing the application logic. However, Drizzle’s reliance on specific driver features (like prepared statements or specific transaction APIs) means that the test environment must closely mirror the production environment’s capabilities to avoid false positives.

2. Theoretical Framework: ACID Properties and Test Concurrency

To understand why integration test isolation is difficult, one must delve into the fundamental properties of relational databases: ACID (Atomicity, Consistency, Isolation, Durability). Integration testing strategies essentially manipulate these properties to achieve their goals.

2.1 Atomicity and Test Boundaries

Atomicity guarantees that a transaction is treated as a single “unit of work,” which either completely succeeds or completely fails. The Transactional Rollback strategy leverages this property. By wrapping a test case in a transaction, the test runner ensures that all database writes are provisional. When the test concludes, the runner issues a ROLLBACK command, which atomically undoes all changes. This relies on the database’s Write-Ahead Log (WAL) to revert the state efficiently, without the overhead of deleting records from the disk.

However, Atomicity interacts complexly with modern application logic. If the application code under test itself uses transactions (e.g., a signup flow that creates a user and an organisation in a single transaction), the test runner must support Nested Transactions. In PostgreSQL, true nested transactions do not exist; they are simulated using SAVEPOINT. Drizzle ORM supports this via its tx API, allowing a test to start a transaction (Level 1) and the application to start a nested transaction (Level 2). A rollback of Level 2 reverts to the savepoint, while a rollback of Level 1 reverts everything.

2.2 Database Isolation Levels

The “I” in ACID refers to Isolation—the degree to which a transaction is protected from the effects of other concurrent transactions. PostgreSQL offers several isolation levels:

Read Committed (Default): A query sees only data committed before the query began.
Repeatable Read: A query sees a snapshot of the database as of the start of the transaction.
Serialisable: The strictest level, ensuring that the result of concurrent transactions is the same as if they had executed serially.

For integration tests running in parallel against a shared database, Read Committed is often insufficient. If Test A (running in a transaction) inserts a record, and Test B (also in a transaction) runs a query that counts records, Test B generally won’t see Test A’s uncommitted data. However, if Test A commits (which might happen if the application logic forces a commit), Test B’s state is polluted. This creates a “race condition” where tests pass or fail depending on which one finishes first. Achieving true isolation usually requires ensuring that tests running in parallel never share the same database instance, or use strict row-level locking which degrades performance.

2.3 The Throughput vs. Latency Trade-off in Testing

Architecting a test suite involves a fundamental trade-off between Latency (how fast a single test runs) and Throughput (how fast the entire suite runs).

Serial Execution: Running tests one by one (latency focused) is the easiest way to ensure isolation. You can use a single database and truncate it between tests. However, as the suite grows to thousands of tests, the total execution time (throughput) becomes unacceptable, leading to slow CI pipelines.
Parallel Execution: Running tests concurrently (throughput focused) drastically reduces total time but requires advanced isolation strategies. Each parallel worker needs its own isolated environment (e.g., its own database schema or container) to prevent collisions. This increases the resource cost (CPU/RAM) of the test runner.

The following sections analyze four strategies that navigate this trade-off space differently.

3. Strategy I: Transactional Rollbacks (The “Rails” Pattern)

The Transactional Rollback strategy is often considered the “gold standard” in frameworks like Ruby on Rails and Django. It promises the best of both worlds: speed (no need to recreate schemas) and isolation (changes are never committed).

3.1 Mechanism of Action

In this model, the test runner performs the following sequence for each test case:

Connect: Establish a connection to the database.
Begin: Execute BEGIN to start a transaction.
Run: Execute the test logic. The application performs INSERT, UPDATE, and DELETE operations. These changes are visible to the connection but not to the outside world.
Rollback: Execute ROLLBACK in the teardown or afterEach hook. The database discards the changes.

This approach is highly efficient because ROLLBACK is an inexpensive operation for the database engine compared to TRUNCATE or DELETE, which require scanning tables and updating indexes.

3.2 Implementation Challenges in Next.js

While conceptually simple, implementing this in a Next.js/Drizzle application is fraught with architectural difficulties, primarily due to the Connection Context Problem.

In a typical Next.js app, the database client is a global singleton imported by Server Actions:

// src/lib/db.ts
export const db = drizzle(postgres(process.env.DATABASE_URL));

When a test runs, it typically initiates a transaction on its own connection instance. However, when the test calls a Server Action, that action imports the global db instance, which usually maintains its own connection pool. Consequently, the Server Action executes its queries outside the test’s transaction. The changes are committed to the database, polluting the state for subsequent tests and rendering the ROLLBACK ineffective.

3.3 The Proxy & AsyncLocalStorage Solution

To make Transactional Rollbacks work in Node.js/Next.js, one must force the application code to use the test’s transaction connection instead of the global pool. This requires a form of “Context Propagation.”

AsyncLocalStorage (ALS) from the node:async_hooks module provides a mechanism to store data that is unique to the current asynchronous execution context (similar to Thread-Local Storage in multi-threaded languages).

Implementation Architecture:

The Wrapper: Create a database accessor that checks the ALS store.

import { AsyncLocalStorage } from 'node:async_hooks';

export const txStorage = new AsyncLocalStorage<any>();

export const getDb = () => {
  const tx = txStorage.getStore();
  return tx ? tx : globalDb;
};

The Test Hook: In beforeEach, start a transaction and enter the ALS context.

test('example test', async () => {
  await globalDb.transaction(async (tx) => {
    await txStorage.run(tx, async () => {
      // Inside this block, getDb() returns 'tx'
      await myServerAction(); 
      tx.rollback(); // Force rollback at end
    });
  });
});

Refactoring: The application must be refactored to use getDb() instead of importing db directly, or the db export must be a Proxy object that handles this logic internally.

3.4 Pros and Cons

Feature	Analysis
Speed	High. Rollbacks are nearly instantaneous. No file I/O or schema rebuilding is required between tests.
Parallelism	Low/Complex. Since multiple tests effectively share the same underlying database instance (even if wrapped in transactions), running them in parallel requires careful management of connection limits and locking. It essentially forces serial execution within a single database.
Complexity	High. Requires invasive changes to the application architecture (using ALS or Proxies) or strict Dependency Injection. Testing “transactional” logic within the app becomes confusing (nesting transactions inside test transactions).
Fidelity	Medium. Logic that relies on “after commit” hooks or side effects visible to other connections cannot be tested easily because the transaction is never actually committed.

Verdict: While powerful, the Transactional Rollback strategy fights against the grain of the Next.js App Router’s module system. It is best suited for applications that already use heavy Dependency Injection patterns (like NestJS) rather than standard Next.js patterns.

4. Strategy II: Database Truncation (The “Clean Slate” Approach)

The Truncation strategy is the brute-force alternative to rollbacks. Instead of preventing writes from persisting, we allow them to persist and then aggressively clean the database after every test.

4.1 Mechanism of Action

Global Setup: A real database is spun up (usually via Docker) and the schema is migrated once at the start of the test suite run.
Test Execution: The test runs against this real database. Data is committed.
Teardown (afterEach): A utility function queries the database for all table names and executes TRUNCATE commands to wipe all rows.

4.2 Handling Foreign Keys and Sequences

A naive DELETE FROM table is insufficient and slow. TRUNCATE is faster as it deallocates data pages. However, relational integrity constraints (Foreign Keys) pose a challenge. You cannot truncate the users table if the posts table references it.

To solve this, PostgreSQL provides the CASCADE option:

TRUNCATE TABLE users, posts, comments CASCADE;

Additionally, TRUNCATE does not reset auto-incrementing primary keys (IDENTITY columns) by default. If Test A creates User ID 1, Test B will create User ID 2. If Test B asserts “User ID should be 1”, it will fail. The RESTART IDENTITY clause is required.

Optimised Drizzle Implementation:

import { sql } from 'drizzle-orm';

export async function resetDatabase(db: any) {
  // Disable triggers if necessary or use CASCADE
  const query = sql<string>`
    SELECT table_name
    FROM information_schema.tables
    WHERE table_schema = 'public' AND table_type = 'BASE TABLE'
  `;
  const tables = await db.execute(query);
  
  if (tables.rows.length === 0) return;

  // Construct a single query to truncate all tables at once
  const tableNames = tables.rows.map((row: any) => `"${row.table_name}"`).join(', ');
  
  // RESTART IDENTITY resets sequences
  // CASCADE handles foreign keys
  await db.execute(sql.raw(`TRUNCATE TABLE ${tableNames} RESTART IDENTITY CASCADE`));
}

This script is robust because it dynamically discovers tables, ensuring that newly added tables are automatically cleaned up without updating the test helper.

4.3 The Concurrency Bottleneck

The fatal flaw of the Truncation strategy is its incompatibility with parallel testing. Since there is only one shared database instance, you cannot run Test A and Test B simultaneously. If Test A truncates the database while Test B is in the middle of an operation, Test B will fail catastrophically.

This forces the test runner (e.g., Vitest or Jest) to run in Serial Mode (--runInBand or maxWorkers=1). As the application grows to hundreds of integration tests, the feedback loop extends from seconds to minutes, severely hampering developer productivity.

4.4 Pros and Cons

Feature	Analysis
Speed (Per Test)	Medium. Truncate is faster than DROP/CREATE but slower than ROLLBACK. The database must physically clean pages on disk.
Speed (Total)	Low. Forces serial execution. Scalability is linear with the number of tests.
Simplicity	High. Very easy to understand and implement. No “magic” proxies or complex context switching. The database behaves exactly as production.
Reliability	High. Cleans everything. Less prone to subtle state leaks than complex transaction nesting.

Verdict: Database Truncation is a viable strategy for smaller projects or CI pipelines where parallelism is not a priority. It is robust and simple but hits a hard ceiling on scalability.

5. Strategy III: Containerised Isolation (Testcontainers)

To solve the concurrency problem of the Truncation strategy without sacrificing the fidelity of a real database, many teams turn to Testcontainers. This library allows the test runner to programmatically spin up disposable Docker containers for dependencies.

5.1 The Architecture of Ephemeral Infrastructure

Testcontainers for Node.js wraps the Docker API. It allows a test to:

Request a PostgreSQL image (e.g., postgres:16-alpine).
Start the container on a random, available port.
Wait for the database to be ready (using log strategies or health checks).
Return the connection string to the application.
Destroy the container automatically when the test finishes.

The “Ryuk” sidecar container is a special component of Testcontainers that ensures cleanup. It monitors the connection to the test runner; if the runner process dies (even unexpectedly), Ryuk kills the spawned containers, preventing “zombie” processes from consuming system resources.

5.2 Implementation Modes

There are two ways to deploy this for Next.js testing:

5.2.1 Singleton Container with Logical Isolation

Spinning up a Docker container takes time (500ms - 2s). Doing this for every test file is often too slow. A common pattern is to start one container for the entire test suite (global setup). Then, for each test worker, create a unique logical database inside that container.

Setup: Container starts at port 5432.
Worker 1: Connects, runs CREATE DATABASE test_db_1, migrates, runs tests.
Worker 2: Connects, runs CREATE DATABASE test_db_2, migrates, runs tests.

This allows parallelism. However, it requires complex orchestration code to manage unique database names and ensure migrations are run for every new logical database created.

5.2.2 Container Per Test File

The most isolated approach is to give every test file its own container.

Pros: Perfect isolation. No shared state.
Cons: Heavy Resource Usage. Running 10 parallel test files means running 10 PostgreSQL containers simultaneously. This requires significant RAM and CPU. On a typical CI agent (with 2 vCPUs and 4GB RAM), this will likely cause crashes or extreme slowness due to context switching.

5.3 Drizzle Integration

Connecting Drizzle to a Testcontainer is straightforward since Drizzle accepts any standard connection string.

import { PostgreSqlContainer } from "@testcontainers/postgresql";
import { drizzle } from "drizzle-orm/node-postgres";

// In global setup
const container = await new PostgreSqlContainer("postgres:16").start();
const client = new Client({ connectionString: container.getConnectionUri() });
await client.connect();
export const db = drizzle(client);

5.4 Pros and Cons

Feature	Analysis
Fidelity	Maximum. You are testing against the exact binary version used in production. Extensions (PostGIS, pgvector) work perfectly.
Isolation	High. Containers provide process-level isolation.
Performance	Low to Medium. Docker startup overhead is the main bottleneck. High resource consumption limits the degree of parallelism possible on standard hardware.
Infrastructure	Complex. Requires a Docker daemon availability. This can be challenging in certain CI environments (e.g., Docker-in-Docker setups) or restricted corporate laptops.

Verdict: Testcontainers is the industry standard for End-to-End (E2E) tests where accuracy is more important than speed. For integration tests that run frequently (e.g., on every file save), it is often too heavy.

6. Strategy IV: In-Memory WebAssembly Databases (PGlite)

The most recent and transformative development in this space is the emergence of PGlite. This technology fundamentally shifts the “Latency vs. Throughput” curve by offering the isolation of containers with the speed of in-memory execution.

6.1 What is PGlite?

PGlite is a build of PostgreSQL compiled to WebAssembly (WASM). Unlike mocks (which fake the database) or SQLite (which is a different database entirely), PGlite runs the actual Postgres query engine code. It executes within the Node.js process memory. It does not require a daemon, a network port, or Docker.

Because it is just a JavaScript object consuming memory, starting a PGlite instance takes milliseconds (~10-50ms), compared to seconds for a Docker container.

6.2 Architecture for Massive Parallelism

PGlite enables a “Database per Test File” architecture without the resource penalty of Docker.

Integration with Vitest:

Vitest is a modern test runner that supports multi-threading. By combining Vitest’s parallelism with PGlite, we can achieve perfect isolation.

Vitest Config: Configure the test pool to use threads or forks. This ensures each test file runs in its own worker.
Mocking the Database: We use Vitest’s vi.mock to intercept imports to the application’s database module.
Setup File: In the setup for each test file, we instantiate a new PGlite database.

// vitest.setup.ts
import { vi } from 'vitest';
import { PGlite } from '@electric-sql/pglite';
import { drizzle } from 'drizzle-orm/pglite';
import { migrate } from 'drizzle-orm/pglite/migrator';

// Mock the module that exports the singleton 'db'
vi.mock('@/lib/db', async () => {
  // 1. Create a fresh in-memory Postgres instance
  const client = new PGlite(); 
  
  // 2. Connect Drizzle
  const db = drizzle(client);

  // 3. Apply Schema (using Drizzle's push or migrate)
  // Note: PGlite is fast enough to run migrations per test file
  await migrate(db, { migrationsFolder: './drizzle' });

  return { db };
});

6.3 The “Memory Filesystem” Mechanism

PGlite can persist data to a virtual filesystem in memory (memory://). This means that when the test file finishes and the worker process terminates, the memory is reclaimed, and the database effectively vanishes. There is no cleanup required—no DROP DATABASE or TRUNCATE calls are needed. This “fire and forget” model drastically simplifies the test lifecycle.

6.4 Handling Server Actions with PGlite

Since Next.js Server Actions are just functions imported into the test, they will transparently use the mocked db module. When myServerAction() calls db.insert(...), it is inserting into the local PGlite instance running in the test thread. Because each test thread has its own mocked instance, they can run fully in parallel without any risk of data collision.

6.5 Performance Benchmarks

Comparative benchmarks reveal the efficiency of this approach:

Startup Time: PGlite starts in ~20ms vs Testcontainers ~1500ms.
Throughput: In a suite of 100 test files, PGlite allows running as many workers as CPU cores (e.g., 8 or 16). Testcontainers might be limited to 2-3 concurrent containers before saturating RAM.
Execution: CRUD operations in PGlite are slightly slower than native Postgres due to WASM overhead, but the elimination of network latency (localhost TCP loopback) often compensates for this in integration test scenarios.

6.6 Pros and Cons

Feature	Analysis
Speed	Very High. Instant startup enables a “Database per Test File” strategy.
Isolation	Perfect. Every test file gets a completely independent, shared-nothing database instance.
Parallelism	Excellent. Scaling is limited only by CPU cores, not RAM or Docker limits.
Compatibility	High. Supports most Postgres features (JSONB, Triggers). However, as a WASM build, it is single-process (no multi-connection testing) and may lack support for specific native extensions like PostGIS unless custom-built.
Dev Experience	Superior. No need to install Docker. `npm install` is all that is needed to run the test suite.

Verdict: For 95% of Next.js applications using Drizzle, PGlite is the optimal strategy. It provides the best balance of speed, isolation, and developer experience.

7. Data Management: Seeding and Determinism

Regardless of the isolation strategy chosen, integration tests require data. “Seeding” is the process of populating the database with the necessary state (users, organisations, items) before a test runs.

7.1 The Drizzle Seed Library

The Drizzle team recently introduced drizzle-seed, a library designed to solve the problem of generating realistic, relational data. Unlike traditional faker libraries which are purely random, drizzle-seed uses a seedable pseudo-random number generator (pRNG).

Why Determinism Matters:

In testing, you want randomness to cover edge cases, but you also want reproducibility. If a test fails because a random name contained an emoji that broke your validation, you need to be able to reproduce that exact failure. drizzle-seed allows you to set a specific seed (e.g., seed: 12345). Every time the test runs with that seed, it generates the exact same set of “random” data.

7.2 Relational Integrity

Generating data for relational databases is hard because of foreign keys. You can’t create a Post without a User. drizzle-seed introspects your Drizzle schema relationships.

await seed(db, schema, { count: 10 }).refine((funcs) => ({
  users: {
    count: 5,
    columns: {
      email: funcs.email(), // deterministic emails
    }
  },
  posts: {
    count: 10, // Distributed among the 5 users
  }
}));

This automatically handles the foreign key dependency, ensuring that every post created is linked to a valid user ID from the users generation step.

7.3 Factories vs. Global Seeds

Global Seeding: Populating the DB with a massive set of “standard” data at the start of the suite. This is good for read-heavy tests but bad for isolation (tests share the seed state).
Factories (Recommended): Using helper functions within each test to create only the data needed for that specific test.

// test/factories.ts
export const createUser = async (overrides = {}) => {
  return await db.insert(users).values({
    ...defaultData,
    ...overrides 
  }).returning();
};

Combined with PGlite, this is extremely fast. Since the DB is empty at the start of the test file, the test creates 1 user, asserts against it, and finishes.

8. Comparative Analysis & Decision Matrix

To guide the selection of the appropriate strategy, we synthesise the findings into a decision matrix.

Criterion	Transactional Rollback	Database Truncation	Testcontainers	PGlite (WASM)
Isolation Quality	Medium (Connection sharing risks)	High (If serial) / Low (If parallel)	High (Process Isolation)	High (Memory Isolation)
Parallelism	Low (Difficult to implement)	None (Must be Serial)	Medium (Resource constraints)	High (CPU bound)
Startup Overhead	Fast	Fast	Slow (Docker spin-up)	Instant
CI Infrastructure	Minimal	Minimal	Heavy (Needs Docker)	Minimal (Node only)
Production Parity	High	High	Exact	High (High fidelity simulation)
Implementation	Complex (Proxies/ALS)	Simple	Moderate	Moderate (Mocking)

8.1 Recommendations

For New (Greenfield) Next.js Applications

Adopt the PGlite Strategy. The benefits of parallel execution and instant startup significantly outweigh the minor complexity of setting up the mock. It allows your test suite to grow to thousands of tests without slowing down your CI pipeline. It aligns perfectly with the isolated module structure of Next.js Server Actions.

For Applications with Specific Postgres Extensions (e.g., PostGIS)

Use Testcontainers. PGlite currently has limited support for complex C-based extensions. If your app relies heavily on geospatial queries or vector similarity search (pgvector), the fidelity of a real Dockerised Postgres instance is non-negotiable. To mitigate the slowness, consider sharding your tests in CI (running different test files on different machines).

For Legacy Applications or Small Suites

Use Database Truncation. If you already have a working Postgres setup and your test suite takes less than 2 minutes to run serially, the effort to migrate to PGlite may not yield immediate ROI. Truncation is simple, reliable, and “good enough” for small scales.

Avoid Transactional Rollbacks in Next.js

The impedance mismatch between the “Stateless Request” model of Next.js Server Actions and the “Stateful Connection” model of SQL transactions makes this strategy brittle. It typically leads to “magic” code (Proxies, ALS) that is hard to debug and hard to onboard new developers to.

9. Conclusion

The landscape of integration testing in the Next.js and Drizzle ecosystem has shifted from managing shared resources to provisioning ephemeral ones. The “old way” of managing a single database and carefully cleaning it up is being superseded by the “new way” of creating disposables.

Technological advancements—specifically containerisation and WebAssembly—have given us new tools. Testcontainers allows us to treat a database server as an ephemeral object, while PGlite allows us to treat the entire database concept as a disposable JavaScript variable.

For the modern Next.js developer, the PGlite approach represents a sweet spot. It respects the boundaries of the framework, enables the blazing speed of Drizzle’s query generation, and provides the isolation necessary to banish flaky tests forever. By combining Vitest’s threading, PGlite’s in-memory speed, and Drizzle’s schema management, teams can build test suites that are not just safety nets, but accelerators of development velocity.

References

Next.js and Drizzle Integration
PGlite Architecture and Benchmarks
Testcontainers Usage and Comparison
Truncation and Seeding Logic
Transaction Management in Drizzle
Server-Only Constraints in Next.js

Why I Built Another Task Runner

Sat, 17 Jan 2026 00:00:00 +0000

Yes, I know. Another task runner. In 2026. Let me explain why.

The Problem

I’m not a Make expert. But somehow I became the person people ask when they need to add a task to our Makefile.

Last week it was “How do I pass an environment variable to this target?” The week before: “Why does this only work if I add a tab character?” And the week before that it was me forgetting to add a task to .PHONY.

Every time, I’m Googling the same things they could be Googling. Our build process has things like this:

deploy-%: guard-% check-git-clean
	@$(eval ENV := $(word 2,$(subst -, ,$@)))
	./scripts/deploy.sh $(ENV)

If you understand what $(word 2,$(subst -, ,$@)) does without looking it up, congratulations — you’re in the 1% of developers who’ve memorised Make’s arcane variable substitution syntax.

The Alternatives Weren’t Much Better

Other repos in the company use npm scripts. Some have custom Python CLI packages run with uv. Every project has its own approach.

The npm scripts repos hit the same cross-platform issues. Shell commands that work fine on Mac and Linux fail on Windows. You end up with:

{
  "clean": "rm -rf dist || rmdir /s /q dist",
  "build": "NODE_ENV=production webpack || set NODE_ENV=production && webpack"
}

Ugly. Fragile. And it still broke in random edge cases.

So I looked at the established alternatives.

Just (~22k stars) is the closest thing to what I wanted: a clean justfile with Make-inspired syntax, recipe parameters, shell completions. But parameters have no type annotations — they’re positional strings and that’s it. Handy for humans, but AI agents can’t reliably introspect them.

Task (go-task) (~15k stars) impressed me with its built-in POSIX shell interpreter, which gives genuinely consistent cross-platform behaviour without relying on the system shell. But it’s shell-only — no inline Python or Node.js. And MCP support is still just a GitHub issue with no implementation.

Mise (~25k stars) is arguably the most feature-rich tool in this space, combining dev tool version management (replacing asdf/nvm/pyenv), environment management, and task running in a single binary. It does have an experimental mise mcp server, which is genuinely interesting — though it exposes the broader dev environment surface rather than being focused specifically on task execution. For teams already using mise for version management, that breadth is a strength.

The Markdown-based runners — Mask, xc, Maid — are a compelling idea (documentation and automation in one file), but none of them have any MCP story, and Maid hasn’t been meaningfully maintained in years.

What I Actually Wanted

I wanted to write tasks the way I think about them:

# Just deploy the thing
deploy(environment) {
    ./scripts/deploy.sh $environment
}

But I also wanted:

Python when I needed real logic (not bash’s string manipulation nightmare)
Node when working with JSON or async operations
Cross-platform support without conditional hell
Something AI agents could actually use — not as an afterthought, but as a first-class feature

That last point turned out to be the gap nobody had filled properly.

So I Built `run`

So I built run. A Runfile looks like this:

# Shell for simple stuff
build() cargo build --release

# Python when you need it
analyze(file) {
    #!/usr/bin/env python
    import json
    with open(file) as f:
        data = json.load(f)
        print(f"Processed {len(data)} records")
}

# Node works too
process() {
    #!/usr/bin/env node
    const fs = require('fs');
    console.log('Processing...');
}

# Platform-specific versions
# @os windows
deploy(environment) {
    .\scripts\deploy.ps1 $environment
}

# @os linux darwin
deploy(environment) {
    ./scripts/deploy.sh $environment
}

No YAML. No TOML. No $(word 2,$(subst -, ,$@)). Just functions with named parameters.

Polyglot Argument Passing

The polyglot model is worth dwelling on — even if you never touch the AI features, this alone is a meaningful improvement over existing tools.

Notice how the Python function above gets file as a native Python variable — no sys.argv[1], no argparse, no parsing. And it goes further than just strings. If you add a type annotation:

resize(file, width: int, height: int) {
    #!/usr/bin/env python
    from PIL import Image
    img = Image.open(file)
    img.resize((width, height)).save(file)
    print(f"Resized to {width}x{height}")
}

width and height arrive as actual Python ints. You can multiply them, pass them to APIs that expect numbers, use them in range expressions — no int(sys.argv[2]) boilerplate. The same applies to Node.js, Ruby, and PowerShell: RunTool generates the appropriate variable declarations for each interpreter, with type-correct values.

Compare this to the typical polyglot task runner experience: you write a shebang script, and everything arrives as a string. You spend the first few lines of every script parsing and converting arguments. It’s not a lot of code, but it’s friction — and it’s the kind of friction that makes people reach for a full CLI framework instead of a quick task.

Just supports multiple languages too, via shebangs — but each shebang recipe has to be a separate named recipe. RunTool lets you mix languages within a single file using function-scoped shebangs. And the @os attribute lets you define the same function name with different implementations per platform, which is more ergonomic than Task’s platforms filter or mise’s run_windows property.

The AI Integration

Here’s where it gets interesting: run has a built-in MCP (Model Context Protocol) server.

I was pairing with Claude Code on some refactoring (it was driving 🙈) when I realised: it would be helpful if Claude could just run our tests or deployment checks directly. With MCP support, AI agents can discover and execute your project’s tools automatically, and use the exact same tools that you use.

Add some metadata:

# @desc Deploy to specified environment
# @arg environment Target environment (staging|prod)
deploy(environment) {
    ./scripts/deploy.sh $environment
}

The function signature is the schema. Claude (or any MCP-compatible agent) sees a tool called deploy with a required environment parameter — no guessing, no multi-step discovery, no verbose Markdown explanations needed. Type annotations and defaults work too:

# @desc Scale a service
# @arg service The service name
# @arg replicas Number of instances
scale(service, replicas: int = 1) {
    docker compose scale $service=$replicas
}

The @desc and @arg annotations serve dual purposes: they generate human-readable help text and provide structured metadata that MCP clients use for tool discovery and invocation. That’s architecturally different from Just’s approach, where three independent community-built MCP servers have to parse the justfile format externally without any structured argument metadata to work with.

There’s a useful security property here too: the MCP server only exposes the annotations — the function name, description, and argument signatures. The implementation is not surfaced. An agent knows that deploy takes an environment argument of type string; it doesn’t know (or need to know) that internally it shells out to ./scripts/deploy.sh, or that there’s a secrets file being sourced, or any other implementation detail. If the agent wants to read the Runfile itself it can, but that’s an explicit read operation — not something the MCP server hands over automatically. You can give agents access to your tooling without giving them a map of your internals.

As of early 2026, the landscape looks like this:

Tool	Polyglot scripting	Typed arguments	MCP integration
RunTool	✅ Shell/Python/Node/Ruby/PS	✅ Signature types + defaults	✅ Built-in
Just	✅ Via shebangs	⚠️ Params, no types	⚠️ 3 third-party servers
Task	❌ Shell-only	⚠️ `requires` + enum	❌ Requested only
Mise	✅ Via shebangs	✅ usage spec	✅ Built-in (experimental)
Mask	✅ Code block langs	⚠️ Flags with types	❌ None
xc	❌ Shell-only	⚠️ Inputs attribute	❌ None
Make	❌ Shell (with quirks)	❌ None	⚠️ Third-party only

RunTool and mise are the only tools with built-in MCP servers. The difference is scope: mise’s MCP server exposes its full environment management surface (tools, tasks, versions, env vars), which is powerful if you want an agent that can manage your entire dev environment. RunTool’s is narrower by design — purpose-built around task execution and structured argument metadata — which means less surface area to audit and a simpler mental model for what the agent can do.

Zero Dependencies, Instant Startup

It’s a single Rust binary. No runtime. No package.json with 500 dependencies. You run run deploy staging and it just works.

And it comes with shell completions out of the box — bash, zsh, fish, and PowerShell. Tab completion for all your tasks, no extra setup required.

Honestly, the Tradeoffs

I’d be doing you a disservice if I didn’t acknowledge what the alternatives have that RunTool doesn’t — yet.

Just has years of battle-testing, a backwards-compatibility guarantee (“there will never be a just 2.0”), and extensive editor support across VS Code, JetBrains, Helix, Kakoune, and Zed. Its community has built three independent MCP servers. That ecosystem trust takes years to build.

Task has something technically impressive that’s easy to overlook: its built-in shell interpreter (mvdan/sh) plus built-in Unix utilities means it’s genuinely cross-platform without platform-specific shims. If your team has Windows developers and you’ve suffered through shell compatibility issues, that’s a compelling argument.

Mise bundles tool version management, environment management, and task running into a single binary — reducing the number of tools a project depends on. If you’re already using mise for version management, its experimental MCP support and rich task runner make it the most comprehensive single-binary solution. And its broader MCP surface area is genuinely useful if you want agents that can manage your full dev environment, not just run tasks.

RunTool is a new entrant betting that AI-agent interoperability will become a first-class concern for developer tooling. That bet might be right or wrong. What I can say is: when I’m working alongside Claude Code on a project and it can discover and run the project’s tasks through the same interface I do, that workflow genuinely changes how I work.

Is This Useful to Anyone Else?

I don’t know. Maybe you’re happy with Make. Maybe Just works perfectly for you. Maybe mise already does everything you need. Maybe you don’t care whether your AI agent can introspect your task arguments.

But if you’ve ever thought “there has to be a simpler way to do this” — whether that’s passing typed arguments to a Python task without argparse, writing cross-platform tasks without conditional hell, or giving your AI agent access to your project’s tools — maybe give it a try:

brew install nihilok/tap/runtool

cargo install run

The code is on GitHub. It solves my problems. Maybe it’ll solve yours too.

Offline Encryption Using WebAuthn

Sat, 08 Mar 2025 00:00:00 +0000

I’ve been diving deep into browser-based cryptography lately, and implementing truly secure offline encryption is frustratingly complex. The core problem? Getting consistent key material across sessions without storing anything sensitive.

After countless late nights of cursing at my keyboard, I stumbled upon an intriguing solution: using WebAuthn (e.g. those little USB security keys like YubiKey) not just for authentication, but as a source of encryption key material.

The Encryption Challenge

Creating secure encryption that works completely offline in a browser environment is difficult for several reasons:

Your key material needs to be consistent across browser sessions
You can’t just derive keys from code (that’s basically security through obscurity)
Browser storage isn’t secure enough for sensitive key material
And of course, the whole thing needs to work offline

After banging my head against various approaches (including some truly questionable experiments with IndexedDB), I realised WebAuthn might be the perfect solution. Why? Because those little hardware authenticators never expose their private keys — they’re basically tiny HSMs that live in your USB port!

WebAuthn: Not Just for Logins Anymore

WebAuthn wasn’t designed to be a “key extractor” - it’s an authentication standard. But with a bit of creative thinking (and some cryptographic sleight of hand), we can use it to generate consistent encryption material.

Here’s how it works in a nutshell:

Register a credential (one-time setup)
Store the credential ID (not secret, so browser storage is fine)
Get an assertion using a fixed challenge whenever you need key material
Use the signature as input for your encryption key derivation

The real magic here is that for a given credential and challenge, the hardware authenticator will produce a signature that we can use as key material. The private key never leaves the device!

Implementation

Here’s how to implement the registration phase:

// A randomly generated challenge (only used during registration)
const challenge = window.crypto.getRandomValues(new Uint8Array(32));

// Configure your publicKeyCredentialCreationOptions
const publicKeyOptions = {
  challenge: challenge,
  rp: { name: "Offline Demo RP" },
  user: {
    id: Uint8Array.from("unique-user-id", c => c.charCodeAt(0)),
    name: "demo-user",
    displayName: "Demo User"
  },
  pubKeyCredParams: [{ type: "public-key", alg: -7 }], // ECDSA with SHA-256
  timeout: 60000,
  attestation: "none"
};

navigator.credentials.create({ publicKey: publicKeyOptions })
  .then(cred => {
    // Save the credential ID for future assertions
    const credId = new Uint8Array(cred.rawId);
    // Persist credId (e.g., in IndexedDB)
    console.log("Registration done, credentialId:", bufferToBase64Url(credId));
  })
  .catch(err => {
    console.error(err);
  });

function bufferToBase64Url(buffer) {
  const str = String.fromCharCode(...new Uint8Array(buffer));
  return btoa(str).replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '');
}

And here’s the really interesting part - getting key material later:

// Retrieve the previously saved credentialId
const savedCredId = /* retrieve your stored credentialId */;
const credIdBuffer = base64UrlToBuffer(savedCredId);

// Use a deterministically derived challenge
const fixedChallenge = new Uint8Array([ /* 32 bytes of fixed challenge data */ ]);

const publicKeyRequestOptions = {
  challenge: fixedChallenge,
  allowCredentials: [{
    id: credIdBuffer,
    type: "public-key",
  }],
  userVerification: "discouraged",
  timeout: 60000
};

navigator.credentials.get({ publicKey: publicKeyRequestOptions })
  .then(assertion => {
    // Extract the signature
    const signature = new Uint8Array(assertion.response.signature);
    
    // Combine with additional secret entropy
    // const combinedMaterial = concat(signature, additionalEntropy);
    // const rawKeyMaterial = await deriveRawKeyMaterial(combinedMaterial);
    
    console.log("Obtained signature for key derivation:", signature);
    // Continue with key derivation...
  })
  .catch(err => {
    console.error(err);
  });

function base64UrlToBuffer(base64UrlString) {
  const base64 = base64UrlString.replace(/-/g, '+').replace(/_/g, '/');
  const pad = base64.length % 4;
  const padded = pad ? base64 + "=".repeat(4 - pad) : base64;
  const binary = atob(padded);
  const buffer = new Uint8Array(binary.length);
  Array.from(binary).forEach((char, i) => buffer[i] = char.charCodeAt(0));
  return buffer;
}

But Wait, There’s a Catch!

Of course there is. When is crypto ever straightforward?

Some authenticators don’t produce deterministic signatures for the same input. They might:

Use non-deterministic signature algorithms (looking at you, ECDSA)
Embed signature counters in the output
Do other vendor-specific weird stuff

This means that your key material might change slightly between uses. Not ideal for encryption!

The solution is a fuzzy extractor — a cryptographic primitive that takes “noisy” input and reliably produces consistent output.

A typical fuzzy extractor has two phases:

Generate: Takes initial input, produces a key and a “helper”
Reproduce: Uses the helper to recover the same key from slightly different input

Security Considerations: The JavaScript Memory Problem

One important consideration: JavaScript memory.

Since all your variables in JavaScript are potentially accessible to attackers with debugging capabilities, your signature values could be exposed if someone can run a debugger or inject code through XSS.

Some mitigation strategies:

Keep sensitive variables around for as short a time as possible
Use Web Crypto APIs which operate outside typical JavaScript memory
Implement strict Content Security Policies
Clear sensitive data ASAP

And for extra security, consider:

Adding user-provided entropy (like a password) to your key derivation
Never, ever, ever storing sensitive key material in browser storage
Considering the security of the device itself in your threat model

Wrapping Up

So there you have it - WebAuthn hardware authenticators can be cleverly repurposed as key material providers for offline encryption. The private key stays safely locked away in hardware, your encryption keys become consistently derivable without network connectivity, and you get to feel like a crypto wizard in the process.

I’ve been using this technique successfully in a few personal projects, and while it’s definitely not the most conventional approach, it’s been surprisingly robust in practice. The combination of hardware security and fully offline operation makes it a winner for certain types of applications.

Give it a try in your next offline encryption project, and let me know if you come up with any clever improvements to the technique! Just remember - no crypto system is perfect, so always consider your specific threat model and use case.

P.S. Huge thanks to the WebAuthn standard creators - I’m pretty sure this wasn’t what they had in mind, but that’s the beauty of well-designed crypto primitives: they enable use cases the designers never even imagined!

Loading tmux when loading shell

Wed, 22 Jan 2025 00:00:00 +0000

This is something I’ve had mixed success with in the past. I’ve had things that worked but not quite in the way I wanted them to, and other things that just didn’t work! (Try sticking an exec before an error in .zshrc, and you’ve just nerfed your shell startup!)

There are a few things we need to consider, e.g. Do we want the same session as last time? What if there is no session? Can we create a new session but attach to an existing one if one by the same name exists? Do we want the same session when on SSH? Or when running a terminal emulator inside an IDE for example?

I’ve finally nailed it down to the following:

#!/usr/bin/env zsh

if [[ $TERMINAL_EMULATOR != "JetBrains-JediTerm" ]] && [[ -z $TMUX ]]; then
        if [[ -n $SSH_CONNECTION ]]; then
            TMUX_SESSION_NAME="tmux_$(echo "$SSH_CONNECTION" | md5sum | cut -d ' ' -f 1 | cut -c 1-6)"
        else
            TMUX_SESSION_NAME="tmux"
        fi
        if tmux has-session -t "$TMUX_SESSION_NAME" 2>/dev/null; then
            exec tmux attach -t "$TMUX_SESSION_NAME"
        else
            exec tmux new-session -s "$TMUX_SESSION_NAME"
        fi
    fi
fi

This script will start a new tmux session if one doesn’t exist, or attach to an existing one if it does. It will also create a new session if you’re on SSH, and it will not run if you’re using the JetBrains terminal emulator (as it doesn’t play nicely with tmux).

I have this in a separate script, ~/.load_tmux which I source from my .zshrc, right at the top of the file (so that it is the first thing that is executed):

source ~/.load_tmux
# rest of .zshrc...

Server status monitor bash script

Mon, 16 Dec 2024 00:00:00 +0000

When managing multiple servers, keeping track of their availability is crucial. Building on an example from “Cybersecurity Ops with bash” by Paul Troncone, I’ve developed a bash script that provides a simple yet effective way to monitor server status across multiple hosts. Let’s break down how this script works and why it can be a valuable tool for system administrators.

#!/usr/bin/env bash

# For a given list of servers, this script will check the status of the server and notify the user if the server is down.
#
# Usage: ./server_status.sh [-i <interval>] < server_list.txt

declare -i INTERVAL=30 # Default interval is 30 seconds

if [ ! -f /var/log/server_status.log ]; then
        # Check if user has permission to write to /var/log
        if [ ! -w /var/log ]; then
                echo "Cannot write to /var/log/server_status.log; hint: use sudo when running this script for the first time" 1>&2
                exit 1
        fi
        touch /var/log/server_status.log
        chown "$USER" /var/log/server_status.log
        chmod 644 /var/log/server_status.log
fi

while getopts ":i:" opt; do
	case ${opt} in
	i)
		INTERVAL=$OPTARG
		;;
	\?)
		echo "Invalid option: $OPTARG" 1>&2
		;;
	:)
		echo "Invalid option: $OPTARG requires an argument" 1>&2
		;;
	esac
	shift $((OPTIND - 1))
done

while true; do
	clear
	echo 'Server Status Monitor'
	echo 'Status: Scanning ...'
	echo '---------------------------------'
	while read -r server_hostname; do
		server_hostname=$(echo $server_hostname | tr -d '\r')

        if [[ $server_hostname == http* ]]; then
            curl -s -o /dev/null -w "%{http_code}" "$server_hostname" | grep -E '(000|404)' &>/dev/null
            if [ $? -eq 0 ]; then
                tput setaf 1
                echo "Server: $server_hostname is down - $(date)" | tee -a /var/log/server_status.log
                tput setaf 7
            fi
            continue
        fi

		ping -c 1 "$server_hostname" | grep -E '(Destination Host Unreachable|100% packet loss)' &>/dev/null
		if [ $? -eq 0 ]; then
			tput setaf 1
			echo "Server: $server_hostname is down - $(date)" | tee -a /var/log/server_status.log
			tput setaf 7
		fi
	done <"${1:-/dev/stdin}"
	echo ""
	echo "Press [CTRL+C] to stop..."

	declare -i i
	for ((i = $INTERVAL; i > 0; i--)); do
		tput cup 1 0
		echo "Status: Next scan in $i seconds      "
		sleep 1
	done
done

Script Overview

The script allows you to monitor the status of a list of servers, whether they’re URLs or hostnames, and provides real-time notifications when a server goes down. Here’s how it functions:

./server_status.sh [-i <interval>] < server_list.txt

Key Features

Flexible Input: The script can read server names from a file or standard input.
Configurable Scan Interval: You can specify a custom scan interval (default is 30 seconds).
Logging: Automatically logs server down events to /var/log/server_status.log.
Support for Both Hostnames and URLs: Can ping traditional servers or check HTTP status codes for web servers.

Implementation

Logging Setup

The script first checks if it can create and write to the log file at /var/log/server_status.log. If the directory isn’t writable, it provides a helpful hint to use sudo when running the script for the first time. This ensures proper permissions are set for logging.

Server Status Checking

The script supports two types of server checks:

# For traditional servers (hostnames)
ping -c 1 "$server_hostname" | grep -E '(Destination Host Unreachable|100% packet loss)' &>/dev/null

# For web servers (URLs)
curl -s -o /dev/null -w "%{http_code}" "$server_hostname" | grep -E '(000|404)' &>/dev/null

This dual approach allows monitoring of both network-level connectivity for traditional servers and HTTP availability for web services.

User Experience

The script provides a clean, interactive terminal interface:

Colorized output (red for down servers)
Countdown timer to next scan
Option to stop monitoring with CTRL+C

Usage Example

Create a server_list.txt with your servers:

example.com
https://mywebsite.com
192.168.1.100

Then run the script:

./server_status.sh -i 60 < server_list.txt  # Scan every 60 seconds

Considerations and Warnings

As with any system monitoring script, be cautious:

Ensure you have permission to monitor the servers
Be mindful of network load, especially with frequent scans
The script is best used in controlled, internal network environments
Unless the hostname starts with http, the script assumes it’s a traditional server and uses ping for monitoring; therefore, it may not work for all types of servers if they don’t respond to ICMP requests (I had to add a custom rule to an AWS security group to allow ICMP traffic for this script to work for my EC2 instances)

Conclusion

This bash script offers a lightweight, flexible solution for monitoring server availability. It demonstrates the power of bash scripting in system administration tasks, providing a simple yet effective tool for keeping an eye on your infrastructure.

Remember to always test scripts in a staging environment before deploying to production, and customize the script to fit your specific monitoring needs.

Happy scripting!