Changelog
Notable changes to mojo-http and the m0serve wheel, by version, in Keep a Changelog form.
Notable changes to mojo-http. Format follows
Keep a Changelog; versions follow
SemVer with the standard pre-1.0 caveat: minor
versions may break the API.
[Unreleased]
Fixed
- A Starlette streaming response no longer leaves a traceback per
response in the log (
bridge.mojo, the executor shim). Starlette -- so FastAPI and FastHTML -- produces aStreamingResponsebody inside an anyio task group, and the shim stamped its streaming mark onasyncio.current_task(), the CHILD. The request task's done-callback then took the stream for a buffered result and raisedTypeErrorunpackingNone(the body had already been delivered, so nothing failed but the log), and a client disconnect cancelled the child rather than the request. The mark and the cancellable task are now the slot's OWNER. Found by running FastAPI against the published wheel;test-shimgains the child-task shape plus its sabotage, andsmoke-fasthtmlrefuses the traceback in its log.
Added
- The live two-tab demo beside the docs (
apps/demo,deploy/demo,poe smoke-demo, SPEC M17; https://demo.m0serve.dev once the Fly app exists). One file of sync Django in the quickstart's shape, served bym0serve --realtime --workers 2from its own Fly app on a subdomain -- never a mount inside the docs app, so untrusted realtime traffic shares no process with the site -- and on ONE machine, because the publish bus is per process. The page holds an SSE stream and a WebSocket side by side, says which m0serve version serves it and which worker published each line, and carries what a public page needs that the tutorial does not: channels namespaced per visitor by a random cookie token (a stranger's tab hears nothing), 280 bytes a message (413), 30 a minute per visitor per worker (429 withRetry-After), a foreignOriginrefused on the upgrade, binary frames dropped, nothing stored.scripts/demo_probe.pyproves every one of those from outside -- against the image built from the tree's wheel in thepid1job, with m0serve as PID 1 anddocker stopdraining with a stream held, and against the live URL in the deploy workflow's newdeploy-demojob (secretFLY_DEPLOY_DEMO). Four sabotages of the application -- shared channel, no rate limit, no Origin check, no size cap -- each failed the probe in the phase that names the guard. - The headline claim, gated clause by clause (SPEC I20, K11, M16;
QUICKSTART.md§7–8,poe smoke-flask-realtime). The sentence names Flask, and nothing held a stream or a socket from a Flask view: K10 proves plain WSGI. The quickstart now carries the same four views in Flask, served with--workers 2and checked with curl, andsmoke-flask-realtimeextracts that file from the page itself and drives it with the Django rows' RFC 6455 probe, one stream and one socket pinned per worker. Gating it found the Flask-specific line the claim was missing: Werkzeug's router answers 400 to an upgrade request on an ordinary rule before any view runs, so the socket route is declaredwebsocket=True. "No second process" and "no dependencies" are asserted rather than stated (exactly a supervisor and two m0serve workers bypgrep -x, an emptyRequires:frompip show), and the README's "degrades under gunicorn" sentence is executed: the Django file served by gunicorn answers the hold views as short plain responses inside curl's deadline, the upgrade as 200, andpublish()reports 0 workers without raising. The published wheel was run through the whole page from a scratch directory on macOS and in apython:3.12-slimcontainer, which is the fresh-project check the sentence was owed.
Changed
docs/WSGI_VS_ASGI.mdis the concise answer to why there are two execution modes, in a page: what each mode is, why one would not do, what WSGI gets from the handler pool and held connections, what ASGI gets from the executor, what free-threading changes, the cliffs, and how to choose. The dated essay it replaced, whose opening still made "the case for not building an ASGI host", is kept as written atdocs/notes/wsgi-vs-asgi-history.mdwith its section numbers, and every citation of §5, §8 and §9 (CLAUDE.md, the README, the conformance and performance pages, the notes) now points there.- The documentation, restructured for a first-time reader and for an
agent (
scripts/docsite.py,docs/ROADMAP.md,docs/notes/,docs/RUNNING.md,docs/README.md,llms.txt,apps/site/home.md). The site now opens on a short home page rather than the repository's README, and its pages are grouped by intent: start here (quickstart, the new Running m0serve guide covering flags, the execution modes and when each applies, proxies, shutdown and exit codes, the capability matrix, the map), understanding the design, measurements, the project record, and the Mojo framework underneath. ROADMAP.md is a state page now: milestones, known issues, planned, not planned and why, recently resolved, at 380 lines where it was 2,726; its twenty-one long-form narratives (the Django server aims, each shipped subsystem, the gates, the open questions, the post-mortems) are design notes underdocs/notes/, kept as written, rendered under/notes/with their own index and picked up by the site without a page-table entry. Every page opens with a one-sentence lede, the description a search result shows; pages with six or more headings get an "On this page" list. For agents,llms.txtis curated (the operating contract, then the essential pages) with the rest under the spec's## Optionaltier, andllms-full.txtcarries only the essentials, at a fraction of its former size, naming what it omits and where to find it.poe milestones, the spec checker and the link checker read the new shape unchanged.
Fixed
- A recycled slot no longer inherits the previous WebSocket's accept.
The executor shim's
spawn_wscleared a recycled slot's stale disconnect mark but not its_exec_ws_acceptedmembership, and the previous task's done-callback — correctly, no longer the owner — does not clean the slot up either. A new handshake landing on a slot whose previous connection was an accepted socket therefore looked pre-accepted: an application that returned without answering it sentws_closeinstead ofws_reject, so the held 101 was never released and the client hung against a clean server log; a pre-acceptwebsocket.sendwas silently tolerated instead of raising. Found by auditing the bridge's slot-recycle hygiene;shim_ownership.pygained the WS→WS recycle test (test_a_websocket_recycle_forgets_the_predecessors_accept) and its sabotage, which fails exactly that test on the pre-fix shim.
0.17.0 — 2026-09-02
Added
-
ASGI on a free-threaded CPython build is refused, not crashed (SPEC L18, E10; ROADMAP Known issues). The weekly py-canary found the asyncio executor segfaulting on 3.14t while building its
ExecutorPortPython type: Mojo 1.0's stdlib laysPyObjectout for the GIL build, soPyModule_Createmisreads the module definition (modular/modular#5726).m0servenow probes the build wherever the executor would engage -- prefork's worker, the threaded path,--doctor-- and exits 78 with a sentence naming the issue and the fix (a GIL-enabled interpreter with--workers); an ASGI app under--threadsis therefore refused on this toolchain.WorkerSupervisortreats a worker's exit 78 as the refusal it is: no respawn, and the supervisor exits 78 itself, where before ten respawns and an exit 1 reported a crash.smoke-django-realtimephase 6 asserts the refusal on a free-threaded build (alone, via--doctor, and under--workers 2) and the full mixed server on a GIL build.py-canary.ymlno longer fail-fasts (Linux was cancelled before reaching the failure, twice) and can file its issue (the default token was read-only, so it never had). -
The documentation site's deployment (
deploy/site/,poe deploy-site,poe smoke-site-image; SPEC F14). Apython:3.12-slimimage with the m0serve wheel, the site rendered forhttps://m0serve.devand the fallback application, on one always-on 256 MB Fly.io machine (deploy/site/fly.toml);.github/workflows/deploy-site.ymldeploys after every successfulRelease, pinning that release's wheel, or on demand with a version.scripts/site_image_probe.pybuilds the same Dockerfile from the tree's own wheel in CI and asserts the served shape through a published port, that m0serve is PID 1, and thatdocker stopis the drain. The PyPI project now linksDocumentationto the site. The first public deploy follows the next release: the XML sitemap and the fallback application's redirect and 404 need thexmlcontent type and the static mount's fall-through above, which 0.16.0 does not have; the probe passes everything else on that wheel and fails at exactly that phase. -
The documentation site (
scripts/docsite.py,apps/site,poe build-site/serve-site/smoke-site; SPEC F13). README, QUICKSTART, CHANGELOG, PROVENANCE and every page underdocs/render to HTML and are served by m0serve itself through--static, withllms.txtat the root (the repository's own, its links made absolute, plus an index of every page) andllms-full.txtbeside it for agents, a Markdown twin beside every page advertised by<link rel="alternate" type="text/markdown">,sitemap.xml,robots.txtandspec.jsonat stable URLs, and canonical links, Open Graph and JSON-LD on every page. Titles and descriptions live in one table and are written for the question a reader searched, not the file's name. Every relative link is resolved at build time and a link to nothing fails the build; everydocs/*.mdmust be listed, so a page cannot ship without a title. The link check is standard-library and runs insidecheck-docs, so doc-only pull requests get it, and--selftestproves it can fail. The rewriter is a fence-aware regex; the build then walks the parsed token stream and refuses any relative link it missed, which found one on the first run (a link whose text wraps a line).xmljoined the static mount's content types asapplication/xml, because the sitemap was going out as octet-stream. Deployment is the open half: the build takes--base-url, and nothing serves it publicly yet. -
The scheduling-stickiness sighting reproduced, and its fix direction corrected (
docs/ROADMAP.mdKnown issues,scripts/accept_placement.py). The one CI failure ofsmoke-reload's two-worker phase — eighty sequential connections all answered by one of two live workers — is CPU placement, not load: with the client on one worker's CPU the other worker wins every accept (80 of 80, measured on the 0.16.0 wheel in a Linux container), because the accept-queue wakeup runs on the client's CPU and the co-located worker is last to run.EPOLLEXCLUSIVE, which the entry named as a fix direction, sends 80 of 80 to one worker in every placement; per-workerSO_REUSEPORTlisteners are the only shape that balances. The entry now records the numbers and why the server keeps its shared listener.smoke-reload's two-worker phase no longer asserts scheduler fairness: instead of waiting for both pids to happen to answer, it stops the worker that did (SIGSTOP; the supervisor reaps withWNOHANG, so that is neither a crash nor a respawn) and requires the other to serve the new module 8 of 8, both pids tied to the supervisor's re-fork log. Sabotaged in both layers; passes 4 of 4 on Linux against the 0.16.0 wheel, including with the whole smoke pinned to one CPU. -
The 0.16.0 real-application soak (
docs/REAL_APP_VALIDATION.md, rewritten; the milestone's soak reads current). Four applications —transcripts,color-separation,textshelfand Wagtail'sbakerydemo— driven byscripts/soak.pyagainst captures from gunicorn, uvicorn and daphne: 373,000 responses byte-identical across the clean rows, with logins, 9.7 MB multipart uploads, abandoned holds, four-worker prefork, and SIGTERM churn. One server defect found and left open as SPEC D9: a request body still arriving at SIGTERM holds the drain to its 5 s deadline, because the drain loop reads nothing new;scripts/drain_upload_probe.pyreproduces it bare and is the gate the fix will land with. Everything else that differed was traced, by measurement, to the application: Wagtail rendering from sets under different hash seeds, typst's per-process font tags and PDF dates, textshelf's SSE views stalling any WSGI pool and its unpooled Postgres connections at four workers. Manifests for both apps, multipart uploads, status-only routes and supervisor-aware sampling in the driver. -
The soak driver —
scripts/soak.py, manifests underscripts/soak_manifests/,poe soak-apps(pre-release, three legs) andpoe soak-selftest.docs/REAL_APP_VALIDATION.md's phase 5 was a request loop that sampled RSS; three of the six real-application defects were silent (a clean status over a short or empty body) and a request loop passes every one. This asserts bytes instead: every response is compared — status, normalised headers, body digest — against a capture recorded from a reference server (--baseline, gunicorn or uvicorn), under five concurrent populations (keep-alive bursts that cross the cap on every connection, streams, uploads and logins, WebSocket echoes, and abandoners that vanish mid-body and reuse the freed slot at once), with the server SIGTERM'd or--reloaded underneath (--churn-every), and the server's own/__metricssampled beside RSS, fds and threads. A manifest'sloginblock is a CSRF form round trip with a cookie jar per session, so the authenticated surface of a real application is reachable and the login response itself is verified —Set-Cookienormalised to its attributes, defect 1's exact shape. The comparator is a pure function with a--selftestthat found the driver's own first hole: a substitution greedy enough to absorb a truncation blinds the instrument, so a capture now refuses any route its patterns blind, and carries a fingerprint of the rules it was recorded under. Shaken down onapps/hybrid_mixandapps/asgi_bare(3.5 M responses against a uvicorn capture, zero differences), then on Wagtail's bakerydemo and textshelf, each byte-identical to gunicorn/daphne with logins and churn — every body difference traced, by measurement, to the application rendering from Python sets under different hash seeds. -
SIGTERM as PID 1 in a container is gated, every pull request (SPEC M11, the last beta row).
docker stopis SIGTERM to PID 1 and nothing else, and PID 1 gets no default signal dispositions from the kernel — a SIGTERM arriving with no handler installed is discarded, not fatal. This server installs its handlers post-fork by design, so every in-process SIGTERM gate proved the handler works once installed while proving nothing about the one environment where the default disposition cannot paper over a missing install.poe smoke-pid1(scripts/pid1_probe.py) runs the shipped wheel exec'd as PID 1 inpython:3.12-slim— checked via/proc/1/cmdline's argv[0], not trusted — and stops it in both process shapes: one process alone, and the supervisor reaping two workers whose exits must be clean rather than by the propagated signal. The sabotaged shapes were each measured in a container before the gate counted: the worker install skipped is SIGKILL at the deadline (exit 137 at 10.1s of a 10s grace) alone and workers dying by signal 15 under a supervisor; the supervisor's arm skipped silently is the deadline again with the single shape green; the announced degradation is caught by its own log line. The probe's PID 1 premise check failed its first sabotage —sh -c's cmdline contains "m0serve", so a substring check over the whole cmdline blessed the shell — and checks argv[0] alone for that reason. -
/__metricsrenders a request-latency histogram (SPEC F5). Six log-spacedlebounds — 100µs, 1ms, 10ms, 100ms, 1s, +Inf — as a standard Prometheus histogram (_bucket/_sum/_count), integer-only and O(1) on the event loop thread: one band counter incremented per response, accumulated cumulatively at render time. Sampled in_after_sendfrom the same clock the access log reads, and only when a header stamp exists, so a pushed frame on a streaming slot cannot sample the epoch as a latency. Per-loop like every other metric; the scraper aggregates. The serve smoke's metrics phase now checks coherence throughscripts/histogram_check.py— the documented bounds in order, cumulative counts non-decreasing,le="+Inf"equal to_count, and a_countcovering the phase's own requests, that last being the assertion that fails when recording is never called. The checker's selftest runs in the same phase (six doctored expositions, each flagged by the rule that names it), and three sabotages were caught by name before the gate counted: recording never called (_count is 0 after at least 5 requests), the cumulative render broken (counts decrease: [5, 1, 1, 0, 0, 0]and+Inf is 0 but _count is 7), and anleboundary off by one (twotest_metrics.mojotests). 6 unit tests pin the boundary math. -
Autobahn|Testsuite is wired to the pre-release cadence (SPEC I13).
poe autobahndrives the suite's sections separately (a single pass wedges on the slot a cap-killed connection just released), skips 9/12/13, and compares both directions against the pinned baseline — 240 of 247, every failure being I17's ≥64 KB outbox cap: a failure outside those seven cases is new and fails the run, and one of the seven passing fails it too, the cap having moved out from under the sheet. The image is version-pinned (digest-identical to the 2026-08-30 baseline's), which is what lets the per-section case counts be asserted exactly; the server is the runner's own pure-echo ASGI app, becauseasgi_bare's/wsprefix-echoes text and Autobahn's byte-identity cases would score that as failures. The comparator's--selftest(five doctored result sets, each flagged by the rule that names it) runs before anything is believed. The wired run reproduced the baseline exactly, and the live sabotage — I16's close-code validation reverted — was caught as nine named new failures (7.9.1–7.9.9) on the first section-7 run.docs/RELEASING.mdnow lists it besidestress-asgi, along withfuzz-request-longandsabotage-outbox-cap, which were pre-release tasks the checklist never named. -
Coverage is declared by the gate, not merely cited by the spec sheet (SPEC F12; ROADMAP "Traceability", phase 2). Every one of the 119
verified (every PR)rows now declares its coverage in its own gate: acovers: A7line in the cited test's docstring for the 39 unit-cited rows, and ascripts/emit.py --covers A7call in what the cited step runs for the 80 step-cited ones — the latter also recorded by the real run through$M0_RESULTS, rendered in the CI summary as a tally. Two new checker rules run beside the citation rules: every declared id must name a row that exists, and every gated row's declaration must AGREE with its citation — a row declared only somewhere its evidence does not cite is the exact mis-citation class the 2026-08-30 audit found six of, now a red build instead of an audit finding. Weekly and pre-release rows keep declared-static citations (their runs are absent from PR CI); the citation-shape rules stay, guarding what declarations cannot (real cadences, unconditional steps, the two closed sets). Four new sabotages revert the rules, each caught by the failure that names it — and the migration itself surfaced a masking hazard: appending a recorder call after a smoke body's last command replaces the exit status poe reads, sosmoke-poolandsmoke-ws-inbound, whose final probe's status was the task's status, now carry an explicit|| exit 1there. -
The request decoder is fuzzed, every pull request (SPEC G13).
scripts/fuzz_request.mojomutates a seed corpus of real and hostile requests throughparse_request_headersandHTTPChunkedDecoder.decode— no socket, no server, because the decoder is a pure function over bytes. Deterministic from its seed, so a CI failure names the seed and iteration and the same run reproduces it.Beyond "does not crash" it asserts four properties: parsing is deterministic; an INVALID request cannot become valid by appending bytes (the smuggling-relevant one — "invalid, not incomplete" is what stops an attacker's payload being read as the next request); a request that parses is unchanged by bytes after it, consuming the same count; and the chunked decoder's
ret, decoded length andpending_bytesall index the buffer they were given, since those feed copy sizes in the loop.480,000 mutations across eight seeds found nothing, which is a believable result for a decoder with this unit suite and worth nothing on its own — so two things guard the negative. The run refuses to pass on thin coverage (it counts parsed, rejected, incomplete and both chunked outcomes, and fails if any bucket is empty), and
poe sabotage-fuzzbreaks each invariant in the decoder and requires the fuzzer to report that invariant by name. Without them "no findings" and "checks nothing" are the same output.poe fuzz-request-longis the release sweep (8 seeds x 250k). -
The chunked decoder's trailer states are gated (SPEC A10). The servers build their decoder with
consume_trailer = True— which is what makes a body end where RFC 9112 says it ends — but the round-trip tests set that flag over a wire carrying no trailer section, so every state belowIN_TRAILERS_LINE_HEADwas reached by no test at all.Eight tests in
test_parsing.mojonow cover the section: consumed whole (including its terminating CRLF, whose absence is what makes a close send an RST), several fields, no trailer byte reaching the decoded body, the framing fields RFC 9110 §6.5 says a trailer must not honour (Content-Length,Transfer-Encoding,Host), the pipelined tail surviving byte for byte, and the section bounded by the existing abuse ratio — trailer bytes advancesrcand neverdst, so they are charged as pure overhead and no second limit is needed. The default-setting half is asserted too, so a decoder that swallowed to the end of the buffer cannot pass.poe sabotage-trailersreverts each of the six rules and requires a failure for every one; it runs intest-alland in CI. Nothing was found wrong with the implementation — the gap was in the evidence.
Changed
-
Three renames the next Mojo release forces, applied now because their replacements already compile on 1.0.0:
InlineArray→Array(header.mojo,test_sendfile.mojo),std.ffi._CPointer→OptionalPointer(bridge.mojo's twoPyBytes_*signatures) andmemcpy→unsafe_memcpy(four fork files). Verified by building the tree on1.1.0.dev2026090205in an isolated copy: with these plus the two renames that cannot be applied ahead of time (Atomic[DType.X]→Atomic[X],_CTimeSpec.tv_subsec→tv_nsec),build-all, all 1011 Mojo tests,build-apps,build-serveandsmoke-djangoare green there. ThePythonObjectleak entry indocs/ROADMAP.md(Known issues) now records the upstream issue and fix commit (#6833,c9d5048575, authored nine hours after the 1.0.0 wheel was uploaded), the leak measured per operation on both toolchains, and the verified break list in place of the one read from the release notes, which was three items short and one item stale. -
A static mount's miss falls through to the application.
StaticFiles.serve(and so--static) used to answer every path under its prefix definitively — a missing file was the mount's own JSON 404 and a POST anywhere under it a 405 — which made--static /=dirswallow the application entirely. It now answersNonefor a path that names no regular file under the root, so a root mount can front an application's routes (thetry_files/ whitenoise shape); a missing asset under/static/now gets the application's 404 page rather than the mount's JSON. What still never reaches the application: a traversal, an encoded slash that would open a segment, a malformed segment — those stay the mount's 404 (G5, G6). The 405 for a method other than GET/HEAD is now about a file the mount holds, checked after existence. Found by the documentation site, which needs the redirect for/docs/specand its own 404 page from the application behind the mount. -
poe autobahnprovisions its own docker on a Mac. With no daemon answering it starts a 4 GiB colima VM (enough for the wstest container; the echo server runs on the host) and stops that VM when the run ends, pass or fail. A daemon that was already up — colima started for other work, or native Linux docker — is used as found and never stopped: only what the run started is the run's to reap. Both branches measured: a stopped VM is started at 4 GiB (colima start --memory 4resizes the existing profile down from 8) and reaped after the suite; a running one is left running. The sizing matters on a 16 GB machine, where the forgotten 8 GiB reservation was half the RAM. -
Bench prose numbers are now generated in place, not pattern-matched after the fact.
check_bench_proseheld 24 hand-written regexes against 12 quantities across three documents — every legitimate rewording broke a pattern, only a phrase's first occurrence was checked, and nothing proved the checker could still fail. It is replaced by inlinenum:spans:render_bench_docs.pycomputes each quantity from the newest artifacts and writes the number between markers naming the quantity and its decimals (~1.2x), so the sentence around it stays free to be reworded and--check— already run bypoe check-docs— refuses any stale span. A span naming an unknown quantity, a quantity whose artifact row vanished, or an opener whose closer was deleted is an error, not a skip, and the renderer's new selftest (run at the top ofcheck-docs, so doc-only pull requests prove it too) insists each of those failures fires. All 25 span sites were migrated value-neutrally — every number the prose showed is byte-identical to what the newest artifact computes — and the migration corrected one overstated claim found along the way: WSGI_PERFORMANCE.md said the old checker held the mixed-workload prose to its artifact, but that artifact records throughput medians only and no checker ever recomputed the p99 narrative; the page now says which numbers are held to the file and which are quoted measurements. -
SO_REUSEPORT(SPEC D6) now records the property that is actually gated, and M13's reason is corrected. D6 claimed the option itself and satimplementedfor want of a smoke covering "the zero-downtime handover it would enable" — but no shipped path can enable it:reuse_portis opt-in onListenConfig, defaults off, and has no caller, no flag and no environment variable, because workers and threads all accept from ONE listener bound before the fork. The property that matters and IS gated is the default:smoke-serveproves a second server on a busy port fails to bind loudly rather than silently taking a share of the connections, which is what it did until 0.14.0.M13 (systemd socket activation) was
out of scopebecause "SO_REUSEPORTcovers the restart case it is usually wanted for". That was not true for anyone runningm0serve. The row keeps its status on the honest reason — nobody has asked for it — and now says what a restart does get, which is the supervisor's graceful drain. This is the class of error the checker cannot catch: it validates that a citation resolves, never that a reason is true. -
The blocking
listen_and_serveloop no longer sends aKeep-Aliveheader. It was the only site that did — the event loop, which every shipped binary runs, has never sent one — so the two paths disagreed on the wire for a header that is not in RFC 9110 or 9112 (RFC 2068 §19.7.1 described it; RFC 2616 dropped it) and that browsers ignore.It was also unreachable and wrong. Nothing in this tree calls
listen_and_serve, no test asserted the header, and themaxvalue was off by one:maxcounts ADDITIONAL requests, andkeepalive_countis not incremented until after the response is built, so request 99 of 100 advertisedmax=2and served one more.Aligning the other way would have put a header nobody reads on every keep-alive response of the hot path, plus a spec row and a permanent gate. If a client pool ever needs
timeout=to avoid racing the idle close, the event loop is where to add it, with a row and a gate.listen_and_serveis public API (README), so this is a visible change for a library caller that was reading the header;Connection: keep-aliveis unaffected.
Fixed
-
The nightly canary could not alert, and would have misreported its first success.
nightly-canary.ymlfailed on all three scheduled runs (2026-08-18, 08-25, 09-01) and filed no issue:gh issue create --label nightly-breakagefailed because the label did not exist. It does now. Andtrailer_sabotage.py,fuzz_sabotage.pyandpool_sabotage.pyran the compiler asuv run mojo, which re-syncs the venv touv.lockeven under a parentuv run --no-sync(measured: the child printed Mojo 1.0.0 and the venv stayed there) — so thesabotage-trailersstep oftest-allwould have swapped a canary back to stable mid-run and the next step's "precompiled file is newer than the compiler" would have read as a nightly break. They run the venv's ownmojonow; on the nightly copy the step passes and the toolchain stays put. -
A request whose body was still arriving at SIGTERM held the drain to its 5 s deadline (SPEC D9). The drain loop dispatched writes only and read nothing new, so a half-received upload was neither completed nor closed: the client was reset at 5.03 s and the process left at 5.09 s — half of
docker stop's patience for a request that completes in milliseconds. The same loop cut any response too large for onesendat its first write readiness. The drain now runs ordinary event-loop passes for its budget, with_close_between_requestsafter each so only bytes already sent are served; gated bysmoke-drain-uploadin both execution shapes, two-sided (answered whole, exited inside 3 s), and sabotaged by restoring the old loop. Found by the soak driver's uploads population on color-separation. -
A chunked body that arrived with its headers was bounded by nothing. Both request-body limits — the decoded cap and the raw ceiling at twice it — lived in the
READING_BODYbranch, but a chunked body whose bytes arrive in the same read as its headers is decoded and dispatched inline at the header site, which never runs that branch. Sending head and body in one write was enough to escape both: 512 one-byte chunks are 3,077 raw bytes against a 2,048 ceiling and answered200, where the same body paced across several writes answered413.So whether a request was bounded came down to how the client's writes happened to be coalesced by the kernel — which is also why this survived:
chunked_overhead_probe.pysends 512 separate 6-byte writes, and they coalesce differently on a loaded CI runner than on a laptop. It surfaced as an unrelated pull request going red on macOS.The probe now sends the over-ceiling body three ways — paced, head and body in one write, and split across two writes at the ceiling — so the shape that reaches each decode site is chosen rather than left to the kernel. Reverting the fix fails the one-write phase and no other, which is how the fix was scoped: a second check added in the
READING_BODYbranch turned out to change no observable behaviour, because the existing pre-decode buffer-size test already refuses those shapes, and it was removed rather than shipped unpinned. -
The keep-alive request cap destroyed the response it fired on, for a streamed body and for a WebSocket upgrade alike. On the hundredth request of a keep-alive connection (
max_keepalive_requests, 100) a streamed ASGI response went out as a200carrying itsContent-Lengthand zero body bytes, and a WebSocket upgrade completed its handshake with101and then never sent a frame. Both were silent — correct status line, nothing logged._finish_responseclearsshould_closefor a stream and for a 101, because neither is keep-alive reuse: each owns the connection until it ends. The cap check below those two branches was guarded bynot should_close, which is exactly the state they had just established, so the two shapes that had opted out were the two it caught;_after_sendthen closed the slot as soon as the head drained, before any body frame arrived over the executor's chunk channel. The blocking loop cannot reach this —gate_streaming_responseturns both shapes into a 409 before the cap is consulted — and it is left unchanged rather than given a condition that can never be false.Found by re-soaking a real Django application against 0.16.0 (
docs/REAL_APP_VALIDATION.md): 9 truncated responses in 6,000 requests, all on one 124 KB static file, at intervals of exactly 700 — every hundredth time that route was hit. Gated bypoe smoke-keepalive-cap(SPEC A3), whose third phase asserts the cap still fires for an ordinary response, so the gate cannot pass on a build whose cap never fires.
0.16.0 — 2026-08-31
Four WebSocket correctness fixes, two of them silent — a slot leak and a message loss that no client could detect. Three were found by gating a mechanism nothing gated, which is the theme of the release rather than a coincidence.
Fixed
- A WebSocket peer that never answers Close no longer holds its slot for
ever. v0.15.1 made the server wait for the peer's Close reply (RFC 6455
§5.5.1) and bounded that wait with the idle sweep. The bound did not work:
none of the linger's four sites is a transition, so the drain re-stamped
the deadline on EVERY pass while a slot lingered — about once a second,
two seconds into the future — and the sweep could never overtake it.
Measured: a peer that received Close and never replied still held its slot
at 40 s. Armed once now, only when the deadline is still zero.
A4is gated (--idle-timeout,scripts/idle_timeout_probe.py,poe smoke-idle-timeout) andL16is a new row for the BOUND, separate from L15 for the ORDER, because L15's 64-way concurrent-close phase passes on both broken servers. - Close frames are validated, not echoed. RFC 6455 §7.4.1 divides close codes into ones a peer may put on the wire and ones it may not; the parser copied the first two payload bytes into its echo unexamined. The contradiction is sharpest at 1006, "abnormal closure", which names the ABSENCE of a close frame — so a close frame carrying it cannot be honest, and the server answered it with its own 1006. Now 1000-1003, 1007-1014 (1012-1014 were registered with IANA after the RFC, hence the range ending at 1014 and excluding 1015) and 3000-4999 are echoed; everything else fails the connection with 1002. A one-byte body is a protocol error (§5.5.1), and a reason that is not valid UTF-8 is 1007 (§8.1). Autobahn section 7 goes 24 OK / 3 informational / 10 FAILED to 34 / 3 / 0.
- Inbound WebSocket messages are no longer dropped when a client stalls.
The outbound direction was credit-gated and the inbound direction had no
backpressure of any kind, so once the executor's submit channel filled,
each further message was discarded with a log line the client can never
see: 2932 of 3000 lost at 4 KB. The two directions were coupled, which
is why the threshold was so low — an app that awaits
sendinside itsreceiveloop stops receiving when its client stops reading, which stops the drain that inbound messages depend on. Inbound now has a window (WS_IN_WINDOW, acked cumulatively as the application consumes), the loop suspends the socket's READ when it cannot forward, and what it has already taken off the wire is parked and delivered late. A parked message is owed, never dropped. - A WebSocket client sending more than one socket read's worth no longer
stalls on Linux. The WebSocket read path took one
recvper readiness event and never re-armed —A13's defect in the one path nothing had ever sent a large inbound burst to. kqueue's level trigger hides it entirely; on epoll the edge is spent, and the stall needs the client to STOP SENDING, which is exactly what the inbound window above makes it do. So the fix above exposed a bug that had been waiting for it. Re-armed only on a full staging buffer, so an ordinary small-message socket pays no extra syscall.
Added
--idle-timeout, exposing the connection idle deadline that was previously aServerConfigfield with no flag or environment variable.poe smoke-idle-timeout,poe smoke-realtime-holds,poe smoke-ws-inboundandpoe check-phase-stamps— four new gates, each sabotage-proven.- Every probe now stamps the PHASE it was proving, so a traceback naming a
shared socket helper says which phase failed rather than only which call.
scripts/phase_stamp_check.pyholds it across all 16 probes and reverts each rule to prove the checker bites.
Changed
- A client that sends without ever reading, against an app that echoes, now BLOCKS rather than losing data. That is the correct end of a deadlock every echo server has, uvicorn included; the old behaviour only avoided it by discarding the client's messages.
docs/SPEC.mdgrew to 149 rows. Autobahn|Testsuite was run once by hand to decide whether to wire it (I13): it scores the build with a known RFC 6455 §5.5.1 violation and the fixed build IDENTICALLY, because its fuzzing client always initiates the close and the bug was on the app-initiated path. The ROADMAP's claim that "the bar is unambiguous and the result is comparable" is withdrawn. It still found the close-code defect above, and outside its performance section now scores 240 of 247 — every remaining failure being the deliberateMAX_PENDING_BYTEScap (I17).B8split intoB8(h2spec) andB9(the PortSwigger desync scanner), bothout of scope: h2spec needs HTTP/2, whichA18refuses, and a pair-scanner has nothing to compare against a server with no proxy in front of it.
0.15.1 — 2026-08-30
Fixed
- A WebSocket the server closes now ends in a FIN, not an RST. When an
application sent
websocket.close(1000), the loop wrote its Close frame and closed the TCP connection in the same pass — before the peer could reply. The reply then reached a socket that was already gone, TCP answered with an RST, and that reset flushed the peer's receive queue, taking the FIN with it and, for a client far enough behind, the Close frame itself: against thewebsocketslibrary at 200 concurrent closes, 33 of 200 sawConnectionClosedError: no close frame received or sentinstead of the application's own code 1000. The loop now follows RFC 6455 §5.5.1's order — having sent Close, it waits to receive one, bounded by a 2 s deadline so a peer that never replies cannot hold the slot. Measured after: 0 resets at 20, 50, 100 and 200 concurrent closes, and 200 of 200 cleancode=1000from the real client.ws_probe.pygained a close-order phase (64 concurrent closes, all required to end in a clean FIN) which reports 2 of 64 against the unfixed server.
Changed
poe stress-asgicovers the WebSocket path, in both loop modes. The pre-release timing gate drovechunked_keepalive.pyonly, so the seam a 2026-08-30 CI flake landed in — the WS path, the loop inversion and CPU contention together — was gated by nothing. Each round now runschunked_keepalive.pyand thenapps/asgi_bare/ws_probe.py, so the handshake lands on the slot the streamed connection just released, and the whole loop runs on the pump and again underM0_INVERTED=1(asserted from the banner, not assumed from the variable), atsmoke-asgi's 300 ms heartbeat so a timer is queueing frames into the outbox the application is filling. A failure names its mode, round and probe, andM0_STRESS_MODES=invertedreruns just the half that failed. Reverting thewebsocket.sendcredit gate fails the new gate on round 1 — 15 of 400 frames — and passed the old one 30 of 30. It did not reproduce locally (150 rounds per mode across three runs, up to 40 CPU hogs on 10 cores, all green) but did reproduce on CI, where the probe's new phase stamp named it at once: see "A WebSocket close races the peer's close reply" under ROADMAP's Recently resolved — it was diagnosed and fixed in this release.ws_probe.pyreports the phase it failed in. The CI failure was an unhandledConnectionResetErrorwhose traceback namedrecv_exact, a helper four phases share. A reset is now a finding carrying its phase — and, being anOSErrorrather than anEOFError, it no longer bypasses the flood phase's frame-count diagnosis in silence. It earned its keep immediately: the next occurrence named the app-initiated close handshake, which is a different phase from the one two investigations had assumed, and is what identified the underlying bug.
0.15.0 — 2026-08-29
The Mojo-native release: the tier for handlers written in Mojo grows the
ergonomics it lacked, and the HTTP layer under everything gets measurably
faster. Nothing on the wire changed — every existing response is
byte-identical, which the parse-sensitive smokes assert — and nothing an
application already does breaks: HTTPService implementers that spell
out all nine methods keep compiling.
What is visible to an application:
- A handler is
funcand nothing else.HTTPService's other eight methods carry defaults, soapps/hellois 30 lines instead of 57 and 268 lines of empty stubs left the tree. Adding a defaulted hook no longer breaks every implementer — which is howdirect_jobarrived. m0_http.reply(json,html,problem,redirect,empty,no_content,body_string,param_int, …) andRouter.allow_header— the helpers three apps had each rewritten.MojoPool: the handler pool for Mojo handlers, so one blocking Mojo handler no longer stalls the loop (fast-route p50 405 ms → 0.3 ms with two 400 ms blockers in flight).M0_INVERTED=1, experimental: the Mojo loop inside the executor's asyncio loop on one thread. Correct under every gate, −14% CPU at low concurrency, not the default — the CHANGELOG entry below says exactly why, with the numbers, and the one limitation to know before trying it (a request mid-await at SIGTERM is answered at the 5 s drain deadline).docker stopunder traffic costs about the slowest in-flight request, not 5 s: a keep-alive connection answered during the drain no longer holds the drain to its deadline.
And under it all: the request parser went from 1.96 to 0.86 µs and the
per-pass outbox sweep is skipped while nothing streams, which together
take apps/hello from ~122k to ~157k rps at c16 on the reference
machine (+29%), the ASGI executor from 55.7k to 60.1k on the benchmark
page's row (1.03x uvicorn --loop asyncio, above it for the first
time), and M0_INVERTED=1 to 62.0k. One wrong answer was fixed on the
way: a bare-LF line ending followed by a CR within 64 bytes used to
swallow the next header.
The one API that moved is inside the fork: lightbug_http's
HTTPHeader is four offsets into the parse buffer rather than two
Strings. Nothing outside packages/m0-http imported it.
Changed
-
The per-pass outbox sweep is skipped while nothing streams — except under the pump. Every pass swept all 1,024 slots for a streaming connection to drain, and the miss path alone cost 1.2 µs per pass.
OffloadLoopState.streaming_hint— an upper bound raised by the two sites that set a stream flag and recounted by the sweep itself — now gates it:apps/hello+3.3% at c16 (152.3k → 157.4k rps, +5.5% per core),M0_INVERTED=1+4.6% (59.3k → 62.0k). The pump's loop keeps sweeping every pass (OffloadPool.sweep_every_pass, set by its wiring alone), because measured without it the pump lost 2.9% rps at +6% CPU at c16: the microsecond was accidental pacing — a loop thread that parks sooner batches fewer submits and wakes the executor more — and ±0 at c256. Guarded by the streaming smokes, sabotaged three ways (never sweep; either flag site not raising the hint), each failing exactly the smoke it should. Artifacts underbench/results/outbox-sweep/. The follow-up the finding named — an explicit pause in place of the accidental one — was measured the same day and recorded, not built (ROADMAP "Pacing the pump's loop thread"): a 1.2–2 µs spin on every pass reproduces the sweep exactly, a spin only before a partial flush does nothing, and re-polling to merge batches is worse; the pump keeps the pacing it already has. -
The request parser is under a microsecond.
parse_request_headerson the twelve-header browser GET: 1.96 → 0.86 µs, the whole user-space request 3.33 → 1.97 µs (−40%),find_header_end45 → 16 ns, andOK()construction 0.66 → 0.52 µs as a side effect (scripts/bench_http_parts.mojo, medians of three; the instrument gained a warm-up pass, because once the parse got cheap its row — the first heavy loop — was reading the allocator's cold start). Four changes, each measured on its own first: the SIMD scanners name the first matching lane with aselectofiotaand onereduce_mininstead of a scalar walk over up to 64 lanes (9.4 → 0.8 ns per chunk), and run 64 lanes wide, then 16, then scalar, so the last headers of a request no longer fall to atry_peekper byte; the 100-entry offsets array is uninitialized rather than filled (3.2 KB of stores per request — 0.3 µs in context, though 66 ns measured alone); theHeadersblob and index are sized once from the bytes consumed and the field count, and a value goes in as oneextendrather than a byte at a time; and the wrapper's three post-loop RFC scans (Host, the Transfer-Encoding/Content-Length pair, the last-coding rule) are flags set in the loop that already dispatches on the name — the Host one had been building aStringto measure it.On the wire, byte-identical responses:
apps/hellounder wrk with browser headers, old and new builds side by side, 122.1k → 150.2k rps at c16 (+23%) and 123.1k → 152.1k at c64, p50 113 → 90 µs (bench/results/parse-lever-ab/hello-wrk-parse-ab-*.json); through the ASGI executor on stdlib asyncio, 55.7k → 60.1k (+8%) — see theM0_INVERTEDentry below for what that did to the inversion's gate.One answer changed, and it was wrong before: the wide scan looked for the first CR and only then for any other control byte, so a field value ended by a bare LF ran on to the next line's CR whenever one lay in the same 64-byte chunk — the next header vanished into the value. The mask is now the scalar tail's own predicate, so both widths agree (
test_a_bare_lf_ends_the_line_even_with_a_cr_further_onfails on the old scanner). Ten more tests sweep a line ending across every offset from 1 to 140 bytes for values, names and request targets — every hand-off between the three widths, both sides — and pin the invalid-versus-incomplete verdicts of the token scanner, which are the byte-at-a-time loop's exactly (a header line with no colon is invalid, not "still arriving"). -
run_event_loopis now three functions and a struct, with no change in behaviour:prepare_loop(the registrations and slot tables, returning aLoopState),_run_pass(everything between onebackend.waitand the next; returns whether the shutdown pipe fired) and_run_shutdown(the drain). The pass and the drain are the former inlinewhilebody, moved verbatim behindrefbindings into the state, so the nine helpers' 20- argument signatures are untouched. This is the step the loop inversion needs — a pass that something other than thatwhilecan call, one per asyncio readiness callback on the executor's own thread — and it is landed on its own so the zero-diff claim is checkable in isolation: the full suite, the warning ratchet (68, baseline) and the nine seam smokes (shutdown, ASGI streaming with its RSS guard, the blocking pool, the counter under prefork, pipelining, the Mojo pool, hybrid mounts, the Django WebSocket hold) pass unchanged, andstress-asgiN of N. -
Header parsing no longer builds a
Stringper name and per value.HTTPHeaderholds four offsets into the parse buffer instead of twoStrings that existed only to be read back as bytes and copied into theHeadersblob — two copies of every header per request, the first made to be the source of the second. Measured with the newscripts/bench_http_parts.mojoon the twelve-header browser GET:parse_request_headers2.52 → 2.05 µs, the whole user-space request 3.74 → 3.31 µs (−12%). The instrument is the point as much as the number: SERVER_PERFORMANCE.md's "allocations are invisible" verdict came from loopback sampling at 50k rps, and at 116k rps the parse turns out to be two thirds of the user-space request. Nothing on the wire changed; the eight parse-sensitive smokes and all m0-http tests pass unchanged.parse_http_versionalso stopped allocating a list of the literal it compares against (39 ns, fixed because it was silly, not because it showed). -
HTTPServicenow requires onlyfunc. The trait's other eight methods carry default bodies, so a handler writes the hooks it uses and nothing else.apps/hellowent from 57 lines to 30 — 4 lines of handler had been carrying 25 lines of empty stubs — and 268 lines of the same boilerplate came out of the five demo services inservice.mojo, all six Mojo apps and the README example. Nothing on the wire changed: every stub deleted was byte-identical to the default replacing it, andsmoke-hello,smoke-notes,smoke-counter,smoke-todo,smoke-ws,smoke-chatandsmoke-clientpass unchanged.Overriding a default is ordinary — define the method and yours wins. The practical effect is on the trait itself: adding a hook with a default no longer breaks every implementer in the repo at once, which is what the old contract's warning was about. Adding one without a default still does.
-
Router.allow_header(path)builds a 405'sAllow:from the routing table.apps/notes_apihad been probing the router once per method over a hardcoded["GET","POST","PUT","DELETE"]list — fivematchcalls to answer a question the table already knew, and a route registered in any other method (PATCH,HEAD) was silently missing from the header it produced.matchandallow_headernow share one_path_matches, so the two cannot disagree about which routes a path reaches;Router.method_ofreads a registration back. The headersmoke-notesasserts byte for byte (GET, PUT, DELETE, OPTIONS) is unchanged.
Added
-
M0_INVERTED=1— the loop inversion, experimental and behind the variable. For an unmounted, pool-free ASGI application, the Mojo event loop runs INSIDE the executor's asyncio loop on one thread: the backend's kqueue/epoll fd is registered withadd_reader, one readiness callback runs one non-blocking pass, a request reaches the app throughWSGIHandler.direct_jobwith no datagram, and its response reaches the wire throughservice_direct_completionswith no wake. Every other topology stays on the pump, unchanged.Correct, proven:
smoke-asgi(0 KB RSS over 10k requests),-fanout,smoke-django-asgi,smoke-fasthtmlandstress-asgi(30/30 under 20 hogs) all pass under the variable, on kqueue and — verified in a Linux container before CI,scripts/epoll_inverted_check.sh— on epoll; CI runs the ASGI smoke under it on both platforms. Two single-thread rules were found on the way and are documented onExecutorPort._place_frameandPyBridge.notify_disconnect: a producer that waits for the loop to drain waits for itself, so a full chunk channel is drained by running a pass; and a direct job would overtake a disconnect still on the FIFO submit channel, so the disconnect goes direct too.Not yet a throughput win, and the numbers say why. Same session, uvloop executor, c16, two samples of three rounds: inverted 59.1–59.6k rps at 0.87–0.88 cores (p50 263 µs), pump 62.6–63.1k at 0.98 (p50 237 µs). The two cross-thread wakes per request are gone — that is the −11% of CPU — but the pump's two threads were also overlapping Mojo parse/write with Python app work, and at c16 wrk is a closed loop, so +27 µs of serialized latency is −6% rps. Per core it is +5%; on stdlib asyncio (the gate's row) it is +1% rps at −10% CPU, +12% per core, with both arms at 0.93x uvicorn asyncio. The default therefore stays the pump; the flag is the A/B,
bench_asgi_wrk.shnow recordsinverted=in its artifact, and the session's eight A/B artifacts are underbench/results/inverted-ab/rather than the canonical glob the benchmark page renders from.Re-measured after the parser change above, on the gate's own row (stdlib asyncio, c16, medians of three, uvicorn asyncio beside each arm;
bench/results/parse-lever-ab/): pump 55.7k → 60.1k rps, 1.03x uvicorn asyncio; inverted 54.5k → 59.7k, 1.01x, at 0.88 cores (67.9k/core against uvicorn's 59.6k). On uvloop: pump 63.2k → 69.1k (0.83x uvicorn uvloop), inverted 60.2k → 66.4k at 0.86 cores, 77.2k/core (0.79x on rps, 0.92x per core). Both arms now clear the gate; the inversion's edge over the pump is still per-core (+9% and +12%), not throughput (within noise, and −4% on uvloop), and the default is unchanged.Evaluated for the default and declined, on two more measurements: at c256 the per-core edge is gone (pump 88.1k @1.02, inverted 85.5k @0.99 — +0.6%/core, −2.5% rps, on an idle machine with the comparators within 0.5%), so it does not buy capacity; and under the flag a request mid-await at SIGTERM is answered at the 5 s drain deadline (5.30 s for a 1.5 s request, where the pump answers at 1.50 s), because the drain is still the blocking first cut. That limitation is recorded beside the flag in
m0serve.mojoand as ROADMAP design item 6, deferred to the inversion's promotion bar rather than built for a mode nothing runs; an inverted server wants a stop grace of 10 s or more. -
A handler pool for Mojo handlers (
lightbug_http/mojo_pool.mojo):MojoPoolputs N handler threads behind one event loop for anHTTPServicewritten in Mojo, the way--blocking-threadsdoes for a WSGI app — no interpreter, no GIL, noDetachingBackend. A handler conforms toPoolHandler(two methods:make,shutdown; the rest isHTTPServiceand its defaults) and the loop becomes an acceptor:Server.listen_and_serve_nonblockingnow carries theoffload_addrparameterrun_event_loopalways took andServerdropped.Measured (
apps/pool_spike, three runs, M4): p99 of/faston a keep-alive connection beside N handlers blocking 400 ms in a syscall —configuration slow=0 slow=1 slow=2 slow=6 loop only 0.1 ms 406.0 ms 405.9 ms 2026.5 ms pool of 4 0.2 ms 0.2 ms 0.3 ms 404.0 ms At
slow=2the loop-only p50 is 405 ms — every request, not a tail. The last column is the deliberate saturation boundary (6 blockers against 4 threads): the pool degrades to about ONE blocking duration (p50 31.8 ms) where the bare loop degrades to the queue's sum (bench/results/pool-probe-20260828T175956Z.json). Deliberately only for handlers that block: CPU-bound work already parallelises inside one handler withstd.runtime.asyncrt'sTaskGroup(measured 3.6x on four tasks), so the pool exists for threads parked in a syscall. A streaming response from a pool thread is refused with 409 (the loop drains its own handler's registries, not a pool thread's), andbefore_requestruns on both the loop's handler and the pool thread's — the loop's is where an always-responsive/healthbelongs.Guards:
test_mojo_pool.mojo(intest-http),poe smoke-pool(in CI: which thread served, saturation behaviour, clean SIGTERM),poe sabotage-pool(in CI on Linux, where all four rules are observable — closing the submit channel wakes a blockedrecvon macOS and not on Linux), andpoe probe-pool, the pre-release p99 table with a deliberate saturation column. -
m0_http.reply— the response constructors every Mojo app was writing for itself.apps/notes_api,apps/datastar_todoandapps/datastar_countereach carried their own_json,_html,_no_contentand_parse_id, the last two byte-identical copies. The module holdsjson,html,empty,no_content,redirect,problem(RFC 9457),vary_accept,accept_header,body_stringandparam_int, lifted from those bodies rather than invented, and all three apps now use it —apps/notes_apiwent from 388 lines to 308.redirectis new surface:common_response.mojoshipped onlySeeOther, so 301/302/307/308 had no constructor at all.One behaviour change came with it.
param_intrefuses a parameter longer than 18 digits; the hand-written copies multiplied without bound, so/notes/99999999999999999999wrapped to some other note's id. Every caller already treats-1as a 404. -
packages/m0-http/test/test_service.mojo— the guard for the above, and the first unit coverage the handler contract has had.MinimalServiceimplementsfuncand nothing else, so the file failing to compile is the failure; the value assertions pin that the defaults return what the event loop expects (no short-circuit, an empty drain, a non-streaming slot), since a wrong default would be a silent behaviour change across every app rather than a compile error. Proven by reverting each of the eight defaults to...in turn and confirming the suite fails for every one.
Fixed
- A keep-alive connection answered during the drain no longer holds
the drain to its 5 s deadline. The graceful shutdown closed the
connections that were already idle when SIGTERM landed, but a request
still running at that moment — on a pool thread or the executor —
completed during the drain, went out in one
send, registered no write interest for the drain'sEVFILT_WRITE-only dispatch to see, and re-armed its slot for a next request the drain never reads;active_countthen held at one until the budget ran out. Measured with a 1.5 s request in flight at SIGTERM: the response at 1.5 s either way, but the process exited at 5.35 s with keep-alive against 1.55 s withConnection: close, in every execution mode. The between-requests sweep now runs after every completion pass of the drain as well as before it (_close_between_requests); the process exits 0.04 s after answering.smoke-shutdowngained a fourth phase on the Mojo pool (scripts/drain_inflight_probe.py), which fails at 5.33 s with the in-drain sweep removed.docker stopduring traffic now costs about the slowest in-flight request rather than 5 s.
0.14.1 — 2026-08-28
A hardening release for the ASGI streaming seam. No wire format, default
or API changed, and one thing is visible to an application:
websocket.send now applies backpressure. An app that sends faster
than its client reads waits, where before it silently lost messages:
430,693 of 1,638,400 bytes arrived under a clean close frame, a message
stream with holes the peer had no protocol-level way to detect. The same
flood now arrives whole, byte for byte.
The rest turns failures that were invisible into failures that are named
and terminal, and adds the two guards the 0.14.0 slot-ownership fix
shipped without. If you run ASGI streams or WebSockets under m0serve,
this is worth taking; if you run WSGI only, nothing here reaches you.
Added
- The slot-ownership race has guards. 0.14.0 fixed a silent hang in
the ASGI executor — a stream on a recycled connection slot could leave
the loop holding a subscription with no producer, which a client saw as
a 30 s stall against a clean server log — but the fix was verified only
by an ad-hoc reproducer, so nothing would have caught a
re-introduction. Two guards now do:
poe test-shim(in CI, insidetest-all) exercises the shim's ownership rules with no server, no Mojo and no threads:scripts/shim_ownership.pyextractsSHIM_SOURCEfrombridge.mojo,execs it, and drives it through real socketpairs exactly as the event loop does. Seven tests, one per rule; four of them fail on the shim reverted to its pre-fix ownership shape.--sabotagereverts each of the eight rules in turn and insists the suite fails for every one, so the repo's sabotage rule is enforced rather than remembered.poe stress-asgi(a pre-release check, deliberately not in CI) runschunked_keepalive.pyN times under CPU hogs — the shape that recycles a slot mid-stream. Reverted build: failed on round 5 of 15. Current: 45 of 45 across three runs.
check-docsfails when atest-*poe task is not reachable fromtest-all, which is the single step CI runs. Itssmoke-*twin exists because a smoke once shipped that CI never ran; a test task can drop out the same way, with no ghost step to notice.
Fixed
-
Six silent failures in the streaming seam are now terminal and named. A frame the seam could not place was discarded at five of six sites, and a drain ack could credit the wrong stream. Each was measured by forcing the failure:
- a dropped stream begin frame served a clean, EMPTY 200 (the log
naming only a downstream
KeyError) and left the application's task awaiting credit for the life of the process; it now answers 500, closes, and cancels that task through the same disconnect tag the loop would have sent. - a dropped stream end frame hung the client until its own timeout (12 s, curl exit 28) against a silent log; it now aborts — the body's bytes, then a close with no terminator — in 13 ms, and says so.
- a response chunk the loop's outbox refuses aborted nothing and hung the connection for 12 s; it now aborts, so the truncation is visible to the client.
- a WebSocket frame the outbox refuses delivered 430,693 of 1,638,400 bytes under a clean close frame — a message stream with holes the peer had no way to detect. It now flushes what is queued and closes abruptly (1006).
- the WebSocket begin frame and the close/end pair get the same
treatment — and a socket can now be aborted at all: the loop's abort
path gated on
slot_sse, which a held 101 never sets, so aborting a socket was a silent no-op. It readsslot_sse or slot_wsnow, and a 101 records its generation after the non-stream branch that was clearing it. - a stale drain ack — acks name a slot and carry no generation, and the loop recycles a slot the instant it closes a connection — could credit a recycled slot's new stream past its window, over-committing the one chunk channel every stream on an executor shares. Credit is now clamped to the window.
A tear-down is claimed once per stream (
stream_lost): the producer does not learn its connection is gone until the loop closes it, so one flooding WebSocket announced itself 336 times before. - a dropped stream begin frame served a clean, EMPTY 200 (the log
naming only a downstream
-
websocket.sendapplies backpressure. It was the one path above reachable by an ordinary application, and until now it was not credit-gated at all: an ASGI app that sent faster than its client read filled the loop's 64 KB per-slot outbox and every frame past it was dropped — 430,693 of 1,638,400 bytes under a clean close. It now waits for drain credit exactly as a streaming HTTP response does, so the same flood arrives complete and in order (measured byte for byte: 400 x 4 KB plus framing, 1,640,193 bytes on the wire). Almost all of the machinery was already running — the loop acks a socket's drained bytes, because a WS slot on an executor lane answersslot_channel_stream— and what was missing was the window to credit them to, seeded atwebsocket.accept. Credit is charged in ENCODED frame bytes, which is what the loop acks; charging the payload instead drifts by the header on every message, threefold on one-byte sends.apps/asgi_baregrew a/ws/floodroute andws_probe.pya phase that asserts the exact count, sosmoke-asgifails if the gating is removed (measured: 15 of 400 frames arrive).Two limits this does not lift, both now loud rather than silent: a single WebSocket message larger than
MAX_PENDING_BYTES(64 KB) is still refused by the outbox, since the cap applies to one frame as well as to the queue; and a--realtimehold on a WSGI lane has no window, because the loop does not ack those sockets.
0.14.0 — 2026-08-27
A streaming and throughput release. Three changes are visible on the wire
or in your terminal, so read them before upgrading: unsized WSGI bodies (a
generator, Django's StreamingHttpResponse) now stream chunked where
0.13.0 buffered them into one sized response; a second m0serve on a busy
port now fails to bind instead of silently sharing it, SO_REUSEPORT
having become opt-in; and m0serve's startup output is one line of its
own, printed after the application loads, so "ready" means ready.
The ASGI executor went from 0.72x to 1.06x uvicorn --loop asyncio on
the benchmark page's 16-connection row — 1.22x at 256 — by inverting the
pump: Python calls into Mojo through a type built in-process, and the
executor thread never leaves its event loop. And one silent hang is fixed:
a stream on a recycled connection slot could leave the loop holding a
subscription with no producer, which a client saw as a stall on a clean
server log.
Added
-
check-docscounts the tests in the tree (def test_perpackages/*/test/*.mojo) and fails when README's "What's in the box" table or itstest-allcomment says otherwise; the table sat at 618 while the tree held 928. -
Unsized WSGI bodies stream. A generator or iterator the application did not size — Django's
StreamingHttpResponse, a FlaskResponse(generator)— is produced chunk by chunk from a--blocking-threadspool thread (the WSGI zero-config default) through the same chunk channel the ASGI executor streams through: chunked on HTTP/1.1 with the connection reusable after, close-delimited on HTTP/1.0,close()called, and the thread back in the pool when the client leaves. Every release before this joined such a body whole, so a never-ending SSE generator never answered and pinned its thread until shutdown. Sized bodies (Content-Length— every Flask page, every Django page behindCommonMiddleware,FileResponse), list bodies, Django'sHttpResponse, HEAD, bodiless statuses andM0-Holdresponses buffer exactly as before, so no framework page changes on the wire; a server with no pool keeps joining.poe smoke-wsgi-streampins the contract. -
apps/wsgi_baregains/stream,/stream-forever,/stream-raises,/stream-empty,/stream-write-inside,/stream-cland/stream-hold;apps/django_wsgigains/events, aStreamingHttpResponse. -
poe bench-asgi-wrk/scripts/bench_asgi_wrk.sh: the wrk run behind theasgi-wrk-helloartifact, which had no producing script. It puts the venv first onPATH(so a bare invocation embeds the same interpreteruv run poedoes), stampsexecutor_pythonandexecutor_loopin the artifact, takesM0SERVE_BINfor an A/B of two builds andBENCH_CONNSfor the concurrency; the generated block on the benchmark page prints the executor's loop.check-docsnow ratchets the uvloop ratio and per-core gap too.BENCH_NAME,BENCH_APP_DIR,BENCH_M0_SPEC,BENCH_UV_SPECandBENCH_PATHrun a framework app through the same script: theasgi-wrk-fasthtml-*andasgi-wrk-django-*artifacts behind WSGI_PERFORMANCE.md's framework table (FastHTML 0.85x / Django ASGI 1.18xuvicorn --loop asyncio). -
bench/results/asgi-wrk-conns-*.json: the 2026-08-27 concurrency matrix — the executor with and without pump batching, and on uvloop, at 16/64/256 connections.
Changed
m0serve's ready line carries the Mojo flame:🔥 m0serve: app:application on http://…, and both READMEs now document it as the contract it is — printed once per worker, after the application is imported, so "it printed" means "it is serving". There is no cross-server standard to match here (uvicorn saysUvicorn running on …, gunicornListening at: …); for an orchestrator use--health-path /healthor a TCP check rather than the log line. One emoji, once per worker, on the line that already means "ready" — the fork's🔥🐝 Lightbug is listeningbanner it replaced was Lightbug's branding (attribution lives in NOTICE) and printed before the application loaded. Error and shutdown lines stay plain: a flame on a failure reads as celebration. The Mojo example apps keep the fork's banner unchanged.- The loop↔executor pump is batched in both directions. The loop
sends a pass's submits to an executor lane as one datagram at the
bottom of the pass, and the executor answers a pump pass's completions
with one datagram; every existing ordering (begin frame before head,
park before poke, pill FIFO behind every job) is preserved by
construction, and a batch the channel will not take runs inline, which
is what a refused submit always meant. Measured under wrk with the
uvicorn rows re-measured beside it: +5% at 16 connections
(a pass batches three submits on average there), +7% at 64 and +19% at
256, where the executor passes
uvicorn --loop asyncio(1.10x; 0.85x against uvicorn with uvloop). Table and artifacts in docs/WSGI_PERFORMANCE.md;bench-asgi's stdlib throughput gate is retired — its harness read 1.4x the same day wrk read 0.75x, so it measures its own client; the ratio is printed as information and the mixed-tail gate stays. - The executor pump parks in
run_forever, onestop()per pass, instead of arun_until_complete(batch())per pass (38 µs on stdlib asyncio against 17): every shim event is appended to a list, the first append while the pump is parked schedules the stop for the end of the next iteration, andwait_eventsreturns the list. Measured on top of batching: +16% at 16 connections (50,747 rps, 0.90xuvicorn --loop asyncio), +18% at 64 (67,258, 1.17x), +2% at 256. A stop is armed only while the pump itself is parked — never insidefinish_executor's post-pill gather orlifespan_shutdown, which it would end early with "Event loop stopped before Future completed", skipping the application's shutdown;smoke-asgi's new outlive-the-drain phase pins it, andapps/asgi_barewritesM0_SHUTDOWN_MARKERfrom its lifespan shutdown so the phase can tell. - Python calls into Mojo for every executor event; the executor thread
never leaves
run_forever.ExecutorPortis a Python type built withPythonModuleBuilderinside the embedded interpreter (no shared library, noPyInit_, no ctypes; ~70 ns a call) and set into the shim as_port; every event that used to be queued for a Mojo pass is_port.dispatch(ev), handled at once inside the loop iteration that produced it, and completions are poked to the loop once per iteration by acall_soon-scheduled_port.flush. The per-passrun_until_complete(38 µs on asyncio, 64 on uvloop) is gone. Measured beside therun_forever+stop()pump: on stdlib asyncio within noise at 16 connections (49,713 rps, 0.93xuvicorn --loop asyncio) and 1.07x / 1.13x at 64 / 256; on uvloop, which this shape finally lets pay, 60,419 rps at 16 connections — +24% over the pump on the same loop, 1.05x the asyncio comparator, 0.74x uvicorn with uvloop — and 0.94x uvicorn with uvloop at 256.bench_asgi_wrk.shgainsBENCH_EXECUTOR_PYTHON=systemfor an A/B of the executor's loop. - The benchmark page's ASGI row is re-measured (0.72x → 0.75x
against
uvicorn --loop asyncioat 16 connections, executor on uvloop) and now also states the uvloop number a defaultpip install uvicorn[standard]produces (0.53x). Which loop the executor ran on was never recorded before:bin/m0serveoutside the venv embeds the system Python and runs on stdlib asyncio — measured to be a wash either way (−3% at 16 connections, +4% at 256).
Fixed
-
A stream on a recycled slot could stall silently. The executor's per-slot state (credit window, event, disconnect mark) was keyed by slot alone; the loop recycles a slot the instant it closes a connection, and the previous task lives on for an iteration or two. When the HTTP/1.0 client of
chunked_keepalive.pyclosed after the head and the keep-alive stream that followed landed on the same slot, the new task saw the OLD connection's disconnect mark, cancelled its own stream, and — the mark being the slot's — skipped its end signal, leaving the loop a subscribed stream with no producer: the client waited 30 s on a clean server log (CI macOS, 1 in 2; 8 of 11 runs under twelve CPU hogs locally). A stale task's late cleanup could also wipe the live task's window. Now a slot's state belongs to the slot's current task (_exec_slot_task): cleanup only by the owner, a disconnect stamped on the task it hit (with the dead connection's in-flight bytes refunded there), a stale mark cleared when a new task takes the slot, and every "am I gone" check asking both. 0 of 6 under six CPU hogs (the plain build: 4 of 5) after, same load. -
A second
m0serveon a busy port fails, loudly, instead of binding beside the first.SO_REUSEPORTwas set unconditionally on every listener, so a second server on an occupied port bound successfully, printed "Ready", and on Linux took a share of the connections (17 of 40, measured from the wheel inpython:3.12-slim; macOS: served nothing). The option is now opt-in onListenConfig(reuse_port=False; workers and threads share one pre-fork listener and never needed it), andm0servebinds with five attempts a second apart — a restart racing the previous process's 5 s drain still succeeds — then exits 1 withaddress already in use: HOST:PORT -- is another server running?.smoke-servepins it. -
"Ready" means ready. The listener's banner printed before the application was imported, so a failed import logged "Ready to accept connections" and then exit 1.
m0servenow binds quietly (ListenConfig(quiet=True)); its own startup line, printed after the load, is the ready signal. -
A load failure shows its traceback. An application whose import raises (a settings module without its environment variable, a missing dependency) printed only the exception's one-line text. The shim now attaches the Python traceback for every failure that is not the spec's own module or attribute being absent — those stay one line, because a bare
MODULEtries four discovery candidates and the misses must stay quiet — and discovery stops at a candidate that exists and raises instead of trying the next convention.apps/wsgi_bare/deep_failis the case;smoke-servepins it and the quiet misses. -
A stream that raises after its head is truncated honestly. The connection closes without the chunked terminator — for a WSGI generator and for an ASGI application alike; the executor used to end such a body cleanly, which made a short body indistinguishable from a complete one.
-
A chunk that outlived its connection can no longer land in a recycled slot's new stream. Every stream frame carries its stream's generation and the loop handler drops one that is not the subscription's. One producer's frames are FIFO behind its own begin; with two producers on one loop (an executor and a pool thread, two executors, or a hold arriving on the other bus fd) there was no order between them at all.
-
A stream whose head completes during the shutdown drain is now told goodbye. The drain loop dispatches write-readiness only, so such a connection was never closed and its producer waited out the bounded join as a straggler; a second farewell pass after the drain closes it.
0.13.0 — 2026-08-27
A security-audit release plus two wire-protocol conformance fixes. Several requests that previous versions accepted are now refused, and one class of client that previous versions hung is now served — read Changed before upgrading anything that speaks non-standard HTTP at this server.
Security
- A cross-connection injection hole is closed. Channel names opening
with the reserved
\x01byte address connection SLOTS on the loop, and an application channel is frequently user input (%01in a form body decodes to the control byte): an unauthenticated POST could reach another client's SSE stream throughpublish(). Every publish boundary now refuses reserved and over-long names —publish_to_channels, the shim's_M0Broadcast, and both copies ofm0pub.publish_frame. /ws/messageis the server's path, not an application's. Under--realtimeit carries synthetic POSTs with trustedM0-*headers and a CSRF exemption; a request for it arriving over the wire is now 404, so only the in-process synthetic one reaches the app.- Response headers carrying CR, LF or NUL are dropped, not transmitted (and an injected status reason phrase is emptied) — a header an application built from user input can no longer end the header block and add headers or a body of its own.
Proxyrequest headers never reach WSGI/ASGI environs (httpoxy): CGI's mechanical mapping would turn them intoHTTP_PROXY, the variable outbound HTTP clients read to choose a proxy. Nothing else is dropped —X-Forwarded-*is load-bearing behind a real proxy.- Static mounts serve only regular files, auth token comparison runs in constant rounds, WebSocket datagrams are bounded, and the WS outbox drops whole frames past a 64 KB cap instead of growing unboundedly.
Fixed
- Pipelined requests are all answered, in order (RFC 9112 §9.3).
Every release through v0.12.0 answered only the first request of a
pipelined burst and left the client hanging on the rest: the keep-alive
reset cleared the receive buffer, and bytes consumed together with a
previous request get no readiness event of their own. The request's end
is now stamped at dispatch, the keep-alive reset preserves the tail
(and ONLY the keep-alive reset — a tail can never leak across
connections), and a drain loop answers buffered requests after every
completed response, on both backends and on the blocking path.
poe smoke-pipeliningpins it. - A client that half-closes (
shutdown(SHUT_WR)) after sending gets its response. kqueue reports a half-close asEV_EOF, and closing there discarded the response already written: macOS lost 24–30 of every 30 requests on GET, Content-Length and chunked alike; Linux lost none. The loop now finishes the buffered request and only turns off keep-alive.poe smoke-half-closepins it. - The epoll backend registers
EPOLLRDHUP, ending the half-close divergence. A half-closed INCOMPLETE request now releases its slot promptly on Linux too, where it used to be indistinguishable from a silent client and waited out the full 10 s header timeout for a 408. The guard is layered — a recv returning 0 marks the peer gone even where the flag is absent — and a half-close while a request is out on a--blocking-threadspool thread no longer detaches the fd (which dropped the response the pool thread was about to complete): the completion answers through the still-open fd. - A request bigger than one read is answered on epoll too. The header
path performed one recv per readiness edge and never re-armed, so a
request over 8192 bytes — a large cookie jar or a JWT, not an attack —
stalled on Linux until the header timeout answered 408. Headers now
re-arm exactly as the body path always did; requests are served up to
the 32 KB header cap and answered 431 beyond it.
poe smoke-large-requestpins the boundary. - A chunked request body is decoded incrementally by one persistent decoder per connection. Rebuilding the decoder per read event made a dribbled chunked body O(N²) on the loop thread — 3 MB took 1.37 s, 0.006 s after — and reset the decoder's own abuse-ratio guard so it could never trip.
- The chunked terminator is consumed (
consume_trailer), so closing aConnection: closesocket no longer leaves the final CRLF unread — which made the kernel send RST instead of FIN and discarded the response, measured at up to 53% of chunked requests and 100% when the client paced its writes. - HTML escaping passes UTF-8 through intact (
escape_htmlin m0-core):caféno longer renders ascafé. - An application's
Set-Cookiegoes to the wire verbatim (subject only to the CR/LF refusal). Round-tripping it through the server's own cookie model silently droppedexpires,SameSite, everything after the first=in a value, and any unmodelled attribute — on every Django session and CSRF cookie of every app.
Changed
Requests that v0.12.0 accepted and this release refuses, or handles differently — each is a smuggling or correctness surface:
- HTTP/1.1 without a
Hostheader → 400. Hand-rolled clients and crude health checkers sometimes omit it. - Malformed
Content-Length→ 400 (5, 5,0x10,+5,-1,5abc, longer than 18 digits). Previously read as 0 and served as bodyless — which a proxy in front could read differently. Transfer-Encodingwhose final coding is notchunked→ 400.gzipalone was previously served as bodyless.- An encoded slash stays encoded:
unquotere-emits disallowed bytes as%XXinstead of deleting them, so/adm%2Finno longer collapses to/admin. - A chunked body is bounded by raw bytes consumed as well as decoded size — a body whose framing outweighs its payload roughly twice over now answers 413 (1 MB in 3-byte chunks is 3.6 MB on the wire).
- A chunked request that omits the final CRLF now waits for it, as it would for any truncated body, instead of being answered early.
Added
poe smoke-pipelining,poe smoke-half-close, andpoe smoke-large-request— socket-level regression probes for the fixes above, run on both CI runners because each bug was invisible on one platform.- The PyPI publish path is hardened: the
pypienvironment is the authorization boundary (deployment branch policy + required reviewer), the publishing action is pinned, and the wheel set is validated against the tag before upload.
0.12.0 — 2026-08-26
Added
-
WebSocket holds work with
--blocking-threadsand with--mount. The last thing--realtimerefused. A pool thread performs the 101 — the client's key is in the request it holds — and sends the loop anHframe carrying its own lane; the loop records which mount holds the socket, and an inbound frame rides the submit channel back as aTAG_WS_MESSAGEdatagram (the executor's shape plus the channel, since a pool thread's registries are empty).next_jobdecodes both shapes off one channel, told apart by length, into a buffer owned by the thread so a WebSocket's size is not charged to every request. Two orderings are load-bearing and both are pinned bysmoke-django-realtime-wsphases 3 and 4: the pool must be asked about a socket before the executor (on a mixed mounted server the executor's fd is set, and asking it first hands every message to an executor that never accepted the connection), and the "not pool-held" sentinel cannot be -1, which is a real lane. The whole socket probe — handshake, fan-out, relay through Django, channel isolation — now runs against a pool and against mounts, and behind two 1.5 s views it costs +13 ms against its own baseline. -
--realtimecomposes with--mount. One process can now hold the SSE streams a synchronous application publishes to and run an ASGI mount for the streams whose view is the producer — the shape a mixed application needs, and the one that kepttextshelfin two processes (docs/REAL_APP_VALIDATION.md). The loop tells a held stream from an executor's per slot rather than per server (OffloadPool.slot_is_executor, read from the lanesubmitalready stamps): asking globally was the same question only while the two could not share a loop, and a held stream drained as an executor's would be chunk-framed, acked to an executor that never issued the credit, and denied the comment heartbeat that keeps it alive through an idle proxy — none of which stops delivery, so none of which looks wrong.smoke-django-realtimephase 6 holds one stream of each kind on one loop and asserts both. Still refused: awebsockethold under--mount(409 — an inbound frame is a synthesised POST into one urlconf), and--realtimeon a server with no WSGI mount at all. -
--realtimecomposes with--blocking-threads. A hold taken on a pool thread is forwarded to the event loop's registries as a reserved frame on that loop's own bus channel, before the response completes — the executor's begin-before-head seam, applied to a second producer — so a held-stream server no longer has to run its views on the loop. On textshelf with eight slow views in flight that is the difference between a 1 543 ms fast-path p50 and 0.3 ms, and between M0-Hold being demonstrable and deployable. SSE holds only: a WebSocket hold under the pool answers a 409 that says why (inbound messages still reach the view on the loop thread), and--mountwith--realtimestays refused. The loop needs no ordering guarantee for the forwarded frame because, without an executor, it has no end-of-stream signal to misread a not-yet-subscribed slot as.smoke-django-realtimephase 5 pins the composition;smoke-blocking-threadsandsmoke-doctornow assert the pair is accepted where they asserted the refusal. -
The isolation benchmark has an artifact, and the ratchet caught sixteen stale sentences. Gate 3's last item.
bench/results/now carries amixed-workload-*.json— the pool's ~100x p99 claim, the largest effect in this repository and previously the only one with no machine-readable source — plus a post-pin re-run of the WSGI layer split.Re-rendering moved every derived figure, and
check_bench_prosefailed the build naming all sixteen prose sentences that had gone stale across README.md, docs/BENCHMARKS.md and docs/WSGI_PERFORMANCE.md, each with the value it claimed and the value the artifact computes. That is the whole reason it exists: the tables re-render themselves, and before this the sentences around them would have quietly kept the old numbers. Per-core ratio 0.83x → 0.85x, bridge tax 1.44x → 1.36x.Two findings the run itself produced. Granian's
--blocking-threadsrow is better than ours — ~0.6 ms flat against our best ~2 ms — which is now on the page, because the honest claim is that the pool removes a hundredfold stall, not that it wins the tail that remains. And the earlier note that granian "is not in this repo's lock file" was wrong: it is, in thebenchgroup at the pinned 2.8.1, oneuv sync --group benchaway. That is why its row had been missing from the isolation table.Recorded because it changes how these are read: one anomalous round per run is normal on this box. Three recorded layer-split runs each had exactly one round land well off the other two, in a different position each time, while their medians agreed to within 0.03 on the per-core ratio. Median-of-three is doing real work here, not ceremony.
-
The first screen leads with what the server is for. Gate 5. All three surfaces that have a first screen now open on the same claim — realtime from a synchronous Python app, with no added infrastructure — with the hero shown as six lines of ordinary sync Django rather than described: README.md,
packaging/m0serve/README.md(which is what PyPI renders, and is the one nothing was checking until this release), andllms.txt.The gaps are on the first screen rather than in an issue: no TLS or HTTP/2 (terminate at a proxy), the platform floors, pre-1.0, and — stated with numbers and a link — that this is not the fastest server on raw throughput. A page that hid that would be contradicted by the benchmark page two clicks away.
The snippet was run before it was published: a Django app containing exactly those six lines, served by
bin/m0serve --realtime, deliversid: 2 / data: deploy finishedto a livecurl -Nsubscriber. The fuller path stays covered in CI bysmoke-quickstart. -
docs/BENCHMARKS.md: the public benchmark page, and it leads with the losses. Gate 3 of the launch checklist. Four generated regions across two documents, all driven by
render_bench_docs.pyand all CI-checked — hand-edit a table, or land a new artifact without re-rendering, andcheck-docsfails naming the file.Two things it does on purpose. It states plainly that m0serve is ~0.83x Granian per measured core on bare WSGI and 0.72x uvicorn on ASGI throughput, because the win it does claim — fast-request p99 under mixed load — is only credible next to them. And it renders a stated absence for the mixed-workload bench rather than omitting it: the handler pool's ~100x p99 improvement is the strongest claim in this repository and currently the only one with no machine-readable source.
render_bench_docs.pygrew from one region in one document to a table of targets, with a renderer per bench kind. The isolation bench gets its own: its finding is a comparison across slow levels within one configuration, so the rows are pivoted — a flat row isolated the slow work, a climbing one did not — and the percentiles are re-medianed fromrows, sincebench_record.medians()folds only rps and cores. It also stops naming a comparator the artifact's environment stamp recorded but the bench never ran: on a public page, "granian 2.8.1" beside a table with no granian row reads as if it had been measured and lost. -
m0serve --doctor: the configuration as JSON, starting nothing. The launch checklist's machine-readable startup diagnostic, and the reason is narrower than "nice to have": every refusal this server makes already names its fix, but seeing one meant attempting the run — which binds a port, forks, and imports the application.--doctorreports platform and wheel architecture, the interpreter it resolved and the virtualenv it came from, the spec discovery chose and the protocol it classified as, the resolved topology (including whether the handler pool is a default or configured), and achecksarray whose failures each carrydetail,fixandexit.The contract is the exit code:
--doctorexits with the codem0serveitself would exit with for the same arguments — 0 serve, 2 usage, 1 startup, 78EX_CONFIG. A diagnostic that reports "fine" where the server refuses is worse than no diagnostic, and the doctor mirrors the startup path's check order rather than sharing its control flow, so nothing but a test keeps them in step:poe smoke-doctorruns both binaries over every refusal and compares.That test earned its place before it was committed. The first implementation recorded the free-threading check before the usage conflicts, so
--workers 2 --threads 2reported 78 where the server exits 2 — the interpreter is never reached, becausemaindecides topology conflicts before any Python runs.Report.exit_codereturns the first failure rather than the largest for the same reason, andtest_doctor.mojopins it.A bare
m0serve --doctorwith no application is the "is this environment sane" call and exits 0 — reporting the interpreter facts is precisely what a failed install needs, and previously libpython not resolving was visible only as a traceback at serve time.
Changed
-
The loop's
before_requestruns before a request is offloaded, not only on the queue-full fallback — where, under a pool, it never ran on the loop at all.WSGIHandleranswers its static mounts and the health path there, so under--blocking-threadsthose are served on the loop in Mojo rather than by a pool thread: a stylesheet stays readable whatever the pool is busy with, and/healthreports the registries the loop actually drains. Before this, under the newly-composed--realtime --blocking-threads, it reported zero subscribers while events were being delivered — a pool thread's own registries are always empty. -
The mounted-isolation guard had 138x headroom and now has 12x.
hybrid_isolation.py'sISOLATION_BUDGET_MSwas 400 ms against an observed p99 of 1.4–4.1 ms, so it discriminated "isolated" from "sharing an execution mode" (~2000 ms, the sync view's hold) and nothing in between: a regression that parked the async mount for 300 ms — a hundredfold degradation, plainly visible to a user — passed. It is now 50 ms, chosen from 17 recorded CI runs across both runners rather than from the gap to the failure signal, and the docstring carries that evidence and its one limit (every run is the prefork phase; the--threadsphase is skipped wherever there is no free-threaded interpreter with fasthtml).Every run now prints its headroom, pass or fail, because a number drifting from 12x to 2x is the warning that comes before the failure.
-
A total loss of isolation reported a stack trace. With the pool off (
--blocking-threads 0) the sync mount's warm-up never returns at all, and the script exited on a urllibTimeoutError— the smoke failed, but whoever read the log had to work out why from a traceback. A request that never returns is now reported as what it is, with the two things to check named. The ordinary failure still reports a number: sample timeouts are bounded well above the 2000 ms hold, so a request genuinely queued behind the sync work is measured rather than erroring. -
check_wheel_platform_claimsnow checks what its docstring always said. It asserted that a wheel gets built at all and that neither README points at 3.13t; it never compared a platform table to anything. It now reads theplat:entries out ofrelease.yml'sbuild-wheelsmatrix — the only place a wheel that reaches PyPI is declared — and holds both READMEs to them in both directions: a built platform must be marked supported, and a platform marked supported must be built. Either failure is a claim with no artifact behind it, which is the whole premise of this file.Recorded while wiring it, because it is the opposite of the guess:
test.yml'spaths-ignorelists*.md, and a GitHub path glob's*does not cross/. A PR touching only the rootREADME.mdtherefore skips CI entirely, while one touching onlypackaging/m0serve/README.mdruns the full suite. The published README is the guarded one. -
The quickstart's version echo is machine-checked. QUICKSTART.md showed
m0serve 0.10.0against a 0.11.0 tree. The doc promises every command in it is executed by CI, and that promise is kept forbash blocks — but the echo lives in atext block, whichrun_quickstart.pydisplays rather than asserts. That is the right design (the other text block interleaves output from three commands and is not byte-stable), so the check belongs incheck_docs.py, where prose facts with a machine source live. docs/RELEASING.md names the fourth bump site;poe check-docsfails on all four.
Fixed
-
--app-diris prepended tosys.path, not appended. It appended where gunicorn, uvicorn andrunserverallsys.path.insert(0, ...), so an application module could be shadowed by an installed package of the same name — and the shadowed application simply is not the one served, with nothing to see. The help text,cli.mojoandapp.mojohad all said "prepended" since the flag existed; now it is true.prepend_to_pathalso declines to move an entry already at the front, and leaves duplicates further down alone — a path the user put there is not the server's to edit. Found by dogfooding the wheel, reconfirmed by the three-project pass, and guarded by asmoke-servephase that puts a module nameddjangounder--app-dirin a venv where the real Django is installed. -
Every
Set-Cookiean application set lost itsexpiresandSameSiteattributes. The WSGI/ASGI bridge parsed eachSet-Cookieline into aCookieand re-serialised it, and that round trip was lossy four ways:Expirationis a stub whosefrom_stringparses nothing,SameSitematched only lowercase values, a value was cut at its first=(base64 pads with one), and any attribute the struct does not model was dropped. Django's session and CSRF cookies therefore reached every browser withoutexpiresorSameSite— a persistent cookie silently demoted to a session cookie, and a CSRF cookie without its defence. Application lines are now transmitted verbatim (ResponseCookieJar.add_raw), which is also the cheaper path on the measured response half of the bridge. Found by serving three real Django projects;smoke-djangonow reads the cookie off the wire and requires all four attributes, because curl's jar stores name and value only and could never have seen it. -
Uploads between ~1.5 MB and
--max-bodywere refused with400. The per-connection receive buffer had its own 2 MB ceiling that--max-bodynever raised, so a body the server advertised as acceptable was rejected by the wrong check with the wrong status — under the default 4 MB cap too. The limit is now derived (ServerConfig.recv_buffer_limit()= headers plus body allowance, floored byrecv_buffer_max), so raising the body cap raises the buffer with it. A 7.1 MB image upload to a real Django app found it. -
Concurrent ASGI streams truncated each other, and enough of them wedged the executor. The chunk credit window is per stream (64 KB) while the chunk channel is one shared
SOCK_DGRAMpair, so N streams over-commit it;send_stream_chunkthen dropped the datagram it could not place — a short body under a clean terminator, or, with the end frame dropped, a response that never completed at all. Twelve concurrent WhiteNoiseFileResponses under Django were enough. Now bounded globally (_ASGI_TOTAL_WINDOWin the shim, where waiting is anawaitrather than a Mojo spin that would hold the GIL against the very loop that has to drain the channel), with the loop keeping owed credit and retrying it when the ack channel is momentarily full.smoke-asgiruns 32 concurrentFileResponse-shaped streams and checks every byte. -
SIGTERMnever returned while a handler thread sat in a response that never ends. AStreamingHttpResponseserved under WSGI is buffered, so an SSE generator never returns and its pool thread never comes back;stop_and_joinwaited for it forever, turningdocker stopinto aSIGKILLafter the grace period. The join now has the same 5 s budget as the drain (ThreadSet.join_within), after which the process exits naming how many threads it left inside the application. Nothing in the process can unwind Python on another thread, so leaving is the only correct answer; waiting was not. -
smoke-wheelleaked a server on every run, and could pass against the wrong binary. Nine orphanedm0serveprocesses accumulated over one day of development, all stillLISTENing on port 8129, one of them still answering200 OKa day after the run that made it.Two independent defects. The launch was
(cd "$work/app" && env ... m0serve ...) &— a list, which bash cannot exec-optimize, so$!was the subshell rather than the server.kill $pidkilled the wrapper and leftm0serveorphaned to init. Addingexecmakes the subshell become the server, which is howbench_mixed_workload.shhad been doing it all along. And theEXITtrap only removed the temp directory, so everyfailafter the launch leaked one too; the server pid is in the trap now.The consequence was worse than untidiness. The server sets
SO_REUSEPORTbecause prefork needs it, so the kernel adds a new listener alongside a stale one and load-balances between them: a leaked server from an earlier run can answer this smoke's request, and the assertions then pass against a binary that is not the one under test.smoke-wheelnow refuses to start when 8129 already has a listener, naming the pids and the command to clear them.Verified by running the pre-fix task once (leaks exactly one process,
ppid 1) against the fixed one (leaks none, on both the success and the failure path), and by putting a decoy listener on 8129 to trip the new refusal. -
The README quoted a decomposition its own measurements had retired. It said the one-worker gap to Granian "splits evenly, 1.58x HTTP layer and 1.58x bridge" — numbers from before the CPU-normalized re-run, which WSGI_PERFORMANCE.md had already replaced with ~1.0x × 1.35x and explicitly marked as "records of what was measured, not descriptions of the present". The README kept quoting them, in raw rps, against a comparator since found to be running 1.75 cores. Rewritten from the artifact.
-
"There is no chunked encoding" was no longer true. The server has chunked transfer-encoding and ASGI responses stream through the executor chunk-framed on HTTP/1.1. WSGI responses are still fully buffered, but the reason is PEP 3333 — a WSGI response carries a
Content-Length, which means knowing the length — not a missing feature. -
The PyPI project page told aarch64 users their wheel did not exist. 0.11.0 shipped a
manylinux_2_35_aarch64wheel and its release notes claimed "the platform matrix on the README is the platform matrix on the index" — which was true of the repository's README and false ofpackaging/m0serve/README.md, thereadmenamed by the wheel's pyproject.toml and therefore the page PyPI renders. That one still readLinux aarch64 | buildable, not yet shippedfor the whole of the release. Two READMEs, one of them published, and the ratchet was pointed at the other. -
A README number quoted twice, guarded once. The mounted-isolation p99 (2.8 ms) now appears on the first screen as well as in the mounts section. It is not artifact-backed —
hybrid_isolation.pyasserts a deliberately generous ceiling rather than recording the figure — socheck_hybrid_p99_consistentchecks the two copies against each other instead. A number edited in one place and not the other is the ordinary way a README starts contradicting itself. -
The bench prose was answerable to nothing, and it was wrong.
render_bench_docskept the generated tables honest; the sentences around them — where the headline claims actually live — were checked by no one. docs/WSGI_PERFORMANCE.md stated the WSGI result as a decomposition, "roughly 1.0x HTTP layer × ~1.35x bridge", and it does not reconcile with the artifact directly beneath it: the measured per-core gap is 1.21x, and a 1.35x bridge term requires an HTTP layer term of 0.89x — this server's HTTP layer slower than the comparator's, which the same sentence denies. -
The boolean-flag dispatch had a fallthrough.
parse_argsended its chain withelse: opts.metrics = True, so a new flag added to_is_booland forgotten in the dispatch silently enabled Prometheus metrics instead of doing its job.--doctorwould have been the first victim. Theelsenow raises, andtest_cli.mojoasserts each boolean sets only itself.
Documentation
-
textshelfre-measured after stage 1 (REAL_APP_VALIDATION.md, Revisited). With--realtimeand the pool composing, the recommendation the record pointed at changed, so it was re-measured rather than re-reasoned. Two findings. m0serve's ASGI executor matches uvicorn and daphne to the millisecond on both a sync and an async generator — whatever streams under them streams under it. And the application's own AI streaming endpoints do not stream anywhere, including its production daphne: the producer is a sync generator, which Django's ASGI handler consumes before serving. That makes those endpoints free to move, which leaves the--mount-with---realtimerefusal as the only thing standing between a mixed application and one process — recorded in the ROADMAP as a re-ordering of stage 2, ahead of the WebSocket half. The real-application pass produced one finding about the shape of the server rather than a defect in it:--realtimerefuses--blocking-threads, so the cheapest way to hold a stream (M0-Hold: +2 MB per 200 held, no Python state, no database connection) costs the pool that cures the hostage pathology — measured on textshelf as a 1 543 ms fast-path p50 under--realtimeagainst 0.3 ms with the pool, with eight slow views in flight. The entry records the numbers, the mechanism the executor already uses to solve the identical problem (a reserved begin frame the loop's handler turns into a subscription), a staged design for SSE holds then sockets, the narrower--mountrefusal that follows from it, and what must be shown before it is built. Verdict recorded with it: the larger of the two is the difference between the realtime claim being demonstrable and deployable. -
Three real Django projects, served — the record (REAL_APP_VALIDATION.md). The plan that file used to hold has been executed:
transcripts(plain WSGI,src/layout),color-separation(numpy/Pillow pipelines, uploads, downloads) andtextshelf(four SSE endpoints, three pubsub modules, WhiteNoise, djstripe) served from clean clones against scratch databases, through--doctor, byte-parity againstrunserver, the feature matrix, the topology matrix, a realtime retrofit and a soak. Four defects, all fixed below, three of which no application inapps/could have shown. After the cookie fix, every remaining parity difference on every route of all three apps isconnection: keep-alive,x-thread, or Django's debug page echoing its own port. -
The desktop-Mac hypothesis, and the packaging tension under it (ROADMAP, Open questions). Recorded because the relevant decision is already shipped and otherwise invisible:
poe build-servepins--target-cputoapple-m1, the oldest Apple Silicon, so the PyPI wheel deliberately forfeits M-series-specific capability — including the +sme/+sme2 matrix extension the build comment notes this M4 would otherwise target. The pin exists because the first release crashed with SIGILL in a clean container, so it is not a mistake to undo; it is a tradeoff that points the other way from "exploit the Mac's silicon", and the two should be reconciled deliberately. Also recorded: what has to be established first, including that this toolchain has nogpumodule at all, and that the neural engine is a CoreML surface rather than something a language targets directly.
0.11.0 — 2026-08-26
The release that makes three published claims true at once: the quickstart
works against the PyPI package, pip install m0serve includes the publish
helper the realtime story depends on, and the platform matrix on the
README is the platform matrix on the index.
Added
-
m0serve.m0pubships in the wheel. The publish half of the realtime feature —m0pub.publish(channel, data)from any sync view, oneos.writeper worker plus an atomic fetch-add for the globally unique event id. It was previously only in the repository's demo app, so a pip user had a server that could hold connections and no way to publish to them. Pure stdlib; degrades to 0 workers under any other WSGI server and to unnumbered frames without the shared counter, exactly as documented. -
QUICKSTART.md, and it is executable. Ten minutes from
pip install m0serveto live multi-tab sync from one synchronous Django file — SSE verified by curl with expected output stated, WebSockets in the browser, cross-worker fan-out with--workers 2. CI extracts the fenced blocks and runs them against the tree's own wheel on every pull request (poe smoke-quickstart), so the doc a stranger follows is pinned, not aspirational. It caught its own author before anyone else: a first draft asserted event ids survive a server restart, and the runner failed the doc — ids are unique across one server's workers, by design. -
llms.txt— the operating contract for agents: strict flags, exit 78 refusals that explain themselves, theM0-Holdprotocol, where to start. -
Linux aarch64 wheels (Graviton, Ampere, arm64 Docker). The platform was already proven — built by hand in an arm64 container, passing the full wheel smoke including the removal sabotage — so the only thing between it and users was a CI runner.
build-wheelsandwheel-consume-linuxare now matrices over both Linux architectures, and the aarch64 wheel is consumed on real arm64 hardware rather than under emulation, which would defeat the purpose of a job that exists to run an artifact on a machine that did not build it.Also added to
test.yml, not just the release path:release.ymlruns on a tag, so aarch64-only would have meant discovering a break during a release, after the GitHub release exists and with the upload gated behind it. That is the failure shape the consume jobs were built to prevent.The README claimed Linux arm64 support before the wheel existed, so a Graviton user would have got
No matching distribution found— the literal "didn't install" comment the release checklist names as its first risk. -
scope["client"]andREMOTE_ADDR: the peer reaches Python. The fork'saccept()passed a 4-byteaddrlen, so the kernel truncated the peer address before the IP bytes — it was unreadable even in principle.accept_with_peerkeeps the full sockaddr, the event loop stamps each request (HTTPRequest.remote_addr/remote_port, captured once per connection on the provision), and the peer crosses to Python on both protocols: WSGI getsREMOTE_ADDR/REMOTE_PORTin the environ (per request, C-API only, same no-leak discipline), ASGI getsscope["client"] = (host, port)on http and websocket scopes. Django populatesrequest.METAfrom these only when present —client: Nonedoesn't error, it silently logs every visitor as address-less, which disables rate limits, IP allow-lists and audit logs. -
apps/django_asgi+poe smoke-django-asgi: Django's own ASGI handler, proven. A baredjasgidiscoversdjasgi.asgi:application, detection classifies it ASGI, and the executor serves it zero-config. The smoke pins: discovery + theasgi-loopbanner;/metashowing the real peer (verified load-bearing against a server sendingNone— it answers empty); four overlapping 400 ms async views completing in ~1x;StreamingHttpResponsestreaming live rather than buffered; a signed-cookie session counter surviving three round trips; SIGTERM.smoke-wsgigains the environ-sideREMOTE_ADDRassertion. -
Cross-worker fan-out for ASGI applications:
state["m0"]. Every ASGI app now finds a pub/sub object in its lifespan state — the Channels channel-layer shape with no Redis, riding theBroadcastBusthat already existed for GRIP.m0.publish(channel, payload)writes one datagram per worker channel (m0pub's exact protocol, shared-atomic event ids included, best-effort on a full or dead channel);m0.subscribe(channel)is an async iterator fed by frames the loop's handler forwards to each executor as tagged submit datagrams. The bus fd conflict that blocked this — the executor consumesbus_read_fdfor its ASGI chunk channel — is answered by a second registered fd on the loop (peer_bus_fd), same codec, same drain, same handler entry. The bus (plusSharedAtomicsids and env exports) is now created unconditionally pre-fork: an ASGI app cannot be detected until after the fork, and a single worker publishing to its own subscribers rides its own channel — there is no separate local-delivery path to keep in sync.poe smoke-asgi-fanoutpins the spread (6 streams over 2 workers), delivery of one publish to every stream on both workers, distinct cross-worker ids, supervisor SIGTERM with a live subscriber, and the single-worker case. -
An ASGI server validator — the
wsgiref.validatethat never got written. WSGI has a stdlib conformance checker and this repo runs it; ASGI has nothing standard (theasgireftesting helper plays the server rather than checking one, and uvicorn/hypercorn/daphne verify themselves bespoke).apps/asgi_bare/bareapp/validate.pyis the analog, written from the ASGI 3 spec: every required scope key with its exact type (bytes-vs-str is THE classic server bug), the receive stream's protocol,server/clienttuple shapes.M0_ASGI_VALIDATE=1wraps the app; violations raise and answer 500.smoke-asgigains the validated pass, with/validate/canaryproving the wrapper is engaged — a bogus message type the unvalidated server ignores (200) and the validator refuses (500), the/pep3333/canarypattern exactly. -
--mount PREFIX=SPEC: several applications in one process. Am0serveprocess can now host more than one application, routed by longest prefix before either sees the request —--mount /=djangoproj --mount /portal=portal.wsgi:appserves a Django project and a Flask app from one listener, one set of workers and one graceful shutdown. Each mount detects its own protocol (discovery included) and gets its own bridge and shim namespace; a path no mount claims is a 404 answered in Mojo, never entering Python; prefixes match on segment boundaries, so/appnever swallows/application.The prefix reaches both protocols through one seam, because they disagree about what it means: WSGI gets
SCRIPT_NAMEwithPATH_INFOtrimmed to the remainder, ASGI getsroot_pathwithpathleft whole (Django'sASGIHandlerstrips it itself). Getting that backwards leaves every direct request working while every generated URL breaks, sosmoke-hybridcompares Django'sreverse(), Flask'surl_for()and both frameworks'request.pathbyte for byte. New row:apps/hybrid_mix, deliberately two frameworks rather than two Django projects — those would sharedjango.conf.settingsand the first import would win, which would make the isolation claim a lie.Refused rather than guessed, each with a message saying why: mixed WSGI/ASGI mounts (routing them is done; giving each its native execution mode is the next stage),
--mountwith--realtime(an inbound WebSocket message has no defensible destination among several urlconfs), and a mounted server taking the asyncio executor (one submit channel cannot say which mount a job is for). See docs/WSGI_VS_ASGI.md §9. -
Mixed mounts, each in its native execution mode. A sync Django app and an async FastHTML app now run in ONE process, sharing one listener and one graceful shutdown, with Django's requests on handler-pool threads and FastHTML's on the asyncio executor. The mechanism is a submit lane per mount — the single submit channel became one
SOCK_DGRAMpair each — so the loop hands a job to the worker that can run it; oneProvisionPoolper loop stays, since a slot indexes that loop's provisions.match_path_prefixis now the single implementation of the matching rule, so the lane a job takes and the application the handler picks cannot disagree, and each worker builds only its own mount's application rather than every mount's.Measured and smoke-pinned: with four blocking 2-second Django views holding every pool thread, the FastHTML mount answers at p50 1.3 ms / p99 2.8 ms.
apps/hybrid_mixis now Django + Flask + FastHTML in one process. -
Several ASGI mounts, one executor each. The one-ASGI-mount limit is lifted: every ASGI mount gets its own executor thread, its own bridge and lifespan, and its own drain-ack pair (
OffloadPool.enable_stream_ack), with the loop routing each ack by the lane recorded at submit (slot_lane) — credit belongs to the executor that owns the slot, and an ack routed anywhere else is a stream stalled forever rather than an error, which is why the smoke streams 256 KB (four credit windows) from two executors concurrently and byte-counts both. Executors share the one chunk channel: its datagrams were always slot-addressed, and a singleSOCK_DGRAMqueue is globally FIFO across writers, so the recycled-slot safety argument survives. The reserved channel names now carry the executor's lane (\x01<kind>/<slot>/<lane>; the unmounted wire format is unchanged), which is how disconnect tags and inbound WebSocket messages route back to the owning executor — parsed from the slot's own subscription record, no side table to drift. Shutdown sends one pill per executor on its own lane.apps/hybrid_mixgainsfeed.asgi, a second async mount beside FastHTML's.
0.10.0 — 2026-08-25
Promotes 0.10.0rc1 unchanged. The rc's whole purpose was to run the upload path once on a filename that could be spent: it published, installed from the real index on machines that never built it, and served. Nothing needed fixing afterwards, so this is the same artifact under a stable number.
What the release candidate cost, kept here because the reasoning outlives the incident: three attempts and four defects, every one of them invisible from the machine that built the artifact.
- The binaries were compiled for the build host's CPU (
mojo builddefaults--target-cputo it), som0serve --versiondied with SIGILL in a clean container after passing on the runner that produced it. Not detectable by static inspection at all — only by running the artifact on different silicon. wheel-inspectruns on Linux and checks both wheels, but read Mach-O throughotool, which macOS has and Linux does not.- The glibc negative control ran
pipwith no shell in the container, so its glob stayed literal and pip refused the wheel for the wrong reason. The guard caught precisely that and declined to score it as a pass. - The release published as "Latest", above the current stable, because
gh release createdoes not infer pre-release status from a tag.
Changed
- Version only. No source changes from 0.10.0rc1.
[0.10.0rc1] — 2026-08-25
First release published to PyPI, as a release candidate: it claims the name and exercises the production upload path — trusted publishing, the two-platform wheel set, the tag/version cross-check — before a stable number is spent on an untried path.
One correction, because this entry originally claimed otherwise: pip install m0serve does install it. pip excludes pre-releases only when a
stable version also exists, and there is none here, so the rc is the only
candidate and pip takes it. The rc therefore buys rehearsal, not
invisibility; quietness rests on nothing being announced. The upside is that
the index-install path — pip choosing the right file from several platform
wheels, from a real index, on a machine that never built them — is proven
rather than deferred.
Added
-
pip install m0serve— the server as an installable binary. A WSGI/ASGI server for Python applications, with no Mojo toolchain on the target machine and nothing fetched at install time (the wheel declares no dependencies).m0serve myproject.wsgi:applicationserves either protocol, detected from the object.One wheel per platform covers every supported CPython — 3.10 through 3.14 including free-threaded builds — because
m0servedoes not link libpython; Mojodlopens the interpreter at run time, so there is no CPython ABI in the archive to be compatible with and no CPython in it to redistribute. Verified across all five, and by serving Django 5.2.17 on CPython 3.11 from a wheel built on 3.13 with Django 6.1.Built from
packaging/m0serve/, a separate project holding the only[build-system]in the repository: one in the root would make uv treat the repo as installable, and that build needsbin/m0serve, which needs the.venvuv is creating. -
The platform tag is measured, not declared (
scripts/wheel_tag.py). It readsLC_BUILD_VERSIONand versioned glibc symbols out of the staged binaries and takes the strictest floor. Copying the toolchain's ownmacosx_13_0tag would have shipped a wheel requiring macOS 26. -
poe bundle-serveandpoe check-serve-portable, mirroring thelibm0corepair, plusstage-wheel/build-wheel/smoke-wheel. -
Clean-consumer release jobs. The wheel is installed and made to serve in containers and on a runner that never built it — four CPython minors,
--network none, and a permanent negative control asserting an older glibc is refused by pip rather than crashed at startup. Each job asserts its own cleanliness first, andcheck-docsfails the build if one ever acquires a checkout.
Fixed
-
Binaries were compiled for the machine that built them.
mojo builddefaults--target-cputo the host CPU, so every artifact this project has produced was effectively-march=native. The first release run proved the consequence:m0serve --versiondied withIllegal instruction (core dumped)in a clean container after passing on the runner that built it, and on a developer machine the effective target wasapple-m4with+sme/+sme2— Scalable Matrix Extension, which no M1, M2 or M3 has.build-ffiandbuild-servenow pin the oldest CPU each platform must support (apple-m1,x86-64-v2,generic), andcheck-docsasserts they do. Unlike the rpath defect this resembles, it is invisible to static inspection — only running the artifact on different silicon can find it, which is exactly what the clean-consumer jobs do. -
The portability checker could not see an executable, and the bundler was blind the same way.
otool -Lprints a dylib's ownLC_ID_DYLIBbefore its dependencies; anMH_EXECUTEhas none, so dropping the first entry discardedbin/m0serve's only real dependency. The checker reportedSELF-CONTAINEDfor a binary that resolved the Mojo runtime through a developer's.venv, andbundle_ffi.pywould have copied zero runtime libraries and called the bundle complete. The parse now lives once inscripts/binfmt.py, keyed on the load command's presence, self-tested in CI against both cases. -
build-servehad no post-link surgery at all, so everybin/m0serveever built recorded a search path into the venv that produced it. It now sharesbuild-ffi's, viascripts/relocate.py, and completes its bundle in place — sobin/is the shipped shape rather than a development arrangement that worked for a different reason. -
The macOS deployment target is pinned to 13.0.
mojo buildhonoursMACOSX_DEPLOYMENT_TARGET; without it the binary inherited the build host's SDK, making the wheel's reach a property of whichever image GitHub callsmacos-latest. -
"The Linux artifact is statically linked" was true of one file only. Measured on
libm0core.soand carried as a platform fact; them0serveexecutable links the three Mojo runtime.sofiles on Linux exactly as on macOS. Nothing in the tooling decides by platform now, only by what the file records.patchelfis a Linux build requirement in consequence, andnightly-canary.yml— which buildsm0serve— had no dependency step at all.
Changed
-
NOTICE describes artifacts, not just the source tree. The wheel is the first artifact to redistribute the lightbug_http fork in object form, and MIT requires its notice to travel with copies — so
licenses/LICENSE.lightbug_http.txtandlicenses/NOTICE.m0serve.txtship inside the wheel and them0servebundle. NOTICE gains a table of which notice covers which artifact, and no longer implies the Linux builds contain no Modular code. -
The README's platform claim is architecture-qualified, because once wheels exist
pipenforces it: macOS Intel is not untested but impossible (no toolchain wheel), Linux aarch64 is buildable and unshipped, and the glibc floor excludes musl. The free-threading claim moved from "3.13t+" to 3.14t, which is what is actually tested — 3.13t systematically immortalizes objects, aspyproject.tomlanddocs/WSGI_VS_ASGI.mdalready recorded.
0.9.0 — 2026-08-24
Added
-
ASGI WebSockets (Phase 3b):
app.wsworks. A WebSocket handshake on an ASGI app gets awebsocketscope on the executor's loop. The ready 101 (built by the loop's own validator from the original request's key) is held until the application'swebsocket.accept— the same approve/perform split as M0-Hold — and released through the completion channel behind a FIFO-anchoring begin frame; outbound frames are RFC 6455-encoded executor-side and ride the 3a chunk channel into thesocketsregistry;websocket.closequeues the close frame plus the end marker and the loop closes after both land; inbound messages travel as tagged submit-channel datagrams into per-slot queues behindreceive(); a disconnect cancels the task and a never-answered handshake resolves as 403 so no slot leaks.smoke-asgidrives a raw RFC 6455 probe (verified accept, text and binary echo, close(1000) to the FIN, abrupt-vanish cleanup);smoke-fasthtmlproves FastHTML'sapp.wsend to end — its full surface (pages, SSEEventStream, WebSockets) now runs onm0servewith zero configuration. -
ASGI streaming responses (Phase 3a): SSE actually streams. FastHTML's
EventStream, Starlette'sStreamingResponse, and Datastar patch streams now stream live through the asyncio executor instead of meeting the buffered watchdog's 500. Response chunks travel from the executor thread to the event loop as datagrams on a private per-loop channel and ride the existingSSERegistryper-slot outboxes under reserved channel names (a leading control byte no HTTP header value can carry, so GRIP channels cannot collide). Correctness is ordering and credit, both smoke-pinned: a stream's begin frame precedes its head on one FIFO channel (so a recycled slot can never receive another stream's chunks), a 64 KB credit window with 32 KB chunk split means the loop's drain acks pace the producer (a 100 MB stream behind a slow reader grows server RSS ~2 MB), bodies are close-delimited and the loop closes on end-of-stream via the previously-uncalledsse_is_streaminghook, client disconnects cancel the app task (uvicorn's contract), and comment heartbeats are suppressed on ASGI streams so an SSE event split across chunks can never be corrupted — asserted byte-exact under a 300 ms cadence, alongside an md5-checked 1 MB streamed body.smoke-fasthtmlnow asserts live/sseticks; the buffered escape hatch (--blocking-threads+ ASGI) keeps its watchdog refusal. WebSocket scopes are Phase 3b. -
The asyncio executor: real await-concurrency for ASGI (Phase 2). Zero-config ASGI now serves through one executor thread per event loop (
m0_wsgi.asgi_executor) running the bridge's persistent asyncio loop: the loop parks each request and submits its slot through the unchangedOffloadPooldatagram channel,loop.add_readerturns slots into tasks, and completions answer throughput_response/complete. Requests overlap wherever the application awaits — eight concurrent 1.5 s awaits complete in 1.51 s on one loop with zero threads — and the banner saysasgi-loop. The executor path crosses method, path, query, headers (ready lowercase byte-pairs), and body straight through the C API as stolen tuple slots — no environ, no CGI names, no Python-side re-transform — and the RSS guard stays flat (20 KB–1.7 MB across runs, allocator noise against the 12 MB limit). Exactly one lifespan runs per loop (fallback handlers are built withlifespan=False); the executor picks uvloop for its own loop where installed. An explicit--blocking-threads Nkeeps the Phase-1 buffered pool as the escape hatch;--threads Ncomposes (one executor per loop, free-threaded CPython only, as before).poe bench-asgiis the standing gate against uvicorn: the mixed slow/fast fast-request p99 passes (2.87 ms vs 3.27 ms); hello-world throughput stands at 0.88–0.94x with the remainder located and its fix paths recorded in docs/WSGI_PERFORMANCE.md §"The ASGI executor vs uvicorn". -
m0serveis now a hybrid WSGI/ASGI gateway with zero-config detection. The protocol is detected from the application object at load (coroutine-function duck typing — the rule uvicorn and asgiref share;--protocol auto|wsgi|asgioverrides it), so FastHTML, Starlette, FastAPI, and Django'sasgi.pyserve from the same binary that serves Django/Flask WSGI. ASGI requests run on a persistent per-bridge asyncio loop, buffered: the scope is derived in the shim from the same C-API environ,send()events collect into the same(status, headers, body)tuple, and no new per-request Mojo↔Python object traffic exists —smoke-asgi's RSS guard (same 12 MB/10k-request limit as the Django row) measured 356 KB on day one. Lifespan startup/shutdown run with uvicorn's "auto" semantics, and lifespanstatereaches request scopes. Streaming responses are the recorded limit: a response still unfinished 10 s after its firstmore_body=Trueis answered with an explanatory 500 pointing at docs/WSGI_VS_ASGI.md §8 (the buffered bridge cannot carry an infinite SSE/EventStream; that surface is the design's Phase 3). -
Spec discovery: a bare
m0serve MODULEnow triesMODULE:application,MODULE.asgi:application,MODULE.wsgi:application,MODULE:app,MODULE.main:appin order — a Django project or a FastHTML/FastAPImain.pyserves without learning either convention. An explicit:ATTRnever falls back. A total miss lists every spec tried, and a non-callable names both expected signatures. -
Zero-config concurrency: with no
--workers/--threads/--blocking-threadsflag and noM0_*topology variable,m0servenow starts a handler pool ofmin(cores, 8)blocking threads, so one slow view no longer stalls every connection out of the box. Any explicit topology value wins —M0_WORKERS=1orM0_BLOCKING_THREADS=0restore the old single-loop shape — and--realtimekeeps the single loop (its streaming hooks run on the loop's handler). The banner reportsprotocol=and marks the pool(auto). -
New rows and gates:
apps/asgi_bare(the ASGI sibling ofwsgi_bare— every route pins one clause of the contract) withpoe smoke-asgi, andapps/fasthtml_demowithpoe smoke-fasthtml(skips where python-fasthtml is absent).python-fasthtmljoined the dev dependency group.
Changed
wsgi.multithreadisTrueunder the zero-config pool (it is a real thread pool), and ASGI apps refuse--realtimewith an explanatory message — the M0-Hold contract is a WSGI response-header protocol.- docs/WSGI_VS_ASGI.md gained §8: the deliberate revisit of its §6 verdict, with the three-phase gateway design (buffered bridge → per-loop asyncio executor over the offload channel → ASGI realtime over the existing bus/registry transport).
0.8.0 — 2026-08-24
Added
-
poe bundle-ffi— a self-contained, redistributablelibm0core. The macOS artifact resolves the Mojo runtime at load time and is useless without it, so releases now ship<asset>.tar.gzcontaining the library, the runtime it loads, and both licences — 1.65 MB, verified todlopenfrom an unrelated directory withDYLD_LIBRARY_PATHandDYLD_FALLBACK_LIBRARY_PATHunset. The bare library remains a separate asset so existing download URLs keep working; the Linux.sois statically linked and self-contained already.The dependency closure is discovered, not listed — a hand-written first attempt shipped only the library named in the error message and then failed on its dependency, so
bundle_ffi.pywalks the graph and a toolchain bump that adds a fourth library is picked up rather than silently producing a broken bundle. The task refuses to finish unless the result is self-contained, so assembly and assertion cannot drift apart, and CI runs it on every commit: a release can no longer publish an asset that only loads on the build machine.The Mojo runtime is Apache 2.0 with LLVM Exceptions. Rather than rely on the exception excusing attribution for separately shipped files, the bundle complies with section 4 in full — licence text, attribution, and an explicit 4(b) notice that install name, rpath and code signature were changed while the executable code is byte-for-byte as built. See
NOTICEand docs/FFI_DISTRIBUTION.md, which also record the one piece of contrary evidence: themojo_compilerwheel still declares the proprietary MAX licence, though it contains only the compiler and runtime — no MAX components — and ships no licence file of its own.
Changed
-
The WSGI response path was never measured, and was 10x the request path.
build_responsenow costs 3.30 µs instead of 22.97 for a six-header Django-shaped response;serve()25.51 → 5.65 µs. End to end onapps/wsgi_bare— one response header, the least favourable shape — 49,517 → 56,896 rps (+14.5%), p50 291 → 252 µs.The cause was not the Python boundary. Splitting it put the
PythonObjectheader read at 1.27 µs (5%) andname.lower()at 19.36 µs (84%) — a Unicode-lowercased copy of every header name, allocated solely to compare against one constant.name_iswas already in the repo doing this correctly for the identical Set-Cookie dispatch on the request side, and its docstring already named the mistake. The fix is that one call.scripts/bench_bridge_parts.mojonow covers both directions, at one, six, and six-plus-two-cookie response headers, so this cannot go unpriced again.name_isandascii_lower_bytegained unit tests (test_headers.mojo) — they had none despite being the whole of header case folding, now in both directions; the boundary test was verified to fail when theA–Zrange is widened by one byte. -
The Granian layer split is re-measured on 3.14.7t, and the gap it was written to explain is spent. After five bridge changes, the row that has driven every roadmap priority since it was taken: at four workers m0serve is now ahead of Granian, 101,892 rps against 98,489; at one worker the gap is 2.50x, down from 4.31x.
The re-measurement carries its own validity check. Nothing here touched the HTTP layer or Granian, and all three rows that should not have moved reproduced within 2% —
apps/hello0.99x, Granian 0.98x at one worker and 0.99x at four — across five weeks and a Granian bump to 2.8.2. The two rows that moved are the two the bridge work touched: 3.94x at one worker, 2.91x at four.What is left at one worker is 1.58x HTTP layer × 1.58x bridge — dead even, and their product is the whole 2.50x. The original conclusion, "the headroom is in the bridge, not the HTTP layer", was right when measured (6.30x against 1.59x) and no longer is. Part of the four-worker result is Granian's own 19% loss from one worker to four on a four-performance-core box; m0serve scales 2.08x over the same step. Both stated.
-
m0-sqlite: the text-scan cost is measured, and the zero-allocation read is documented.
bench_sqlite.mojogained TEXT rows — the one column type it never priced — showingcolumn_textpays 2.1x for its per-rowStringat 64 B and at 4 KB alike. The fast path already existed:column_blob_intoworks on TEXT columns (SQLite's UTF-8 TEXT→blob conversion is a pointer handoff), and the two docstrings now point at each other. No new API, per the package's own rule. Also recorded in SQLITE_PERFORMANCE.md: the WSGI-bridge techniques checked item-by-item against this package — most already applied or without an analog, and the one fresh suspect (a hiddenStringLiteral→Stringconversion on every binder's happy path) measured at 0.0 ns and was left alone. -
Each request's environ starts as
PyDict_Copyof a finished base template —build_environ1.78 → 1.56 µs, the bridge 2.35 µs. One C call replaces ten per-request hash-and-stores (58 ns vs 214 measured), andPython().cpython()is acquired once per request instead of sixteen times. The template is copy-isolated: an app may overwrite or delete anything in its environ without touching the next request's, probed with ten vandal/inspect cycles and a secondset_base. The decision that didn't ship matters as much: an intern cache for recurring header names/values measured as a net loss — its hit-path byte-compares cost more than the 15 ns decodes they would skip — so it was never built, and the bridge is now near the structural floor WSGI's environ shape sets. -
The response body is read through
PyBytes_AsStringinstead ofctypes— the bridge costs 2.50 µs per request instead of 3.52.body_byteswas 31% of what the bridge had left, and the split named one cause: the shim'sbody_addr(), which built twoctypesobjects per request. It now runs no Python at all —PyObject_Lengthfor the length,PyBytes_AsStringfor the address, onememcpyfor the copy. 1.07 µs → 0.13 µs, and end to end on one worker servingapps/wsgi_bare: 45,891 → 48,852 rps, +6.7%, p50 315 → 295 µs. That is 1.69x cumulative against the 28,853 rps measured before any of the bridge work.The general finding matters more than the optimisation.
Python().cpython()binds noPyBytes_*and noPyUnicode_DecodeLatin1, andexternal_callcannot reach them either, because libpython is not on the link line — Mojodlopens it, which is whyCPythonis a struct of loaded function pointers. But that struct exposes its handle, and the stdlib's ownExternalFunction[name, type].load(cpy.lib.borrow())opens the functions it omitted. The whole CPython C API is reachable, which retires "there is no binding for it" as a constraint on this boundary.PyBytes_AsStringis stable-ABI and checked — NULL plusTypeErroron a non-bytes, where thePyBytes_AS_STRINGmacro would read wrong offsets and is not a symbol anyway. The pointer is resolved once at construction; the call it returns is 1.0 ns.smoke-django's RSS guard still reports 0 KB over 10k requests — reading through a raw pointer takes no reference.The request body followed in the next entry — see below.
-
The request body crosses as a real
bytesviaPyBytes_FromStringAndSize— the blob design is fully retired. Mojo builds thebytesstraight from the request's own buffer (one copy, inside the call) and hands it to the shim as a stolen tuple slot;io.BytesIO(bytes)shares the buffer copy-on-write, sowsgi.inputcosts no second copy where the old bytearray protocol always paid one. Staging a 1 KB body: 1.6 µs → 0.07 µs (~23x); end to end, a 1 KB POST toapps/wsgi_bare's/input/read: 42.1k → 47.3k rps (+12%), GETs unchanged. Deleted outright: the shim's 64 KB transfer bytearray,buf_addr(), the grow protocol, and the shim'sctypesandsysimports — it now imports nothing butio. Every request costs exactly one call into Python, thePyObject_CallObjectthat runsrun(environ, body).
Fixed
-
The C-ABI artifact no longer records the machine that built it.
build-ffirewrites both paths after the link — install namepackages/m0-core/libm0core.dylib→@rpath/libm0core.dylib, search path/Users/runner/work/.../modular/lib→@loader_path— taking the macOS artifact from BROKEN to SATISFIABLE: it now looks beside itself, so a consumer can supply the runtime. Neither path can be suppressed with a linker flag, becausemojo buildadds them itself; macOS also needs an ad-hoccodesign, since arm64 invalidates the signature on any Mach-O edit and an unsigned dylib will not load at all.smoke-ffiwas silently undoing this: it ran its ownmojo buildinto the same output path, so it overwrotebuild-ffi's output and tested an unfixed artifact. It now depends onbuild-ffiand tests what that task emits, supplying the runtime throughDYLD_LIBRARY_PATH— the documented consumer requirement, exercised rather than accidentally bypassed.Linux was never broken the way macOS is. The published
.sohas noDT_NEEDEDentries at all and is statically linked; itsDT_RUNPATHwas inert debris. The missing-runtime problem is macOS-only, andcheck-ffi-portablenow reports three states —BROKEN,SATISFIABLE,SELF-CONTAINED— because pass/fail could not express that. It also reads ELF itself rather than shelling out: the previous version usedllvm-objdump, which exists on macOS, formats ELF differently, matched nothing, and reported a Linux artifact as portable. A guard that answers "fine" when it cannot read the file is worse than no guard.Still not self-contained: shipping the runtime turns on a licensing question, and the 2026-08-23 nightly still declares the proprietary MAX Platform license — five days after the Apache-2.0 relicensing, so the discrepancy is not a same-day packaging slip. Building the runtime from the Apache-licensed sources was assessed and rejected as disproportionate (an MLIR/Bazel compiler stack for a 1.57 MB macOS-only bundle). See docs/FFI_DISTRIBUTION.md.
-
The published
libm0coreartifacts do not load off the machine that built them, and now there is a check that says so. Every release from v0.1.0 has shipped a C-ABI library whose recorded search path is the CI runner's own directory, sodlopen— the entire point of the artifact — fails for anyone who downloads it.smoke-fficould never catch this: it loads the library in the build tree, where the venv it was linked against still exists, so it passes on exactly the machine where the defect cannot appear.poe check-ffi-portablechecks what a load attempt cannot — that every recorded search path is self-relative and every dependency is a system library or shipped alongside — and fails on the current artifact and on every published one, which is how it was verified. The README no longer claims the prebuilt artifacts are usable as-is.The fix is demonstrated in docs/FFI_DISTRIBUTION.md (the bundled artifact loads from an unrelated directory with a clean environment) but not yet applied: two of the three defects are ours and need no permission, while shipping the Mojo runtime's 1.57 MB three-file closure turns on a licensing question that is genuinely unresolved — Mojo's sources went Apache-2.0 with LLVM Exceptions on 2026-08-18, but the wheel shipping those prebuilt binaries still declares the proprietary MAX Platform license. Recorded in NOTICE.
-
body_bytesno longer swallows a pending exception.PyObject_Lengthanswers -1 with the exception set; folding that into the empty-body case returned an empty list and left the error pending, poisoning whatever C-API call ran next. Unreachable through the shim contract today (_bodyis alwaysbytes), found by review, fixed before it could become reachable.
0.7.0 — 2026-08-23
Added
-
m0serve --blocking-threads N/M0_BLOCKING_THREADS— Stage B, the acceptor and its handler pool. The event loop stops callingHTTPService.func. It parses the request, hands it to one of N handler threads, and returns towait(); the thread calls the handler and pokes the loop back, which encodes and writes the response through the sameRESPONDINGpath every other response takes. One slow view no longer stalls the connections pinned behind it: onapps/wsgi_bare, a fast request answered in 1 ms with two 1.5 s views in flight, against 2.7 s for the identical server without the flag — the same code, the same load, the flag as the only variable.This is the failure the mixed-workload benchmark measured and Stage A did not touch (fast-request p99 1.6 ms → ~194 ms, ~120x, under
--workersand--threadsalike), because a keep-alive connection belongs to the loop that accepted it in both modes and adding loops does not change that. Granian's--blocking-threadsis the same architecture and the reason the design had a working reference.Composes with both execution modes —
--workers Wgives W processes of N handler threads,--threads Tgives T loops of N each, one pool per loop because a job names a slot and a slot means nothing outside the loop whose provision pool it indexes. Off by default: it costs N threads and N handlers' worth of per-thread state per loop, and a server whose views are all fast gains nothing.Unlike
--threads, it is not refused on a GIL-enabled interpreter. A pool under the GIL is what gunicorn's--threadsis: CPU-bound views serialize, but a view waiting on a database, a socket or a sleep releases the GIL and the isolation is real — which is the workload the mode exists for. It is refused together with--realtime, because the streaming hooks (sse_drain_slot,sse_slot_disconnected,ws_message) are called on the loop's handler whilefuncwould run against a pool thread's own registries; half-wiring that would fail quietly. -
lightbug_http.offload— the queue itself, and it knows nothing about Python. TwoSOCK_DGRAMsocketpairs (submit, and a completion channel the loop registers exactly as it registers aBroadcastBuschannel) plus per-slot request/response storage. Datagrams because they preserve message boundaries: N threads receiving on one channel each dequeue one whole job, so the kernel is the queue and there is no mutex to write. Each handoff is published by the socketpair syscall that names it, which is the whole memory-ordering argument.m0_wsgi.blocking_poolis the thread side — the only half that attaches to an interpreter, which is what keeps libpython off everything else's link line.Retiring the pool is one method,
BlockingPool.stop_and_join, and that is a safety property rather than tidiness.next_jobblocks with no timeout, so the poison-pill count must equal the thread count exactly: a thread that receives no pill blocks forever, which is a hungpthread_join. Closing the queue does not rescue it — on Linux, closing the write end of a connectedAF_UNIXSOCK_DGRAMpair does not wake a peer already blocked inrecv, while macOS returns 0 and looks fine. That asymmetry cost a 20-minute CI timeout:test_offload.mojoread one pill more thanstophad sent, to "prove" the close was a backstop, and passed locally while hanging ubuntu. The claim is gone from the code and the count is now a property of the type instead of an agreement between call sites.Three things a slot in flight is not: touched by the loop, swept by the idle or header timeout, or recycled. A client that disconnects mid-job detaches the fd but leaves the provision borrowed until the completion arrives, so a late completion has nowhere wrong to land — a generation counter would detect that race, holding the slot removes it. Past 256 jobs in flight the loop runs requests inline rather than queueing them, which is a bound on what the channels must hold and degrades to exactly the behaviour of a server without the flag.
-
c/socketpair.mojo, extracted frombroadcast.mojo— one binding, two callers, and one deprecated-allocwarning site instead of two. See NOTICE. -
poe smoke-blocking-threads. Phase 1 runs everywhere: the--realtimerefusal, the isolation measurement, four clients abandoned mid-job leaving every slot recovered, HEAD through the pool (the loop has to remember it — by completion time the request belongs to another thread), a raising handler, and SIGTERM answering an in-flight request rather than dropping it. Phase 2 needs the GIL off and ispy-canary's new phase F: two loops of four handler threads each, proving Stage B composes with Stage A — and deliberately loading only half a pool, because past saturation a request queues for a thread, which is what a thread pool is and not what the row asserts. Measured at 1 ms against a 400 ms gate, so the row fails on a broken pool rather than on a busy machine.
Changed
-
The WSGI environ is built in Mojo through the CPython C API — the bridge costs 3.5 µs/request instead of 14.9.
PyDict_NewandPyDict_SetItembuild the dict,PyUnicode_DecodeUTF8builds every key and value, andPyTuple_New/PyTuple_SetItem/PyObject_CallObjecthand the finished dict to the shim. End to end on one worker servingapps/wsgi_barewith a browser-shaped keep-alive request: 28,853 → 45,715 rps, 1.57x, p50 508 → 315 µs, p99 1.06 ms → 681 µs.This retires the last large measured item in the Granian gap. The shim used to rebuild the environ in pure Python on every request, parsing a binary blob Mojo had just written — 28
_read_strcalls for a twelve-header request, 12.09 µs of the bridge's 14.23, 85% of it.The blob existed because Mojo 1.0's
PythonObjectleaks a reference per call argument, so the environ could not be passed as one. The raw C API refcounts explicitly and is not that path, which is what made this possible at all.smoke-django's RSS guard — the instrument for any change to this boundary — still reports 0 KB over 10k requests.The request body still crosses as bytes through the persistent bytearray, because Mojo 1.0 has no
PyBytes_*binding of any kind and abytesobject therefore cannot be built from Mojo. A request with no body now skips that path entirely —buf_addr()is not called and nothing is copied. There is noPyUnicode_DecodeLatin1either, so PEP 3333's latin-1 tunneling is spelled as a UTF-8 encode inenviron.mojoand decoded byPyUnicode_DecodeUTF8into exactly the samestr; ASCII, the overwhelming case, is its own UTF-8 and costs no copy.
Removed
serialize_requestand the request blob format. Nothing crosses the boundary positionally any more except the request body, which needs no framing because its length is passed as an argument.environ.mojokeeps the pure half —cgi_header_nameand the CGI/latin-1 byte transforms — so the mapping stays testable without an interpreter, andtest_environ.mojostill asserts the two statements of the CGI rule agree on every shape it distinguishes.
Fixed
-
Graceful shutdown no longer waits the full 5 s drain for connections that are already finished.
active_countcounts a connection that is merely open the same as one with a request in flight, so a server holding idle keep-alive connections waited out the wholeDRAIN_TIMEOUT_NSbudget at SIGTERM — 5.02 s to exit against 0.02 s idle, in every execution mode, which is most of whatdocker stopallows before it escalates to SIGKILL.The shutdown path now closes slots in
READING_HEADERSwhose receive buffer is empty — "between requests" — before it starts the drain clock. Those connections could never have been served by the drain loop anyway: it dispatchesEVFILT_WRITEonly, so a request arriving mid-drain is not read there. 5.02 s → 0.03 s under--workers 4, under--blocking-threads 4, and on a single loop.A slot mid-request, mid-response, or with a job in a pool thread is left alone, and the SSE/WebSocket farewell still runs first, so streaming clients get their close comment or 1001 frame. Both halves of the contract are pinned:
smoke-blocking-threadsalready asserted that a request in flight at SIGTERM is answered rather than dropped, andsmoke-shutdowngained a phase (scripts/drain_idle_probe.py) asserting idle keep-alive connections do not hold the drain open — checked against the unfixed loop, where it fails at 5.01 s.This also retires the standing suspicion that
--threadsshuts down slowly. It does not, and neither does the pool: every mode exited at 5.02 s, and a 5 s wait loses that race.
0.6.0 — 2026-08-23
The release that finished moving the WSGI examples off their own
server.mojo, and made the bridge 2.35x faster. --realtime and
--health-path carry the hold machinery apps/django_realtime used to own,
--reload re-forks workers onto changed Python in ~300 ms, and
serialize_request — 77% of the bridge's per-request cost — went from 48 µs
to 0.44 µs, taking apps/wsgi_bare from 12,289 to 28,911 rps. The
benchmarking also settled what comes next: one slow view raises fast-request
p99 by ~120x in both execution modes, which is the failure Stage B was
built to remove.
Added
m0serve --realtime— the hold machinery behind a flag.apps/django_realtimewas the last example carrying its ownserver.mojo; everything in it now lives inWSGIHandler. The flag turns on twoSSERegistrys (streams and sockets, holding disjoint slots),take_holdon every application response, the WebSocket handshake a buffered WSGI response cannot produce,ws_message_requestfor inbound frames, and — created before the fork and before the first Python call — theBroadcastBusand theSharedAtomicsid slot with theirM0_BUS_WRITE_FDS/M0_SHARED_ID_ADDRexports.M0_CORE_LIBis discovered rather than demanded: beside the binary first, thenpoe build-ffi's output, and left alone if already set. Off by default, because it costs two slot arrays and because it makesM0-Holda header the server consumes rather than one an application may emit for its own reasons.--realtimeworks under--threads N. The bus is built on the main thread before spawning and loopidrainsread_fd(i), exactly as workeridoes. ASOCK_DGRAMsocketpair does not care whether the peer is a process or a thread, som0pub.pyandsse_peer_frameare unchanged — the publisher reaches N threads with the Nos.writes it used to reach N processes and never learns which it is talking to.smoke-django-realtimephase 4 pins it where the GIL is off (py-canaryC3): six streams spread over four loops, one publish from one thread's Django reaching all six with numbered ids, then a clean four-loop drain on SIGTERM.m0serve --reload [--reload-dir DIR]— hot reload. A changed.pyunder a watched directory stops the workers and forks replacements onto the new module, in ~300 ms plus a drain. The flag forces a supervisor even at one worker and even under--threads N, and that composes with both execution modes for one reason: the supervisor never touches Python. It watches withlistdirandstat— libc, and therefore safe in a process forked withoutexec— and the fork still precedes the first Python call because the supervisor never makes one. A reload is a graceful shutdown followed by a fork: workers leave through the existingSIGTERM→ drain →exit_worker()path, unchanged, and the exits are accounted as a reload rather than a retirement, so the crash-respawn budget is untouched. Stragglers past a 5 s drain deadline getSIGKILL. What reloads is the worker; the Mojo binary is never re-exec'd, so a changed.mojostill needs a rebuild.--reloadsetsPYTHONDONTWRITEBYTECODE=1, which is not a tidiness choice. CPython validates a cached.pycagainst its source's mtime in whole seconds and its size, so a rewrite landing in the same second at the same length looks unchanged to the import system. The reloader sees it — it compares nanoseconds — re-forks, and the fresh worker imports the old bytecode: a reload that visibly happened and changed nothing. Writing no bytecode means there is never a cache to go stale.smoke-reloadpins it by editing same-length versions in the same second, and asserts no__pycache__appears.- The
wrkkeep-alive tail row, and the Stage B go/no-go (docs/WSGI_PERFORMANCE.md,scripts/bench_wsgi_tail.sh+scripts/bench_wsgi_tail_ka.sh). Three rounds on 3.14.7t: keep-alive p99 is 1.6–2.9 ms across--workersand--threadsat both 2 and 4, so the 84 ms tail does not reproduce as a property of the design; the single excursion in seventeen valid rows was in prefork. Stage B is a no-go on this evidence, and the gate is restated as a mixed-workload run, because a hello-route benchmark cannot exercise the slow-view isolation Stage B is half about. Granian 2.8.1 measured at 1.4–2.0x either mode on the same interpreter with a byte-identical response — recorded as the better-evidenced target. Also recorded: the ephemeral-port exhaustion that made the firstwrktable report a spurious 8–10x threads-vs-prefork tail gap and five empty rows, and why gunicorn cannot appear in a keep-alive table at all. scheme_separator(lightbug_http/uri.mojo, so also in NOTICE). See Fixed.MtimeScanner(m0-http): the change detector, suffix-filtered with the suffix supplied by the caller som0-httpkeeps no notion of what a source file is. One number per pass — newest mtime and file count, because deleting the newest file leaves the maximum in the past — compared against the previous pass rather than a high-water mark.__pycache__,.git,node_modulesand dotfiles are skipped; the first pass records and never reports.waitpid_nonblocking(WNOHANG) is what lets the supervisor poll instead of parking inwait.--health-path PATHanswersPATHin Mojo with a liveness JSON — under--realtime, with the livesubscribersandsocketscounts, which is how the smokes assert that a vanished client was actually unsubscribed. Opt-in, and separate from--realtime, for the mirror-image reason: an application may already route/health, and a server that took the path silently would shadow it.
Changed
-
The WSGI bridge is 2.35x faster: 12,289 → 28,911 rps on
apps/wsgi_bareat one worker, p50 1.21 ms → 508 µs, p99 2.47 ms → 1.07 ms (same interpreter, two rounds).serialize_requestcost 48 µs per request — 77% of the bridge's whole per-request cost, and six times what the Python shim it feeds costs. Enumerating headers withkeys()+get()allocated a String per name and per value and linear-scanned for each, andcgi_header_nameallocated three more per header: ~70 String allocations to move twelve headers. The projection now walkscount()with the header map's own spans and writes the CGI name's bytes directly, uppercasing and mapping-to_in place — 48.10 µs → 0.44 µs. PEP 3333 conformance green;smoke-django's RSS guard still 0 KB over 10k requests.Recorded because the suspicion was wrong: the Python shim's environ parse looked like the culprit and is only 11.5 µs.
scripts/bench_bridge_parts.mojois the split that found it, andhandle()is now five-sixths of what remains. -
Headers.name_span/value_spanare public (were_name_span/_value_span), so headers can be projected into another representation without allocating.keys()is unchanged and still right for callers that want owned Strings. A fork change; see NOTICE. -
Stage B is justified — reversing the no-go recorded earlier in this release cycle. That verdict came from a keep-alive benchmark on a hello route, which cannot produce the failure Stage B is half designed for, and it named a mixed-workload run as the gate. That run (
scripts/bench_mixed_workload.sh, 3.14.7t, two rounds) is decisive: one slow view alongside fast traffic takes fast-request p99 from 1.6 ms to ~194 ms (~120x) while p50 does not move — a subset of connections stopped dead, not general slowdown.--threadsis affected identically, because a keep-alive connection belongs to the loop that accepted it in both modes. Granian's--blocking-threads, which is the Stage B architecture, is flat under the same load (0.96 → 1.22 ms). -
The Granian throughput gap is the WSGI bridge, not the HTTP layer or the concurrency model (
scripts/bench_layer_split.sh). Three rows differing by one layer, byte-identical 13-byte response:apps/hello(zero Python) 78.3k rps at 178 µs, the same HTTP layer through the bridge 12.4k at 1.18 ms, Granian through its own bridge 124.6k at 109 µs. The bridge costs ~1 ms per request because the shim rebuilds the WSGI environ in pure Python every time (~28 string decodes for a twelve-header request) — itself downstream of thePythonObjectreference leak that forced the blob design. Building the environ through the raw CPython C API sidesteps the leak;PyDict_New/PyDict_SetItemwere compile-checked as reachable viaPython().cpython(). -
apps/django_wsgi's/slowaccepts?ms=, defaulting to the 1500 mssmoke-djangoexpects. The mixed-workload benchmark needs a much shorter hold — 1.5 s swamps the signal instead of measuring it.
Fixed
- A bare
://anywhere in a request target was read as a scheme, so a query parameter carrying an unencoded URL (/go?url=http://x) was parsed as a URI whose scheme was/go?url=httpand answered400before reaching the application.URI.parsesearched the whole target for://;scheme_separatornow accepts one only when everything before it is a scheme as RFC 3986 §3.1 defines it — ALPHA, then ALPHA / DIGIT /+/-/.— a character set that by construction cannot contain/,?or#. It is computed before theByteReaderborrows the string: a second interior reference taken while the reader holds one invalidates it. Clients that percent-encode — every browser form, everyurlencode— never hit this, which is what kept it a Known issue rather than a bug report.test_uri_scheme.mojocovers both directions andsmoke-wsginow sends its/reentrant?url=unencoded as well as encoded, so a real server proves it.
Removed
apps/django_realtime/server.mojo, and with itM0_DJANGO_PROJECT. The row keepsm0pub.py,djangoproj/,realtime_probe.pyandstatic/, and is served bybin/m0serve djangoproj.wsgi:application --app-dir apps/django_realtime --realtime --health-path /health. No WSGI row has aserver.mojoany more.
0.5.0 — 2026-08-22
The release the server grew a command line and a second way to be
concurrent. m0serve is one built binary that serves any WSGI application,
so three example apps stopped carrying a server.mojo each; --threads N
runs N event loops on N pthreads in one process on free-threaded CPython,
at throughput parity with prefork for ~60% of its RSS.
Added
--threads N/M0_THREADS— the threaded execution mode (free-threaded CPython only). N event loops on N pthreads in one process, one interpreter: each thread runs its ownrun_event_loopwith its ownWSGIHandler, and so its ownWSGIApp, bridge and shim namespace — the bridge's per-process singletons become per-thread without a line of the bridge changing, and the event loop is untouched. What it buys: one RSS instead of N, the app imported once, and the whole fork-after-init hazard class gone. What it does not: a keep-alive connection stays pinned to its loop, exactly as under prefork, so the keep-alive p99 shape is unchanged (the thread-pool stage is recorded in ROADMAP.md).m0_wsgi.threadedis the choreography — main initializes and imports before spawning and then detaches; every thread attaches once, serves, and releases;DetachingBackendwraps the loop's one blocking wait so a parked thread never stalls the others' stop-the-world; the process-wide signal pipe wakes a coordinator that pokes one shutdown pipe per thread. A GIL-enabled interpreter refuses to start with exit 78 and a sentence naming the requirement — never warns-and-runs.--threadsand--workersare mutually exclusive.wsgi.multithreadis finally True somewhere; every response carriesx-thread.smoke-threadspins the guard on every runner and the mode itself on the free-threaded canary (phase D ofpy-canary) — green on both backends as of 2026-08-23: kqueue on macOS and epoll on Linux, with all four loops accepting in each, so a listener dup'd into N epoll instances underEPOLLETwakes every one of them and noEPOLLEXCLUSIVEfollow-up is owed.- The threads-vs-prefork benchmark row (
docs/WSGI_PERFORMANCE.md,scripts/bench_wsgi_modes.sh): on 3.14.7t,--threads Nis at throughput parity with--workers N(0.92–1.05x) at ~60% of its RSS, ~3.5x gunicorn on the same free-threaded interpreter. ThreadHandler(m0-wsgi): anHTTPServicethat constructs itself on a serving thread via a staticmake(ctx). A trait rather than a function parameter because Mojo 1.0 cannot materialize a function-parameterizeddefas the runtime value a pthread needs;WSGIHandlerimplements it from theServeOptionsatctx.user.m0_http.threads— raw pthreads from Mojo, packaged:ThreadSet(malloc'd Int64 argument blocks,pthread_create/pthread_jointhroughexternal_call, adef's address as the start routine),ThreadBlock,ShutdownFanout(one shutdown pipe per thread, poked together — the event loop never drains its pipe, so N loops cannot share one),dup_fdandread_one_byte_blocking. The idiomscripts/py_thread_probe.mojoproved, now importable undermojo runand tested without an interpreter (test_threads.mojo). Knows nothing about Python; that discipline belongs tom0-wsgi. The substrate for the threaded execution mode — nothing consumes it yet.M0_THREADSis read byAppConfig(default 1) andthreads_conflict(workers, threads)answers the one message for asking for both execution modes at once. Mutually exclusive withM0_WORKERS>1. Read and validated ahead of the mode that will consume it, so the environment andm0serve --threadswill say the same sentence.m0serve— the uvicorn-shaped serve CLI. One built binary (poe build-serve→bin/m0serve) serves any WSGI application:m0serve MODULE[:ATTR] --host --port --workers --app-dir --static PREFIX=DIR --static-cache-control --access-log --max-body --metrics.ATTRdefaults toapplication;--app-dir(default.) is prepended tosys.path. EveryM0_*variable keeps its meaning with the matching flag winning over it, and flags are strict where the environment loader is lenient —--port 80eightyis a usage error (exit 2), not a silent default. Startup failures exit 1 and name the thing (a missing app dir is caught before any interpreter starts; a module or attribute that will not import is reported in Python's own words).--max-bodyand--metricsare the first two server-onlyServerConfigtunings a command line can reach. The entry file lives at the package root (packages/m0-wsgi/m0serve.mojo), outsidesrc/, for the reasonsm0-core/ffi_exports.mojodocuments; the parser (src/cli.mojo) is pure and interpreter-free, tested intest-wsgi.WSGIHandler(m0-wsgi): the one copy of the handler three example apps used to carry identically, with static mounts (List[StaticFiles]) answered in Mojo ahead of the bridge.M0_HOST: the listen address, read byAppConfigand honoured byaddress(). An IPv4 literal, orlocalhostfor127.0.0.1; the listener is IPv4-only and resolves nothing.poe smoke-serve:--help/--version, the usage and startup exit codes (including a supervisor that gives up under--workers), flag-over-env precedence, a static mount with itsCache-Control,--max-body→ 413,--metrics, and a graceful SIGTERM.
Changed
- The Django, Flask and bare-WSGI rows are Python-only projects served by
m0serve;serve-django,serve-flask,serve-wsgi-bareand the three smokes build the CLI once and reuse it. What the rows assert is unchanged. WorkerSupervisorexits 1, not 0, when its respawn budget runs out with a worker still dead — a worker that crashes on every attempt usually could not start at all (a bad module path), and exiting 0 reported success to whatever launched the server.test_respawn.mojopins it.
Removed
apps/django_wsgi/server.mojo,apps/flask_wsgi/server.mojoandapps/wsgi_bare/server.mojo, and with them theM0_FLASK_PROJECTandM0_WSGI_PROJECTvariables — replaced bym0serve … --app-dir.apps/django_realtime/server.mojokeeps its ownmain()andM0_DJANGO_PROJECTuntil the hold/publish machinery moves behindm0serveflags.
0.4.0 — 2026-08-22
The release the WSGI host grew up in. m0-wsgi went from a spike to a
framework-agnostic PEP 3333 server with a conformance suite, a second
framework row, and a realtime story that gives synchronous Django the SSE
and WebSocket surface people adopt ASGI for. Separately, the HTTP hot path
got substantially faster — headers alone are worth +72% throughput.
Added
-
In-process GRIP: sync Django holds SSE streams and WebSockets. A view answers an ordinary buffered response carrying
M0-Hold: streamorM0-Hold: websocketplusM0-Channel, and the server takes the connection from there.take_hold(m0-wsgi) consumes the instruction headers; an SSE hold keeps the view's body as the head of the stream, and a WebSocket hold cannot — a handshake answers101with aSec-WebSocket-Acceptderived from the client's key, which a buffered, re-encoded WSGI response has no way to produce. So Django approves and the Mojo layer performs the upgrade. Inbound frames make the return trip as ordinary requests:ws_message_requestgives a message the shape of aPOST(payload as body, channel/slot/opcode asM0-headers) and a plain synchronous view handles it.Under a server that has never heard of these headers the same views degrade to short buffered responses — the GRIP property. The header names are M0-prefixed because this is GRIP-shaped, not GRIP-compatible.
-
Publishing that never enters Mojo. The server exports its
BroadcastBuswrite fds once, pre-fork, asM0_BUS_WRITE_FDS;m0pub.py(stdlib only) frames an event andos.writes one datagram per worker, including its own. NoPythonObjectcrosses the bridge, so the reference leak rule andsmoke-django's RSS guard are untouched. One line in a sync view reaches SSE and WebSocket subscribers on every worker: the bus carries one SSE frame and delivery re-encodes per slot, so anEventSourceclient and a WebSocket client on the same channel see byte-identical messages. -
Numbered event ids, so
Last-Event-IDmeans something. Each publish fetch-adds oneInt64on theMAP_SHAREDpage the server allocates pre-fork, and the number goes into the bus datagram's id field and onto the wire as anid:line — which engagesSSERegistry's redelivery filter. An SSE hold seeds that filter from the request'sLast-Event-ID. Python has no atomic read-modify-write over a raw address, som0_shared_fetch_addjoins m0-core's C ABI andm0pubcalls it throughctypes. AbsentM0_CORE_LIB/M0_SHARED_ID_ADDRit degrades to unnumbered frames — the only behaviour available under a plain WSGI host. Suppression, not replay: catching a client up on missed events needs a journal, whichDatastarStreamhas and the raw registry does not.apps/django_realtimeis the working demo;poe smoke-django-realtimeandpoe smoke-django-realtime-wspin it, the latter with four held connections across two workers — one SSE stream and one socket each — all reached by ONE synchronous Django publish. -
PEP 3333 conformance testing, framework-free.
apps/wsgi_bareis a plain WSGI callable with no third-party imports, andpoe smoke-wsgiis the conformance suite over it: thewrite()callable, a secondstart_response, multi-chunk iterables,close(), arbitrary status passthrough, andwsgi.inputread patterns.smoke-djangogains a pass underM0_WSGI_VALIDATE=1, wrapping the app inwsgiref.validate, with a/pep3333/canaryroute that must fail under the wrapper so a misspelled variable cannot silently downgrade it to a second unvalidated run. Reasoning, including why repointing Django's owntests/servers/here was rejected, is in docs/WSGI_CONFORMANCE.md. -
Flask as a second framework row.
apps/flask_wsgiandpoe smoke-flask, with the assertions both rows share extracted intoscripts/wsgi_framework_contract.sh— routing, both directions of the cookie path, body round trips past the shim's 64KB transfer buffer, binary safety, query parsing, the framework's own 404, and a raising view becoming a 500. A row needing assertions of its own would be evidence the host is not framework-agnostic after all. Adding Flask needed no change tom0-wsgi. -
Graceful shutdown on SIGTERM/SIGINT. The loop always knew how to drain — close the listener,
: closeto SSE clients, a 1001 frame to WebSocket clients, in-flight requests for up to 5s — and nothing could ask it to.install_shutdown_signals()returns the fd to pass asshutdown_read_fd. Mojo has no globalvarand a POSIX handler gets no user-data pointer, sosrc/global_slot.mojoreachespop.global_allocfor what C spellsstatic; if that ever stops working nothing is installed and the default disposition stands, whichshutdown_signals_active()reports.WorkerSupervisorpropagates a signal aimed at the supervisor alone to its children — whatdocker stopdoes, and what used to leave workers orphaned on the port.poe smoke-shutdowncovers both paths. -
Static files ahead of the bridge.
StaticFilesgrew aCache-Controlpolicy, emitted on 200/206/304 (a validator response carries freshness too, per RFC 9110), and the Django rows mount it: asset requests are answered in Mojo with type, ETag revalidation and traversal 404s, and never enter Python. WhiteNoise has nothing left to do. Zero-copysendfileremains recorded, not built — it needs event-loop support for fd-backed response bodies. -
m0-sqlite: the result codes callers actually branch on are exported (SQLITE_CONSTRAINT,SQLITE_RANGE,SQLITE_NOMEM, plusSQLITE_OPEN_NOMUTEX/FULLMUTEXso a caller assembling flags can reproduce the package's threading model).sum_ints,min_ints,max_intsandstats_intsare the SIMD passfetch_ints' column-major layout was written for: over 200k rows,fetch_ints + stats_intsbeatsSELECT sum(v), min(v), max(v)8.63 ms to 11.84 ms — mostly because the read-out runs one column fetch per row where SQLite runs three aggregate steps through its bytecode VM, not because of the vectorization, which is 0.5% of that pipeline. -
sse_data_payload(m0-http) — the inverse offormat_sse_event, returning what a browser'sEventSourcehands toonmessage; andSSERegistry.filter_url, the inverse ofsubscribe. -
CI that cannot quietly rot. A warning ratchet (
scripts/warning_ratchet.py,poe check-warnings) holds the unique warning count at a committed baseline, becausemojohas no per-warning suppression and warning number 69 would otherwise land among 68 residual ones unnoticed.poe canaryruns the whole Mojo-nightly probe in one command, with the toolchain restore in anEXITtrap so it happens even when the canary fails.poe py-canaryruns the WSGI suite against free-threaded CPython 3.14t and now runs weekly;poe py-thread-probemeasures Mojo-spawned pthreads calling Python — 3.96x at 4 threads on thread-local state, and 0.71x on a shared dict, a confirmed per-objectPyMutexmechanism recorded in docs/WSGI_VS_ASGI.md.
Performance
Each figure is against its own baseline in its own session; docs/SERVER_PERFORMANCE.md records a 1.7x session-to-session swing on identical binaries, so they do not chain.
- Headers stored as spans into a flat buffer, not a
Dict: +72% throughput — 29,000 → 50,000 req/s onapps/hello, p50 535 → 320 µs, five alternating A/B rounds. ADict[String, String]cost two allocations per header to fill, a third to lowercase each name, and a fourth per lookup, becausekey.lower()builds a probe copy before it can hash. One blob indexed by parallel (offset, length) arrays makes a lookup a linear scan that compares lengths first and allocates nothing — and preserves insertion order, which theDictnever guaranteed. - The hot path cut to two syscalls per request: +24% — 15.2k → 18.9k
req/s, p50 1.03 → 0.83 ms. Persistent read-filter registration
(
slot_read_armed) removes anepoll_ctlADD that failedEEXISTevery time and the MOD it fell back to; idle timeouts move to a once-a-second deadline sweep instead of a per-requesttimerfd_settime;TCP_NODELAYon accepted sockets. Router.matchon spans: 2.8x on/health(158 → 57 ns). Patterns live in one flat blob and matching walks the path by moving span endpoints; nothing allocates until a parameter is captured on a route that matched, and a 404 allocates nothing at all.- WebSocket unmasking and UTF-8 validation vectorized — the 4-byte mask splats across 64 lanes (64 is a multiple of 4, so the pattern stays phase-aligned and the scalar tail needs no special case), and text validation skips pure-ASCII runs 64 bytes at a time while every non-ASCII byte still goes through the same strict decoder.
- Startup RSS down 47–64% — connection buffers are sized on first use
rather than at pool construction, so a server no longer allocates for its
configured ceiling before accepting anything.
apps/hello: 26.5 → 14.1 MB at one worker, 77.7 → 28.0 MB at four. - Log lines assemble in one buffer instead of a dozen
Strings. - The per-slot response buffer is reused via
encode_into, which had sat unused behind a comment claiming Mojo could not move out of a list-element field. It can, byswap. The honest result is 1.04x, andSERVER_PERFORMANCE.mdwas corrected to say so rather than leaving the item ranked first.
Changed
- The Django example enables
django.contrib.sessionson the signed-cookie backend, so it still needs no database.poe smoke-djangoasserts request cookies reach a view intact (including a value containing=), that splitCookiefields rejoin, that a cookieless request stays cookieless, and that a session counter advances across three requests. AppConfigmaps toServerConfigin one place instead of once per app.- Mojo 1.0 deprecations cleared across the repo where a replacement ships:
the memory and origin APIs, positional pointer indexing,
memcpy→unsafe_memcpy,deinit take:→deinit move:, andhttp/common_response.mojoimporting the names it uses instead of resolving them through a star-import — that pattern alone accounted for 57 of the repository's then-143 unique warnings. - The cross-worker smokes place one stream per worker deterministically
(SIGSTOP the worker that won the first open, then open the second) rather
than racing accept. Which worker wins is the kernel's choice and it is not
a fair one: a macOS runner handed a single worker all 24 opens across six
rounds, and opening in bursts made it worse, because the accept path
drains the backlog until
EAGAINand the first worker to wake takes the whole burst.
Fixed
-
Request cookies never reached a WSGI application. The request parser diverted
Cookieout of the header map intoRequestCookieJar, and the WSGI environ is built by walking the header map — soHTTP_COOKIEwas absent andrequest.COOKIESwas always empty. Every Django session, login, CSRF check and message silently behaved as though the visitor had arrived with no cookies at all.Cookienow stays inheadersas well as feeding the jar, and severalCookiefields are rejoined into one"; "-separated list (RFC 6265 §5.4) rather than collapsing to the last one.Set-Cookieon a request is no longer folded into the request's own cookies — it is a response field, and treating it as one invented a cookie the client never sent.Because a parsed request now carries its cookies in both places,
encodeandwrite_towrite the jar only whenheadersdoes not already carry the field, so re-encoding a parsed request still emits oneCookie. -
RequestCookieJarmis-parsed values, and its lookups did not match its storage. Pairs were split on every=rather than the first, so any value containing one was truncated at the first segment — base64 pads with=, so a Djangosessionidroutinely lost its tail. Splitting also ran over the whole field instead of per cookie, soa=1; b=2parsed as one cookieaholding1; b. A pair with no=was stored under the empty name instead of being ignored (RFC 6265 §5.2). And__getitem__lowercased the key while stores,__contains__andto_headerdid not, so a jar holdingsessionIdanswered nothing to any spelling; cookie names are case-sensitive (RFC 6265 §4.1.1) and are now treated that way throughout. The jar's ownparse_cookieswas dead code —HTTPRequesthand-rolled a separate, buggier copy — and both now share one path. -
A request body that could not be read in one
recvnever completed. The event loop registered read interest only while a connection was inREADING_HEADERS; once headers parsed and the state moved toREADING_BODY, nothing armedEVFILT_READagain. Since epoll is edge-triggered, body bytes already waiting in the socket buffer raised no further edge either, so the connection stalled untilbody_read_timeoutanswered408. Both the transition intoREADING_BODYand each incomplete body read now re-register read interest.This hit every request whose body did not arrive inside the first 4KB staging read — any POST or PUT over ~4KB, and any request at all whose client flushed headers before the body, regardless of size. It affected every app in the repo, not just the WSGI host: Django form posts, file uploads and JSON APIs all timed out.
poe smoke-djangonow posts a 256KB binary body and a header-flushed-first body to/echoand compares the echo byte for byte. -
write()discarded every byte. The WSGI shim returnedlambda data: None, so an application using the legacy write callable got a 200 with an empty body and no error anywhere. Django never callswrite(), so nothing Django-shaped could have caught it — including thewsgiref.validatepass, which type-checks the call and not its effect. The iterable is now drained before the writes are joined, because an application may callwrite()from inside the generator it returned. Found byapps/wsgi_barewithin minutes of its existing. -
A second
start_responsewithoutexc_infowas silently accepted, last call winning. PEP 3333 makes it an application error; it now raises. Withexc_info, replacing the stored status and headers is always correct for a fully-buffering server, since nothing has ever been sent. -
Keep-alive connections answered
408to prompt requests.slot_header_startwas re-stamped in_after_send, so the header read deadline measured from the end of the previous response rather than the start of the current request: any connection idle longer thanheader_read_timeoutgot a408for a request the client had just sent promptly and completely. Measured at the boundary — a 9s gap answered 200, an 11s gap answered 408. -
m0-sqliteanswered questions it should have refused.sqlite3_column_namereturns NULL pastcolumn_count()andcstr_lendereferenced it, segfaulting on an out-of-range index; NULL is now guarded incstr_len, one place for every caller. Out-of-range reads were indistinguishable from NULL —column_intanswered 0,column_text"", andis_nullanswered True for a column that does not exist, so the one accessor whose job is removing that ambiguity was adding one. Every reader now bounds-checks, re-reading the count per call becauseprepare_v2silently re-prepares after a schema change and aSELECT *can change its column count mid-life. -
verify_layout.cguarded shipped code and nothing ran it. It re-derives with_Static_assertevery SQLite struct offsetvtab.mojohardcodes as a flat word buffer, but sat inexperiments/, which nothing builds — so the claim that "every offset used here is asserted against the real headers" was aspirational, guarding a failure mode (a wrong offset silently corrupting every row after the first) this repo has shipped once before. It moved intopackages/m0-sqlite/test/andpoe test-sqliteruns it first. -
Three latent faults in the lightbug fork, each reachable only when the import graph is entered from a particular direction and so invisible until one was:
cookie/request_cookie_jar.mojousedHeaderswithout importing it (it resolved through theheader↔http.parsingcycle on the usual path), anduri.mojo's__str__andis_httpstill calledlen(String), which Mojo 1.0 rejects. Bodies elaborate on demand, so all three compiled cleanly until a consumer reached them.
Known limits
Newly documented rather than newly true, and worth knowing before turning on more workers:
urlopenfrom inside a view SIGKILLs the worker on macOS underM0_WORKERS>1:_scproxycalls into CoreFoundation, and Objective-C aborts rather than run in a process forked without exec. Usehttp.client.HTTPConnection, which does no proxy lookup;apps/wsgi_bare's/reentrantroute is the worked example andpoe smoke-wsgipins it. The general rule — afterfork()withoutexec, platform runtimes are off limits, from application code too — is what the free-threading path is expected to retire.
0.3.0 — 2026-08-18
- WebSocket text messages are now validated as UTF-8 (RFC 6455 §8.1) on the assembled message — a multi-byte character split across fragments is fine; an invalid sequence closes with 1007. Binary frames still carry any bytes.
StaticFileshonours single byte ranges (RFC 9110 §14):bytes=a-b,bytes=a-, andbytes=-suffixanswer206+Content-Range; parseable-but-past-the-end answers416withbytes */total; multiple ranges and other units are ignored (full200, as the RFC permits).Accept-Ranges: bytesis advertised;If-Rangedeliberately never matches (weak ETags, strong comparison required) and falls back to the full representation.apps/hello(and the README example) now use the non-blocking event loop — the blocking accept loop remains in the fork but has no in-repo app consumers left.Clientkeep-alive: response boundaries are now computed (classify_response— Content-Length, chunked terminal chunk + trailers, bodiless statuses, HEAD) instead of inferred from EOF, and the connection is kept warm and reused across requests to the same host and port. Reuse rules are conservative (aConnection: closeresponse, an HTTP/1.0 peer, a close-delimited body, or stray bytes past the boundary all retire the connection); a reused connection that dies before yielding a single response byte is retried once on a fresh dial.keep_alive=Falserestores one-connection-per-request.connections_openedreports dials; the smoke asserts a six-request conversation (HEAD included) rides one connection. Breaking:request/get/postnow takemut self.m0_http.WSHub— the handler-side WebSocket registry: connected slots, per-slot outboxes, room broadcast, and cross-worker fan-out over the sameBroadcastBusSSE uses (the bus is transport-agnostic;sse_peer_framedelivers encoded WebSocket frames as readily as SSE events). Newapps/ws_chatdemo — one room, every message reaching every socket acrossM0_WORKERS— andpoe smoke-chat, which proves a message sent on one worker's socket arrives on the other worker's.
0.2.0 — 2026-08-17
- WebSockets (RFC 6455), server side:
websocket_upgradeanswers the opening handshake from an ordinary handler, the event loop parses frames (client masking enforced, fragments assembled, ping/pong and the close handshake answered in the loop), and complete messages arrive at the newHTTPService.ws_messagehook — the ninth trait method, empty in handlers that never upgrade. Outbox, heartbeat (a protocol ping on theM0_SSE_HEARTBEAT_MScadence), and disconnect plumbing are shared with SSE. Protocol violations answer with the RFC's close codes (1002/1009). Newapps/ws_echodemo;poe smoke-wsproves the wire format against a from-scratch stdlib client. Also fixed in passing: a stale keep-alive idle timer could fire mid-stream and kill an SSE connection opened on a reused keep-alive connection. m0_http.StaticFiles— static file serving: a directory mounted under a URL prefix, with lexical path-traversal defense (decoded.././empty segments answer 404), extension-based content types, and ETag/304revalidation. The notes example serves/static/with it.HTTPService.tick(now_ms)— the application timer hook, fired everyM0_APP_TICK_MSmilliseconds (0 = off, the default) on the event loop's timer. Server-initiated pushes no longer need an inbound request; the counter demo gained a live uptime clock driven by it, ticking on one designated worker and reaching every worker's tabs over the broadcast bus. Breaking for handler authors: the trait gains an eighth method (emptytickin non-scheduling handlers).
0.1.0 — 2026-08-17
First release. Everything below is new.
The server (lightbug_http, a maintained hard fork)
- HTTP/1.1 server, Linux (
epoll) and macOS (kqueue), forked from lightbug_http v26.1.2 after upstream was archived — see NOTICE and PROVENANCE.md. - Non-blocking event loop: multiplexed keep-alive connections,
header/body/idle timeouts, graceful shutdown that drains in-flight
requests, opt-in Prometheus-format
/__metrics. - Request-parsing hardening: request smuggling (CL+TE, duplicate
Content-Length,chunkednot last), header-count and size caps, request-target normalization, chunked-size integer overflow — each guard pinned by a test verified to fail without it. - SSE as a first-class server concern: per-slot outboxes with backpressure,
Last-Event-IDredelivery suppression, heartbeats on idle streams (M0_SSE_HEARTBEAT_MS) that double as dead-subscriber detection. - Cross-worker SSE fan-out: a pre-fork
BroadcastBus(one datagram channel per worker) plusSharedAtomicsevent ids makeM0_WORKERS>1and SSE compose; a broadcast on any worker reaches every worker's subscribers. fcntl(F_SETFL)fixed on ARM64 macOS (Darwin passes variadic arguments on the stack):set_nonblockingnow actually works there, which is what lets two workers race on one shared listener without the loser blocking insideaccept().- Outbound
Client: GET/POST/any method with full response parsing — Content-Length with loud truncation detection, chunked, and close-delimited bodies.
The framework (m0-http)
- Router with
:paramcaptures and real405+Allow. - Content negotiation:
Accept(quality factors, wildcards,*/*resolves to JSON),Accept-Encoding(codec-agnostic, RFC 9110identity/*/q=0 rules),Accept-Language(RFC 4647 matching, serve-something-over-406). - Weak ETags (wyhash) with
304 Not Modified, URL-keyed response cache. - API-key auth with constant-time comparison, CORS hooks, health/readiness
registry, JSON-lines access logging,
M0_-prefixed env configuration. - Multi-worker fork supervisor with crash respawn: workers accept from one shared pre-fork listener; a respawned worker takes over its predecessor's identity (index, bus channel).
Datastar (m0-datastar)
- Datastar v1.0.2 wire format with zero dependencies (
consts,sse), so frames are usable without the framework. DatastarStream: subscriptions, five broadcast shapes,read_signals, andLast-Event-IDreplay from a bounded frame journal — including across restarts when the app persists the journal (the todo example does, in ~15 lines of SQLite).
WSGI (m0-wsgi)
- Runs Django (or any WSGI app) on this server by embedding CPython. Bodies
cross the boundary as raw addresses (Mojo 1.0 binds no
bytesAPI), and per-request data avoids the toolchain'sPythonObjectreference leak by design — an RSS guard in CI keeps it that way. Prefork viaM0_WORKERS, benchmarked at ~1.6–2.2x gunicorn's throughput on the same Django app.
SQLite (m0-sqlite)
- Connections, statements, typed columns, transactions, bulk read-out — WAL by default, busy timeouts, honest error text. A sibling package that imports nothing else here. Measured guidance in docs/SQLITE_PERFORMANCE.md.
C ABI (libm0core)
poe build-ffiemitslibm0core.so/.dylib(FNV-1a, xxHash32, wyhash64, JSON escape) for Bundlopen, N-API, orctypes; release artifacts for Linux and macOS are attached to GitHub releases.
Examples (apps/)
hello— the whole server in one file.notes_api— the framework showcase: negotiation, ETags, problem+json, CORS, validation.datastar_counter— multi-tab live sync; the reference wiring for cross-worker fan-out and shared-memory state.datastar_todo— the flagship: HTML-over-SSE broadcasts, SQLite persistence, and SSE replay across restarts.django_wsgi— a real Django project served by the WSGI host.