Stressed

Share

I had put together a stress test of Clojurust, which just runs all the tests currently in the repo, both under AOT compilation and the direct interpreter/JIT path. Some recent interpreter GC-rooting fixes landed, and the stress test just had its first successful run. It's a good milestone: it's doing some non-trivial compute tasks with decent efficiency and complete correctness.

Next up is optional type tagging of primitive types, which can optimize primitive arithmetic (with some caveats), and which can give 2-3x speed improvement with IR interpreter/JIT, at almost 10x speed improvement with AOT compilation.

Probably next up I want to do is implement isolate boundary crossing: the core.async implementation uses N separate GC heaps, each isolated to the N runner threads the async runtime uses. The idea is that async workloads pin to a thread with an independent GC heap, which is necessary because values are !Send, meaning they can't be sent safely between threads. The solution here is to allow sending values across these boundaries via channels, already something used heavily in async Clojure, where the value is copied across the thread boundary at the value level: it's fully, deeply copied and serialized/deserialized across the thread boundary. Naturally, some values cannot be safely copied — transients and arrays, since the boxed internal value is inherently mutable. The aim here is to make Clojurust's async runtime closer in spirit to Erlang, with the sharing seam explicitly exposed to the developer: it’s a choice to send a value across a thread boundary, not something that can be done by accident. Plus the copying cost is measured and inspectable.

Another future item is proper IR lowering of asynchronous code into a Rust-like state machine: right now all async functions are run via the interpreter, even in JIT or AOT contexts. Moving this code generation to a full native async pattern makes async patterns much more honestly compiled than they currently are.

I may take a detour and try writing an interesting sample program that makes heavy use of the async runtime and the networking layer.