Unified low-level IR (LIR)
Design notes and migration log for the arch-neutral, machine-level IR that sits
between AsmIR and the per-arch monoasm! / monoasm_arm64! byte emission.
This is Phase-1 item ① (“a low-level IR that describes amd64 and aarch64
uniformly”).
Status: the great majority of AsmInst families now lower through
encode_linst. The model lives in
monoruby/src/codegen/jitgen/lir.rs;
the per-arch encoder is Codegen::encode_linst, defined once in each
arch/<arch>/compile/… file. As of this writing the integer + control-flow
core, all addressing modes, the GC write barrier, the common guard/check family,
fixnum arithmetic (with overflow deopt), the whole floating-point family,
the bounds-checked heap-ivar load/store, and the large runtime-call macro-op
families (array/hash/string/range construction, defined?, generic binops,
class/method definition, exceptions, yield, the method-prologue guards, …) are
all lowered through encode_linst, as is the entire method-call family —
argument setup and the call itself (the dispatcher pre-resolves the
store/frame-dependent values and the encoder stays store-free) — and the cold
deopt side-exit handler blocks. Even the AsmInst::Inline builtin escape
hatch now lowers to a carrier LInst::Inline (dispatched via the
context-carrying encode_linst_inline), so every AsmInst reaching the
dispatcher lowers to LInst. What remains outside encode_linst is the
specialized inlined-frame family and the zero-byte patch/recompile bookkeeping
that is not byte emission — see §8.
1. Motivation
The original justification (“close the aarch64 bail gap”) is gone: the
full aarch64 port (#704) made aarch64 lower every AsmInst (see
doc/arch_difference.md). The remaining — and real — motivation is
description unification:
- The AsmIR → machine-code step used to be two parallel sets of
emit_*primitives (x86arch/x86_64/compile/*.rs, aarch64arch/aarch64/compile.rs). LIR makesencode_linstthe single seam through which all migrated families emit code. - LIR is also the concrete code-generation target the future interpreter/JIT DSL (Phase-1 item ③, “derive interpreter + JIT from one description”) lowers to. A hand-written, test-validated LIR gives ③ a well-defined target and a correctness oracle.
2. Where LIR sits
TraceIR
→ AsmIR (AsmInst, arch-neutral, register-allocated)
→ [compile_asmir dispatcher] jitgen/asmir/compile_shared.rs
→ LIR (LInst, arch-neutral machine ops) jitgen/lir.rs ← this layer
→ [per-arch Codegen::encode_linst] arch/<arch>/compile/…
→ monoasm! / monoasm_arm64!
→ bytes
AsmIR stays the register-allocated, arch-neutral front-end output (one
AsmInst per semantic operation). The arch-neutral dispatcher
compile_asmir (compile_shared.rs)
lowers each migrated AsmInst into one or more LInsts and feeds them to
encode_linst; not-yet-migrated families still call their per-arch emit_*
directly from the dispatcher.
encode_linst is an inherent Codegen method, defined once per arch in the
file that the compile module includes for the active target_arch. Because
only one compiles per target, no trait/dynamic dispatch is needed — cfg
selects the right encoder.
3. Data model
Defined in lir.rs. Branch and side-exit targets carry a resolved monoasm
DestLabel (the encoder runs after label resolution), so LInst is
#[derive(Debug, Clone)] — DestLabel is Clone but not Copy. PartialEq
is intentionally not derived: several payload types reached by the macro-op
variants (WriteBack, AsmEvict, FnInitInfo, …) are not PartialEq, and
deriving it would force a PartialEq cascade across unrelated types for no
benefit. The lir.rs unit test uses matches! instead of ==.
Operands
| Type | Role |
|---|---|
GP | General-purpose register. Reused from codegen.rs; already arch-neutral. |
LReg | Gp(GP) or Scratch — a register that may be the per-arch reserved scratch pointer (rdx on x86, x9 on aarch64). For intermediate pointers (heap var-table derefs) that must not clobber an allocated value; aarch64’s x9 is outside GP’s allocatable map, so it needs its own kind. From<GP>. |
FPReg | Virtual FP register (physical xmm/d-reg or a stack spill); the encoder resolves the spill via FPReg::loc(base), so FP LInsts carry the frame’s spill base. |
LOperand | Reg(GP) or Imm(i64) — an ALU/compare source the encoder folds or materializes. |
LMem | A logical memory location with unbounded displacement: Slot(SlotId) (LFP-relative, negative), Field { base: LReg, disp } (object field / scratch-relative, positive), RspRel { disp } (callee-frame arg slot). The encoder legalizes the displacement per arch. |
LCond | Signed integer branch condition (Eq/Ne/Lt/Le/Gt/Ge), with from_int_cmp / invert. |
LAluOp | Add/Sub/Mul/And/Or/Xor/Shl/Sar, with from_binop. |
Instructions (LInst)
| Group | Variants |
|---|---|
| Move / immediate | Mov, LoadImm |
| Memory | Load, Store, StoreImm (over LMem::Slot / Field / RspRel) |
| ALU / compare | Alu, Cmp |
| Branch | Label, Br, CondBr { cond: LCond, … }, BranchTruthy { negate }, BranchIfNil, BranchIfNonzero |
| GC / nil | WriteBarrier { parent, value }, NilIfZero { reg } |
Guards (carry a side-exit deopt) | GuardClass, GuardArrayTy, GuardFrozen, GuardConstBaseClass, GuardConstVersion, GuardCapture, CheckBOP |
| Integer arithmetic | IntegerBinOp { …, deopt }, IntegerCmp, FixnumNeg { …, deopt }, FixnumBitNot |
| Floating-point | FprMove, F64ToFpr, FixnumToFpr, FprToStack, FprSwap, FloatToFpr (deopt), I64ToBoth, FloatBinOp, FloatUnOp, FloatCmp, FloatCmpBr, FprSave, FprRestore, CFunc_F_F, CFunc_FF_F (FP ops carry the spill base) |
| Heap ivar (macro-op) | LoadIVarHeap, StoreIVarHeap |
| Construction (macro-op) | CreateArray, NewArray, NewHash, HashInsert, ArrayConcat, NewRange, ConcatStr, ConcatRegexp, ToA, DeepCopyLit, ExpandArray |
| Variables (macro-op) | StoreConstant, LoadGVar, StoreGVar, LoadCVar, CheckCVar, StoreCVar, LoadDynVar, StoreDynVar, AliasGvar |
defined? (macro-op) | DefinedYield, DefinedSuper, DefinedGvar, DefinedCvar, DefinedConst, DefinedMethod, DefinedIvar |
| Dispatch helpers (macro-op) | GenericBinOp, OptEqCmp, ArrayTEq, CheckKwRest, RestKw |
| Definition (macro-op) | MethodDef, SingletonMethodDef, ClassDef, SingletonClassDef, AliasMethod, UndefMethod |
| Method-prologue guards (macro-op) | GuardClassVersion, RecompileDeopt, CheckStack, ExecGc, Init, LoopJitRspBump |
| Control flow / exceptions (macro-op) | Ret, MethodRet, BlockBreak, Raise, Retry, Redo, EnsureEnd, Yield, BlockArgProxy, BlockArg, ImmediateEvict, Deopt, HandleError, Unreachable |
Lir is a thin Vec<LInst> builder used where a multi-instruction sequence is
convenient.
The macro-op variants do not lower to primitives inside encode_linst; instead
each backend’s encode_linst ends with other => self.encode_linst_macro(other),
and the arch-neutral encode_linst_macro
(compile_shared.rs)
delegates each macro-op to the existing per-arch emit_* helper. This keeps the
delegation in one place (rather than duplicating it in both backends) while
still funnelling all emission through encode_linst. See §5.
The legalization contract (the core idea)
LMem displacements and LOperand::Imm values are unbounded; the per-arch
encoder legalizes them:
- x86-64:
[reg + disp32]/imm32cover essentially everything — mostly a no-op. - aarch64: fixed-width encodings allow only small immediates, so the encoder
folds a fitting displacement into the instruction (
ldur/stur±256, scaledldr/str,sub sp) and otherwise materializes the offset into reserved scratchx9/x10(thea64_frame_*/a64_field_*/a64_rsp_slot_addrhelpers).
4. The two model extensions (Stage 3)
Stages 2-A…2-J were byte-for-byte family ports that fit the data model as-is. Stage 3 extended the model so the harder families could be expressed:
-
Scratch register operand (
LReg::Scratch). Lets the LIR express intermediate pointers that map to a different physical register per arch (x86rdx, aarch64x9). Without it, an abstractGPwould either pick a register that aarch64 keeps allocatable (clobber hazard) or have no name for aarch64’s reserved scratch. First user: the self heap-ivar store (Load{Scratch ← [rdi+VAR]} ; Load{Scratch ← [Scratch+MONOVEC_PTR]} ; Store{[Scratch+idx*8] ← src} ; WriteBarrier). -
Deopt side-exits. Guard ops carry the resolved side-exit
DestLabelthey fall through to, making deopt a first-class LIR concept. The arch-neutral dispatcher resolveslabels[deopt]and builds the guard; the encoder branches to it. This pattern carries the overflow exit ofIntegerBinOpand every guard/check op.
5. Encoding styles: decomposition vs. macro-op
Migrated families use one of two encoder styles, both byte-identical to the prior code:
- Decomposition — the LIR sequence is built from reusable primitives and the
per-arch lowering lives in
encode_linst. Used by the move/memory/ALU/branch core, the field load/store + write-barrier, and the FP transfer/convert ops (whose spill-aware bodies were moved fromemit_*into the encoder arms, deleting thoseemit_*). - Macro-op delegation — a single
LInstwhose encoder calls an existing per-arch helper that stays (the tag-test / page-split / tagged-arith / C-call sequence is irreducibly per-arch). Two sub-cases:- decomposed-style delegation (guards,
IntegerBinOp): the per-archencode_linstmatches the variant directly and calls the substantive helper (guard_class/a64_guard_class,integer_binop/a64_integer_binop, …); the thinemit_*wrapper is deleted. encode_linst_macro-style delegation (the large runtime-call families): the variant falls through each backend’sother =>arm into the arch-neutralencode_linst_macro, which calls the per-archemit_*helper. Theemit_*helper is retained verbatim, so the migration is a pure routing change — the dispatcher arm now builds anLInstand hands it toencode_linstinstead of callingemit_*directly. This is how the construction / variable /defined?/ definition / control-flow families were migrated (batches A and B).
- decomposed-style delegation (guards,
6. Migration log
Each stage routes one family through encode_linst, commits, and is verified by
the full cargo test --lib suite (see §7). All stages are byte-identical
(modulo eliding a redundant self-mov when a value is already in the
accumulator).
| Stage | Family | Notes |
|---|---|---|
| 0 / 1 | LIR data model + this doc | scaffolding |
| 2-A | Mov (emit_reg_move) | first encode_linst user |
| 2-B | slot memory (Load/Store/LoadImm/StoreImm over Slot) | frame legalization |
| 2-C | reg-imm ALU (emit_reg_add/sub → Alu) | LAluOp / LOperand |
| 2-D | integer compare-branch (Cmp + CondBr) | LCond; built in shared dispatcher |
| 2-E | conditional branches (BranchTruthy / BranchIfNil / BranchIfNonzero) | |
| 2-F | inline struct-slot load (Load over LMem::Field) | first field-offset legalization |
| 2-G | inline ivar/struct stores (Store{Field} + WriteBarrier) | introduces WriteBarrier |
| 2-H | inline ivar load (Load{Field} + NilIfZero) | introduces NilIfZero |
| 2-I | heap struct-slot load/store | composed from existing ops — no new op |
| 2-J | rsp-relative arg stores (LMem::RspRel) | completes the addressing modes |
| 3-A | scratch operand + self heap-ivar store | model extension ① |
| 3-B | deopt model + class / array-ty / frozen guards | model extension ② |
| 3-C | const-base-class / const-version / capture / BOP guards | |
| 3-D | fixnum IntegerBinOp (overflow deopt) | hottest arithmetic path |
| 3-E | FP transfer/convert (FprMove / F64ToFpr / FixnumToFpr / FprToStack) | first FP family; real decomposition |
| 3-F | FP swap / FloatToFpr (deopt) / I64ToBoth | |
| 3-G | FP arithmetic & compare (FloatBinOp / FloatUnOp / FloatCmp / FloatCmpBr) | NaN-correct conditions |
| 3-H | FP C-calls (CFunc_F_F / CFunc_FF_F) + FprSave / FprRestore | completes the FP family |
| A | bounds-checked heap-ivar load/store (LoadIVarHeap / StoreIVarHeap) | first macro-op via encode_linst_macro |
| B1 | variable + construction macro-ops (g/c/dyn-var, array/hash/range/str, to_a, defined?, generic binop, alias/undef, …) | bulk macro-op routing |
| B2 | remaining construction / defined? / dispatch-helper macro-ops | |
| B3 | control-flow / exception / definition macro-ops (Ret, MethodRet, Raise, Retry, Redo, EnsureEnd, Yield, MethodDef, Init, CheckStack, ExecGc, IntegerCmp, BlockArg, …) | Raise/Retry/Redo/EnsureEnd carry loop_jit_spill_bytes for the aarch64 loop-JIT sp unwind |
| B4 | class-def + method-prologue guards (ClassDef, SingletonClassDef, GuardClassVersion, RecompileDeopt) | last non-store/non-frame arms |
| B5 | elementary moves (RegMove/RegToAcc/AccToStack/RegToStack/StackToReg/LitToReg/LitToStack/RegAdd/RegSub) | dispatcher lowers straight to LInst; the thin emit_* move wrappers deleted from both backends |
| B6 | method-call / argument-setup (SetupMethodFrame, SetArguments, SetArgumentsForwardedHelper, Preparation, OptCase) | dispatcher pre-resolves the store/frame-dependent values (offset, block info, heap-ivar length, jump-table labels) into the LInst; per-arch helpers made store-free |
| B7 | the call itself (Call) | dispatcher pre-resolves codeptr / is_iseq / callee+call-site pcs / x86 JIT entry (get_jit_entry); do_call/a64_do_call made store-free. x86 still records the return-address patch point inside the call’s encoder (it is captured at the emission point); aarch64 ignores the x86-only fields |
| B8 | cold side-exit / deopt handlers (LInst::SideExit { kind }) | both gen_asm side-exit loops build an LInst::SideExit and route through encode_linst; the per-arch encoder dispatches on LSideExitKind (Deopt / Evict / RecompileDeopt / Error) to the existing handler emitters (x86 gen_*_with_label / gen_handle_error; aarch64 a64_gen_deopt / a64_gen_handle_error, which collapse Evict/RecompileDeopt to a plain deopt) |
7. Verification
Because the migrations are pure refactors, the safety net is behaviour preservation under a correct reference Ruby:
- The harness compares JIT output against the system
ruby, and monoruby targets CRuby 4.0+. With an older Ruby (e.g. 3.3) ~33--libtests fail on format differences (3.4Hash#inspect, etc.) — unrelated to LIR. Every stage here is verified under CRuby 4.0.5, where the baseline is 1702 passed / 0 failed; each migration keeps it at 1702 / 0 with no new warnings. - Each family migration is local to its
emit_*/ dispatcher arm and is independently revertible.
Building CRuby 4.x from source in a restricted-network sandbox: the official tarball host (
cache.ruby-lang.org) may be blocked while GitHub is reachable. Fetch the git tag source archive fromcodeload.github.com, installgperf(needed to generatelex.c), emptygems/bundled_gemsto skip the bundled-gem download (or setSSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crtso the bundled-gem fetch trusts the egress proxy CA), thenautogen → configure → make → make install.
8. Remaining work
First, what is no longer a blocker. The whole method-call family is now
migrated — argument setup (SetupMethodFrame, SetArguments,
SetArgumentsForwardedHelper, Preparation, OptCase; B6) and the call
itself (Call; B7) lower through encode_linst. The dispatcher — which holds
&Store / &mut AsmInfo — resolves every store/frame-dependent value (callee
scratch offset, block (fid, arg), heap-ivar table length, jump-table
DestLabels, and for Call the codeptr / is_iseq / callee & call-site pcs /
x86 JIT entry) and carries them in the LInst, so the per-arch helpers
(do_call/a64_do_call, …) are store-free. Call’s encoder also records the
return-address patch point, which is captured at the emission point and so
belongs with the call’s emission. (The elementary register/stack moves were
likewise inlined into the dispatcher in B5, deleting their forwarding wrappers.)
The cold side-exit / deopt handler blocks that guards branch to — laid
out by gen_asm before the main instruction loop, not by an AsmInst — are
also byte emission, and as of B8 they route through encode_linst via
LInst::SideExit (write-back + sp-unwind + VM-resume / unwind). Both arches’
gen_asm side-exit loops now build an LInst::SideExit and the per-arch
encoder dispatches on LSideExitKind.
The remaining things handled directly (not through encode_linst):
-
Specialized inlined-frame family.
MethodRetSpecialized,BlockBreakSpecialized,SpecializedCall,SetupYieldFrame, andSpecializedYieldlower an inlined callee/block frame; they resolve frame-local labels and patch points and are dispatched to a per-arch method of the same name (arch/<arch>/compile/…). (AsmInst::Inlineis no longer in this group — it lowers toLInst::Inline; see below.) -
The
AsmInst::Inlineescape hatch. Most builtin inline generators (e.g.emit_math_sqrt) still run a closure that emits arch asm directly viagen. As of the AsmIR→LIR consolidation,AsmInst::Inlinedoes lower to a carrier LIR op —LInst::Inline(InlineProcedure)— so everyAsmInstreaching the dispatcher now lowers toLInst.LInst::Inlineis the one LIR op whose emit is not store-free: its wrapped closure needs the compile context (&Store,&SideExitLabels, framebase), so it is dispatched at the lowering boundary viaencode_linst_inlinerather than through the store-freeencode_linst. (If it ever reachedencode_linst/_macroit hits theunreachable!fallthrough.) This is a transitional carrier: migrating the closures onto typed, arch-neutral LIR ops — so the variant can eventually go away — is goal 2 of the long-term plan (“express the inline-builtin codegen once, arch-neutrally”). The migratable category is the pure generators — property/field readers and trivial C-function wrappers, whose codegen is one existing LIR op with no arch-specific control flow:- 64-bit field readers →
AsmIr::load_field_to_reg→AsmInst::LoadFieldToReg→ existingLInst::Load { Field }(no new op, byte-identical on both arches). Done:Range#begin/end(hand-writtenemit_range_begin/enddeleted) andArithmeticSequence#begin/#end/#step(via the sharedinline_field_loadhelper;emit_load_value_fielddeleted from both backends). - Bool field readers →
AsmIr::bool_field_to_reg→AsmInst::BoolFieldToReg→LInst::BoolFieldToReg(a small macro-op: 32-bit load +shl 3+or FALSE_VALUE, deduping the two byte-identicalemit_*_exclude_endemitters into one encode arm per arch). Done:Range#exclude_end?,ArithmeticSequence#exclude_end?. - Container length →
AsmIr::array_len_fixnum/string_len_fixnum→AsmInst::ArrayLenFixnum/StringLenFixnum→ the matchingLInst(a macro-op: load inline capa + conditional-select the heap length when capa exceeds the inline cap + fixnum-tag; the conditional select is the only per-arch part, x86cmov/ aarch64csel). Done:Array#size,String#bytesize— one op each (differing only in the inline-cap constant,ARRAY_INLINE_CAPAvsSTRING_INLINE_CAP), replacing the fouremit_array_size/emit_string_bytesizeemitters. - Fixnum → float (
Integer#to_f) → the existingAsmIr::fixnum2fpr→LInst::FixnumToFprop (untag +cvtsi2sd/scvtfstraight into the result fpr). No new primitive; deletesemit_int_to_floatfrom both backends (and drops a redundantxmm0round-trip the old emitter always did). - Bool predicates (
Object#nil?,BasicObject#!) →AsmIr::is_nil_to_bool/not_to_bool→LInst::IsNilToBool/NotToBool(a macro-op: compare + conditional-select TRUE/FALSE; the select is the per-arch part,cmov/csel). Replaces theemit_kernel_nil/emit_object_notemitters. - C-function wrappers (e.g.
Math.sin/cos/atan2,Float#**) are already arch-neutral: they route through the typedAsmInst::CFunc_F_F/CFunc_FF_F(→ existingLInst::CFunc_*), not the closure escape hatch. - FP guard + op (
Math.sqrt) →AsmIr::math_sqrt→LInst::MathSqrt, a macro-op carrying the deopt label (resolved by the dispatcher likeGuardClass). It encapsulates the per-arch domain guard (x86ucomisd+jp/jb, aarch64fcmp+b.vs/b.mi) and thesqrtsd/fsqrtin one encode arm per arch — so even a deopt-branching FP builtin migrates without a closure (it does not declarefpr_operands, matching the prior opaqueInline, so the spill-area sizing is unchanged; the fprs are accounted via the surroundingload_fpr/def_F). - Integer guard + op (
Integer#succ) →AsmIr::integer_succ→LInst::IntegerSucc, a macro-op carrying the deopt label (add+jo/adds+b.vs, deopt → Bignum promotion). The integer analog ofMathSqrt. - Control-flow predicate (
Kernel#block_given?) →AsmIr::block_given→LInst::BlockGiven, a macro-op with a self-contained local exit label (reads[LFP - LFP_BLOCK]). No external context needed.
What stays a closure is the genuinely complex shapes — object allocation,
send,fiddle, the remaining integer shift/division guards,object_id(a runtime call). Several would still benefit from a couple of generic FP/branch/call LIR primitives. - 64-bit field readers →
-
Pure patch / recompile bookkeeping — the x86/aarch64 non-coverage asymmetry of
doc/arch_difference.md§4. The deopt handler emission now goes through LIR (above); what stays out is the part that emits no bytes:ImmediateEvictrecords areturn_addr_tablepatch point (zero bytes), and the actual return-address overwrite happens at BOP-redefinition time, not at compile time. The clean invariant is therefore:
encode_linstis the single seam for byte emission; zero-byte code-position metadata / runtime patch bookkeeping is not emission and stays in the dispatcher /Codegen.
None of this invalidates the model: encode_linst (with encode_linst_macro,
plus the context-carrying encode_linst_inline for the one LInst::Inline
op) is the single seam for byte emission. The only arms still handled directly
in the dispatcher are the specialized inlined-frame family (which resolve
frame-local labels / patch points) and the patch/recompile bookkeeping, which is
not emission by the invariant above.
§9 The two-phase pipeline: whole-function buffering + virtual GP + physical allocation
Everything above is the single-pass seam: compile_asmir lowers each
AsmInst to one or more LInsts and emits them immediately through
encode_linst. That gave us one byte-emission description per op, but it cannot
host a register allocator: allocation needs the whole function’s instruction
stream in hand to compute liveness, so an op cannot be encoded the instant it is
produced.
The next architectural step (the AsmIr → LIR pipeline) splits emission into two
phases while keeping AsmIr and LIR as separate layers (collapsing them
would just reconstruct AsmIr):
AsmIr (AsmInst, frame-INDEPENDENT, FP already virtual, GP physical)
│ lower (resolve frame size / labels / patch points)
▼
LIR = Vec<LInst> ← whole compiled unit, offsets baked in, GP *virtual*
│ arch-dependent physical-register allocation pass (GP virtual → physical)
▼
encode (drain → encode_linst → bytes)
- LIR is barely-arch-independent and offset-baked. Frame base, slot
displacements, field offsets, and labels are all resolved when
AsmIrlowers toLIR(AsmInstis frame-independent;LInstalready carriesbase). WhatLIRdoes not yet bake is the physical register assignment. - Registers are virtual in LIR. FP registers already are (
FPReg= phys-or-spill, resolved byPhysMapat encode time — §25/§27). GP registers are not: todayLInstnames concreteGPs (R15= accumulator,R12= globals,R14= LFP, plus scratch temporaries). The pipeline introduces a virtual GP (VReg) for the allocatable GP uses; the fixed globals (acc / pc / lfp / globals / executor — seeCLAUDE.md) stay pinned and are not virtualized. Only the temporary/scratch GP traffic becomes virtual and is assigned in the post-LIR phase.
Blockers (from the current driver, asmir.rs::compile ~2293–2400)
The single-pass driver interleaves byte emission with ordering operations that
are not themselves LInsts emitted through encode_linst:
self.jit.label()/bind_label(..)— label creation and binding.self.jit.select_page(1)— cold/hot page selection (x86 lays side-exit handlers on the cold page).frame.sourcemap.push((i, pos))— source-position records keyed on the current code position.- Patch-point / return-address-table bookkeeping (
doc/arch_difference.md§4), which emits zero bytes but is position-sensitive.
If LIR buffered only the encode_linst calls and left (1)–(4) inline, replaying
the buffer later would desync every label, page boundary, and source mapping.
Therefore the buffer must model these as ordering pseudo-ops in the same
Vec<LInst> (e.g. LInst::BindLabel, LInst::SelectPage, LInst::SourcePos),
so the entire emission stream — bytes and position metadata — is one
ordered sequence the allocator can walk and the drainer can replay faithfully.
Staged increments
- 9a — Make the emission stream fully reified. Add the ordering pseudo-ops so
compile_asmir+ the side-exit loop produce a singleVec<LInst>covering all of byte emission, label binding, page selection, and sourcemap. Verify byte-identical output by draining the buffer immediately (no allocation pass yet) — a pure refactor, gated/shadowed like the §24 placement harness. - 9b — Introduce
VReg(virtual GP) with an identity map. Replace the allocatable GP operands withVReg, lowered through a trivial virtual→physical map that reproduces today’s assignment. Still byte-identical; this only changes the type carried, establishing the seam. - 9c — Carry liveness / loop metadata into LIR. Per §27.1, re-deriving liveness after LIR-flatten cost 2.5×, so the allocator must consume the metadata the ① fixpoint already computed (loop-carried sets, etc.) rather than recompute it. Thread it onto the buffer.
- 9d — The arch-dependent physical-allocation pass. Walk the buffered LIR,
assign physical GPs to
VRegs (spilling to frame slots under pressure, exactly asFPRegdoes for FP), then drain → encode. This is the first point the output may legitimately differ from today’s bytes; likephys-loop-aware(§42), it is a perf experiment gated behind a flag + the M1 A/B bench gate, and the shadow digest becomes a delta meter, not an equality check.
9d allocatable GP pool (design decision)
The VM’s fixed registers stay pinned (acc=R15, lfp=R14, pc=R13, globals=R12, executor=rbx) and the C-ABI / inline-builtin convention regs (rdi=recv, rax=result, rsi/rdx/rcx=call args) are pre-colored by the operations that use them — the allocator cannot move those. The allocatable pool is the otherwise-unused caller-saved scratch set:
- x86-64:
r8, r9, r10, r11(4 registers). These are caller-saved, so any of them that is live across a C-ABI call (e.g. a runtime helper /CFunc_*) must be spilled to the frame and reloaded — the allocator inserts the save/restore exactly as the FP side already does around calls (fpr_save/fpr_restore). - aarch64: the analogous caller-saved temps left free in the
GP::a64map (x9..x15minus the encode scratch x9/x10), with the same spill-across-call rule.
So a VReg is either pinned (an ABI/convention register the front-end chose)
or allocatable (assigned by 9d to a pool register or a spill slot). The
identity map of 9b is the degenerate all-pinned case; 9d makes the front-end emit
allocatable VRegs for cross-operation temporaries and colours them into the
pool. The payoff is keeping hot bytecode-slot values in r8–r11 instead of
re-loading them from the LFP each use.
9a/9b are byte-identical refactors (safe to land once verified); 9c/9d are the substantive allocator work and stay behind a flag until the bench gate clears.
9d placement build order
The placement phase (filling gp_alloc, colouring VReg::Alloc into pool
registers, and keeping the frame consistent) is built as a sequence of
feature-gated (gp-alloc) increments, each byte-identical while no slot is yet
placed in the pool:
-
Write-back + flush pool-support (done). Two seams, both inert until a slot is placed:
- Side-exit / deopt / GC write-back.
WriteBackcarries agp: Vec<(GP, SlotId)>list — the pool-resident slots and their physical registers — alongside the singler15accumulator. Every such write-back (gen_write_back,gen_write_back_for_deopt,a64_gen_write_back_for_deopt) stores each pool register to its slot’s frame home, exactly as forr15. The producer (wb_gp) scans for slots in modeG(_, VReg::Alloc(_)). - In-function flush-at-boundary.
writeback_acc— already called before every call / store / definition, and already asserting noG(_, _)slot survives it — now also flushes the pool residents (writeback_pool_state): eachG(_, Alloc)slot is stored to its home and dropped toS. Because this runs before every C-ABI call, a pool value never stays resident across a call, so the GP allocator needs no caller-saved spill/reload threading (using_gp) the way the FP side does — the flush already did it.
Until the placement policy creates a
G(_, Alloc(_))slot both lists are empty, so the fields, theirHash/Debug, and the write-back/flush loops are inert and the emitted bytes are unchanged. - Side-exit / deopt / GC write-back.
-
Reserve the pool registers (x86-64) (done). The pool design above assumed
r8–r11were “otherwise-unused caller-saved scratch”, but an audit of the JIT-body lowerings found that is not true:r8–r11are the JIT’s secondary scratch pool, used freely once the primary scratch (rax/rcx/rdx/rsi/rdi) runs out. To hold a value in a pool register across instructions, no intervening lowering may clobber it — so the pool registers must be reserved, exactly as the FP side reserves its xmm pool and confines scratch toxmm0/xmm1.The audit (in-scope: every
compile.rs/compile/*/guard.rslowering that can run while a pool value is live; out of scope:vmgen= the VM interpreter,invoker/wrapper= reached via a call boundary) found:r10,r11: unused in JIT-body code.r9: onlyclass_def, a post-flush call-staging use (safe).r8: every use is post-flush call-staging (a method-send/define/store sequence runswriteback_accfirst, emptying the pool — sor8is free there, the GP analogue of using the pool afterfpr_save) except one:emit_string_setbyte, a pure inline op that can execute with a pool value live. That one was migrated offr8(tag scratch →rcx; negative-index adjust → a sign branch instead of a cmov-through-scratch).
The reservation invariant: a JIT-body lowering may use
r8–r11as scratch only in a post-flush call-staging window (afterwriteback_acc, where the pool is provably empty). Everywhere a pool value can be live, the pool registers are off-limits. After the migration this holds for the whole x86-64 backend. (aarch64 pool reservation is pending:GP::R8–R11map tox5–x8; that backend needs the same audit before placement targets it. Thegp-allocfeature is x86-first and off by default, so nothing places into the aarch64 pool yet.) -
Placement policy + multi-residency (done).
def_reg2acc_guardedis the trigger: when a new accumulator value arrives from a register other than R15 (so R15 still holds the previous accumulator),try_relocate_acc_to_poolmoves that previous value into a free pool register (G(_, Alloc(id))) instead of spilling it. Later reads then come from the register (no LFP reload) until the next flush. Making the state machine track pool residents (not just the single R15 accumulator) required:Placement::Gp(VReg)(load-bearing).SlotStatestoresplace+tyand reconstructs everymode()viafrom_parts— so the placement must carry theVReg, elseset_mode(G(Alloc))loses the pool-register identity on the very next state read (it round-trips toStack).- Register-aware reads everywhere a
Gslot is consumed:on_reg/on_reg_or(binop operands),load_state(GpLoad::Reg(vreg.phys())),fetch_for_callee(call-argument materialization), theG→Sfbridge — all resolvevreg.phys()rather than assuming R15. - Alloc-aware destructive sites + flush (§1, the
gpfield /writeback_ pool_state/ Alloc-awareclear/write_back_slot). - Fixnum-only restriction (GC safety): a pool register is not a GC
root, so only values statically guarded
Fixnum(immediates — no heap pointer) are relocated. A heap pointer kept solely inr8–r11across a collection would be freed; heap-value pooling is deferred until pool registers are made GC-rootable. Hot integer loops — the primary target — are covered. Verified byte-identical on default; gp-alloc passes the full suite includinggc-stress(GC on every allocation).
-
M1 A/B bench gate before the feature becomes default.
Notes:
- C-ABI call-save is not a separate step: the flush-at-boundary (§1) makes it
unnecessary — pool slots are always written back before a call. This holds for
every call, not just Ruby method calls: the method-call path flushes via
writeback_acc, and every runtime helper (deep_copy_lit,new_array,new_hash,to_a,concat_str,generic_binop, class/method def,defined?, …) flushes the pool at its pre-callget_using_fprsnapshot (which now flushes the GP pool as a side effect — the universal pre-C-call chokepoint). The pool registers are caller-saved, so a Fixnum left resident there would otherwise be clobbered by the helper’s C call (e.g.String#clear’sbytesizeinr8, clobbered by the""literal’svalue_deep_copy). The asmir builders that take&AbstractFrame(immutable, so they cannot flush) get an explicitflush_gpin their handler instead. - Branch-merge reconciliation needs no special pool handling: pool residents are
flushed (→
S) before any branch (the compile-loopflush_gpand the back-edge analysis G→S demotion), so a merge never sees aG(_, Alloc)slot.
9d outcome: GP register residency does not pay; the untagged-Fixnum direction does
The temp-relocate placement (§9-3) is correct (full suite incl. gc-stress
passes) but the M1 A/B bench gate fails: no benchmark improves beyond noise
and binarytrees regresses ~14% (the relocate mov + flush is pure overhead
when the pooled value is not reused enough). So gp-alloc stays off.
9d-B: accumulator register file (results born directly in r8–r11)
Instead of the relocate model (new value → R15, then relocate the old R15
resident into the pool), the accumulator register file extends the
accumulator’s register set from {R15} to {R15, r8–r11}: a Fixnum result is
written directly into a free pool register (try_def_G_pool in
def_reg2acc_guarded), with no R15 round-trip and no relocate mov. R15
remains the fallback when the pool is full or the value is not a Fixnum
immediate. This removes the relocate overhead that made the temp-relocate
version regress binarytrees.
Making it correct required eliminating two latent “the current value is in R15” assumptions that the single-accumulator design relied on:
copy_slot’sGarm spilledsrcthendef_G(dst)— which claims R15 without moving the value there. Sound only forPinned(R15); for a pool resident it leftdstreading R15 while the value sat in the pool register. Fixed by transferring the pool register’s ownership todst(no data move).- Caller-saved clobbering across runtime helpers (see the §1 note above):
the pool registers are caller-saved, and the flush-at-boundary invariant was
only honoured by Ruby method calls, not the runtime helpers. Fixed by folding
the GP-pool flush into
get_using_fpr, the universal pre-C-call snapshot.
Result: full lib suite green under --features gp-alloc (1678/1678, reduced
parallelism — the harness flakes on ruby-subprocess spawn under high
parallelism, unrelated to the JIT). A/B (release, best-of-5): binarytrees
0.997× (the −14% regression is gone), app_fib 0.959×, tarai 1.007×,
so_nbody 0.999× — i.e. overhead-free but not a speedup. So the
register-file fixes the cost of the relocate model without changing the
fundamental conclusion below: residency alone is marginal for Fixnums. gp-alloc
stays off pending the untagged-Fixnum representation, which supplies the
missing per-op cost. The work lives on claude/gp-acc-regfile.
Why register residency alone is marginal for integers. Unlike floats —
where the win is avoiding boxing (a heap allocation per spilled float) — a
Fixnum is an immediate, so keeping it in a GP register only saves an LFP
load/store, and that store is already cheap (L1 + store-to-load forwarding).
Locals must also be materialized in the frame at every deopt / call / GC
safepoint anyway. Microbench (hand-asm, faithful to the JIT loop body): keeping
the loop locals of while i<n; s=s+i; i=i+1; end in registers instead of memory
is only ~1.26× on the tightest possible integer loop, diluted to ~noise on
real benchmarks.
Loop-carried integer residency (the float-F-mode analogue: promote
loop-carried Fixnum locals to pool registers at the loop entry, keep them across
the back-edge) was prototyped on branch claude/gp-loop-carried-wip. It
promotes, but the JIT’d loop deopts and recompiles ~every iteration (the VM→JIT
loop entry / OSR does not set up the pool registers the JIT’d loop head expects).
Given the marginal ceiling above, it was not pursued to completion.
The promising direction — untagged Fixnum (“integer F-mode”). A Fixnum is
tagged 2n+1, so every integer op pays a tag adjustment (sub 1 / or 1)
and a type guard (test $1; je). Give integers a second representation —
untagged 2n (LSB cleared), held in a GP register — exactly as a float has
boxed Value vs unboxed F (xmm). Within a run of integer ops the value stays
untagged; it is tagged (or 1) only when it escapes to a Value context
(stored to a boxed slot, passed to a call, …). This removes both per-op costs,
because an untagged-integer slot is statically known to be an integer (no guard,
like F needs no float guard).
Key properties:
- Overflow detection is preserved by the
2nchoice:2a + 2b = 2(a+b)overflows i64 iffa+boverflows i63 (the Fixnum range), so the existingjoworks unchanged. (Rawnwould not preserve this.) - Per-op, not per-loop: the tag/guard elimination helps every integer op, so it benefits straight-line integer code too, not just loops.
- Microbench (hand-asm): untagged + guard-free is ~1.50× over the tagged+guarded in-register loop; compounded with residency, ~1.9× ceiling on the tightest integer loop.
Implementation shape (a substantial change, comparable to introducing F/Sf):
a new LinkMode for an untagged integer in a GP register (I, the F
analogue) plus an untagged-with-boxed-cache variant (Si, the Sf analogue);
integer-op lowerings that consume/produce the untagged form; tag on escape to
Value. This is the recorded future direction for integer JIT performance — it
supplies the “boxing-like” per-op cost that finally makes GP register residency
worthwhile.