Skip to content

Conversation

@ShaharNaveh
Copy link
Collaborator

@ShaharNaveh ShaharNaveh commented Nov 29, 2025

One of the changes made is to sort all variants alphabetically, this will makes it easier for finding if an instruction exist, especially when comparing it to https://github.com/python/cpython/blob/cfcd52490d6531f3b0e8ddd4bb2b40fb6baee854/Include/opcode_ids.h (which is sorted alphabetically).

The second change I did was to change each named field to an associated value because we are only holding the oparg type.

For example:

Foo { index: Arg<OpargType> }

to:

Foo(Arg<OpargType>)

IMO this has the following benefits:


The reason why we had named fields was because in the first iterations of the Instruction enum we held the actual oparg value as a named field inside the Instruction, which is no longer the case.

IMO this enum should be renamed to Opcode and have a method called oparg that looks like:

enum Opcode {
     Foo,
     Bar,
     ...
}

impl Opcode {
    fn oparg<T: OpArgType>(self) -> Option<T> { ... }
}

but that's a different discussion

Summary by CodeRabbit

  • Refactor

    • Standardized the instruction/bytecode representation across the compiler, VM, and JIT for a simpler, more consistent surface.
  • New Features

    • Broader VM/JIT support for many additional bytecode operations, improving data-structure construction, call patterns, async/with, and iterator flows.
  • Bug Fixes

    • Improved import/name resolution, mapping/unpacking, exception unwinding, and async/context-manager protocol handling for more correct runtime behavior.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 29, 2025

Walkthrough

Refactors many Instruction enum variants from struct-style to tuple-style, rewrites the Instruction set with Arg payloads, and updates codegen, bytecode, VM frame execution, JIT, and tests to match the new instruction shapes and dispatch.

Changes

Cohort / File(s) Summary
Instruction constructor updates
crates/codegen/src/compile.rs
Convert emit sites from struct-like Instruction::Variant { ... } forms to tuple-like Instruction::Variant(...) for many Build*, Call*, Load/Store, Jump, Unpack, Swap/CopyItem, Resume/Return, and related emissions.
Instruction enum & metadata rewrite
crates/compiler-core/src/bytecode.rs
Replace and expand the Instruction enum: remove many legacy variants, add numerous Build*/Call*/Jump/Finally/Async/etc. variants using Arg<T> payloads, update LAST_INSTRUCTION, stack_effect, label_arg, display/encoding/decoding logic.
VM frame execution overhaul
crates/vm/src/frame.rs
Rework execute_instruction arms to implement the new opcode set: collection builders, control-flow/unwind (Jump, PopJumpIf*, Enter/EndFinally), async/with protocol handling, imports, attribute/method resolution, and richer error paths.
JIT instruction handling alignment
crates/jit/src/instructions.rs
Consolidate and extend add_instruction handling to the new unified instruction variants (CompareOperation, ExtendedArg, Jump, Load/Store, Resume placeholder, Unpack/Build flows) and reorganize per-instruction logic.
Tests / StackMachine adjustments
crates/jit/tests/common.rs
Update StackMachine::process_instruction arms and semantics (BuildMap, ExtendedArg, LoadConst/LoadNameAny, ReturnConst/ReturnValue, SetFunctionAttribute, StoreAttr/StoreLocal) to align with new instruction shapes and stack conventions.

Sequence Diagram(s)

mermaid
sequenceDiagram
participant Codegen
participant BytecodeLib
participant JIT
participant VM
Note over Codegen,BytecodeLib: emit tuple-style Instruction variants
Codegen->>BytecodeLib: emit!(Instruction::BuildTuple(n)) / emit!(Instruction::Jump(label))
BytecodeLib-->>Codegen: assemble instruction bytes & label metadata
Codegen->>JIT: provide instruction stream / metadata
JIT->>BytecodeLib: query instruction metadata (stack_effect, label_arg)
Codegen->>VM: attach assembled code object
VM->>BytecodeLib: query instruction metadata (stack_effect, label_arg)
VM->>VM: execute instruction stream (dispatch per Instruction variant)
VM->>VM: handle control-flow (Jump / PopJumpIf* / EnterFinally / EndFinally)
VM->>VM: perform collection ops (BuildList/BuildMap/BuildTuple)
VM-->>Codegen: emit debug/error info when emission/encoding mismatch

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • Pay attention to:
    • Consistency between crates/compiler-core/src/bytecode.rs variants and all emit sites in crates/codegen/src/compile.rs
    • VM dispatch and stack behavior in crates/vm/src/frame.rs vs. bytecode.rs stack_effect/label_arg
    • LAST_INSTRUCTION sentinel and encoding/decoding impacts
    • JIT handling in crates/jit/src/instructions.rs and test adjustments in crates/jit/tests/common.rs

Possibly related PRs

Suggested reviewers

  • youknowone
  • coolreader18
  • arihant2math

Poem

🐇 I hopped from structs to tiny tuple feet,
Emitted opcodes in a sprightly beat.
Bytecode shifted, the VM learned to play,
JIT twitched its whiskers and joined the relay.
Hooray — a rabbit cheers this tidy fleet!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.31% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the two main changes: alphabetically sorting Instruction variants and converting them from named-field to tuple-style variants with singular oparg types.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6579723 and 85b17af.

📒 Files selected for processing (1)
  • crates/compiler-core/src/bytecode.rs (5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/compiler-core/src/bytecode.rs
🧬 Code graph analysis (1)
crates/compiler-core/src/bytecode.rs (1)
crates/vm/src/frame.rs (2)
  • arg (2449-2450)
  • jump (2002-2006)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Check Rust code with clippy
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run tests under miri
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Ensure compilation on various targets
🔇 Additional comments (4)
crates/compiler-core/src/bytecode.rs (4)

551-691: Excellent enum refactoring—alphabetically sorted and consistently tuple-styled.

The Instruction enum has been successfully refactored with:

  • All variants properly sorted alphabetically
  • Consistent tuple-style Arg<T> payloads throughout
  • Helpful field name comments (e.g., // target, // handler) where appropriate
  • LAST_INSTRUCTION correctly updated to YieldValue

The conversion from named-field to tuple-style variants is clean and consistent with the PR objectives.


1286-1305: label_arg method correctly updated to tuple-style patterns.

All pattern matches have been properly converted to tuple-style destructuring (e.g., Break(label) instead of Break { target: label }), and the method correctly identifies all label-bearing instructions.


1342-1462: Stack effect calculations correctly updated and preserved.

All instruction patterns have been converted to tuple-style, and the stack effect logic remains accurate:

  • Complex cases like BuildMap (multiplies by 2 for key-value pairs) are correct
  • UnpackEx properly extracts before and after fields
  • Conditional effects for ForIter, SetupAsyncWith, etc., are correctly maintained

1480-1641: Display formatting consistently updated to tuple-style patterns.

The fmt_dis method has been thoroughly updated with tuple-style destructuring across all instructions. The formatting logic correctly preserves:

  • Debug formatting for enums (e.g., ?op, ?func)
  • Name lookups via helper functions (e.g., name = idx, varname = idx)
  • Special handling for constants via fmt_const

All instruction variants from the enum are properly covered.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
crates/codegen/src/compile.rs (1)

783-799: Optional: consider making ResumeType an OpArgType to drop the as u32 casts

You currently encode Resume’s argument as bytecode::ResumeType::* as u32 at all sites (function start, yield, await, yield from, async-with/for, async comprehensions). If ResumeType conceptually is the logical op-arg type, implementing OpArgType for it and defining Instruction::Resume(OpArgMarker<ResumeType>) would let you write Instruction::Resume(ResumeType::AfterAwait) directly, avoiding the repeated casts and making the intent clearer at the call sites.

Also applies to: 2836-2845, 2886-2888, 2922-2929, 4417-4454, 4529-4539, 4993-5003, 5058-5064

crates/vm/src/frame.rs (1)

616-625: Remove or document the commented-out code block.

This commented-out ToBool implementation is dead code. Since there's already a working ToBool handler at lines 1479-1484, this block should be removed to reduce noise. If it's meant as a reference for future changes, consider moving it to documentation or a TODO comment.

-            /*
-             bytecode::Instruction::ToBool => {
-                 dbg!("Shouldn't be called outside of match statements for now")
-                 let value = self.pop_value();
-                 // call __bool__
-                 let result = value.try_to_bool(vm)?;
-                 self.push_value(vm.ctx.new_bool(result).into());
-                 Ok(None)
-            }
-            */
crates/compiler-core/src/bytecode.rs (1)

1413-1414: Misleading comment for MatchKeys stack effect.

The comment states "Pop 2 (subject, keys), push 3 (subject, keys_or_none, values_or_none)" but the implementation in frame.rs (lines 1229-1296) shows that MatchKeys doesn't pop subject and keys—it only reads them and pushes one value (the matched values tuple or None). The stack effect of 1 is correct, but the comment is inaccurate.

-            MatchKeys => 1, // Pop 2 (subject, keys), push 3 (subject, keys_or_none, values_or_none)
+            MatchKeys => 1, // Reads subject and keys from stack, pushes values tuple or None
📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e5aec9d and 1c6a0fc.

📒 Files selected for processing (3)
  • crates/codegen/src/compile.rs (114 hunks)
  • crates/compiler-core/src/bytecode.rs (4 hunks)
  • crates/vm/src/frame.rs (8 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style in Rust code (cargo fmt to format)
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management

Files:

  • crates/codegen/src/compile.rs
  • crates/vm/src/frame.rs
  • crates/compiler-core/src/bytecode.rs
**/src/**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/codegen/src/compile.rs
  • crates/vm/src/frame.rs
  • crates/compiler-core/src/bytecode.rs
🧬 Code graph analysis (1)
crates/codegen/src/compile.rs (1)
crates/compiler-core/src/bytecode.rs (1)
  • marker (405-407)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run tests under miri
  • GitHub Check: Check Rust code with clippy
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Ensure compilation on various targets
🔇 Additional comments (19)
crates/codegen/src/compile.rs (16)

238-251: Tuple-style emit! macro, EmitArg, and const-load helpers look correct

The new emit! arm for Instruction::$op(arg) correctly routes through emit_arg, and the EmitArg impls for OpArgType and ir::BlockIdx ensure that:

  • data-bearing opcodes get their OpArg from OpArgMarker::new(...), and
  • label-based opcodes (e.g. Jump, Break, Continue) store the label in InstructionInfo::target with a marker created via OpArgMarker::marker().

emit_load_const / emit_return_const and the emit_return_value peephole now construct and rewrite LoadConst(idx) / ReturnConst(idx) using the marker form, reusing the existing encoded OpArg under the hood. This matches the new enum shape and keeps indices and labels type-safe. Based on the provided marker() definition, this use of OpArgMarker::marker() as a zero-sized type marker is consistent.

Also applies to: 5091-5120, 5454-5475


430-487: BuildSlice refactor keeps 2-/3-argument slice semantics intact

In compile_subscript and the standalone slice expression, Instruction::BuildSlice(n == 3) and Instruction::BuildSlice(step) move from struct fields to tuple-style args while still using a boolean to distinguish 2 vs 3 stack operands. That matches CPython’s BINARY_SLICE/BUILD_SLICE contract and preserves the existing control flow around 2-element vs stepped slices.

Also applies to: 4390-4407


497-538: Collection-building opcodes (Build*, Unpack*) are converted mechanically and stay consistent

Across starunpack_helper, default-argument handling, dict construction, lambda defaults, gather_elements, and f-string assembly you’ve systematically switched:

  • BuildTuple{...}, BuildList{...}, BuildSet{...}, BuildMap{...}, BuildString{...}BuildTuple(size), BuildList(size), BuildSet(size), BuildMap(size), BuildString(count), with size/count always a u32 derived from lengths (to_u32(), try_from), and
  • UnpackSequence{...}, UnpackEx{...}UnpackSequence(n), UnpackEx(args).

The stack shapes (how many elements are consumed/produced) and the computed counts are unchanged; only the enum constructor syntax and arg plumbing differ. No off-by-one or overflow risks are apparent in the new uses.

Also applies to: 2079-2122, 4305-4323, 4475-4501, 4839-4881, 5340-5341, 5447-5448


783-799: Resume opcodes now take raw u32 tags; usage is consistent across scopes and suspension points

All Resume sites (AtFuncStart, AfterYield, AfterAwait, AfterYieldFrom) now pass bytecode::ResumeType::* as u32, which matches the new tuple-style constructor and appears consistent across:

  • function/module entry (enter_scope),
  • async with / async for,
  • yield, await, yield from,
  • generator-based comprehensions.

Every Resume is still paired with the correct preceding Yield*/GetAwaitable/YieldFrom. As long as Resume’s oparg type in Instruction is indeed u32 (or a type alias thereof), this is semantically equivalent.

Also applies to: 2836-2845, 2886-2888, 2922-2929, 4417-4454, 4529-4539, 4993-5003, 5058-5064


1062-1075: CopyItem / Swap depth indices remain aligned with previous stack discipline

You’ve replaced legacy dup/rot instructions with tuple-style CopyItem(depth) and Swap(depth) in several stack-sensitive places:

  • REPL last-expression printing and block expressions,
  • multi-target assignments and named expressions,
  • __classcell__ capture in class bodies,
  • pattern-matching helpers (sequence/mapping/or patterns),
  • chained comparisons and guard handling,
  • augmented assignment for subscripts/attributes.

All these call sites preserve the old constant depths (1_u32, 2_u32, 3, 1 + remaining, etc.), and the surrounding comments/logic still match the expected stack shapes (e.g., copying the subject before pattern attempts, reordering container/slice/result in augassign). I don’t see any depth mismatches or regressions here.

Also applies to: 1087-1098, 1636-1644, 2626-2633, 3163-3175, 3230-3233, 3331-3337, 3492-3551, 3595-3603, 3838-3850, 3947-3954, 4128-4145, 4162-4170, 4648-4651


1305-1391: Import and attribute opcodes (Import*, Load*/Store*/DeleteAttr, LoadMethod) updated cleanly

The various import and attribute-related instructions now use tuple-style constructors:

  • ImportName(idx), ImportFrom(idx), ImportNameless,
  • LoadAttr(idx), StoreAttr(idx), DeleteAttr(idx),
  • LoadMethod(idx).

Each idx is consistently obtained via self.name(...) (a NameIdx), and the surrounding control flow (from-import *, chained attribute loads, delete-attribute, method calls) is unchanged. This is a straightforward enum-shape refactor with no semantic change.

Also applies to: 1721-1735, 4041-4053, 4362-4366, 4742-4751


1408-1463: Control-flow opcodes with label operands correctly leverage the new Label plumbing

All structured control flow now routes label targets through emit!(..., Instruction::<Op>(block_idx)), relying on the new EmitArg<bytecode::Label> for ir::BlockIdx:

  • Jump(...) in if/elif chains, while/for loops, try/except/else/finally, match, and comprehensions,
  • loop control (Break(exit_block), Continue(loop_block), ForIter(else_block)),
  • exception setup (SetupExcept(handler_block), SetupFinally(finally_block)),
  • boolean / guard jumps (PopJumpIfTrue/False, JumpIfFalseOrPop, JumpIfTrueOrPop).

Targets are always the appropriate BlockIdx (e.g., after/else blocks, handler blocks, loop heads/tails), and there’s no sign of miswired labels. This matches the new label-based Instruction design without altering high-level control flow.

Also applies to: 1553-1607, 1958-2061, 2791-2816, 2896-2965, 2999-3015, 3812-3890, 4207-4277, 4281-4303, 4990-5003


1465-1479: Raise and comparison-related opcodes migrated to tuple-style with unchanged semantics

The refactors:

  • Raise(kind) for the three RaiseKind cases,
  • CompareOperation(ComparisonOperator::...) in mapping/sequence patterns and chained comparisons,
  • ContainsOp(Invert::...), IsOp(Invert::...) in comparisons and pattern checks,

are all one-to-one replacements of the prior struct-style variants. The chosen ComparisonOperator values and Invert flags still align with the surrounding logic (>= for mapping-length checks, == for value patterns, ==/>= in chained comparisons, etc.). No behavioural differences are introduced.

Also applies to: 3421-3443, 3720-3739, 3756-3765, 3898-3973


1806-1955: Type-parameter and generic handling matches the new intrinsic and function-attribute opcodes

In the type-params pipeline:

  • compile_type_param_bound_or_default uses UnpackSequence(1) only when allowed and wraps evaluation in a tiny function called via CallFunctionPositional(0), as before.
  • compile_type_params now emits CallIntrinsic1(TypeVar/ParamSpec/TypeVarTuple) and CallIntrinsic2(TypeVarWithBound/Constraint/SetTypeparamDefault) with tuple-style args, plus CopyItem(1) + store_name to bind each type param. The counts passed to BuildTuple(...) are still derived from type_params.len().

For generic functions/classes:

  • compile_function_def and compile_class_def use Swap(2) / Swap(num_typeparam_args + 1) and CallIntrinsic2(SetFunctionTypeParams) to attach type params, and later CallFunctionPositional(num_typeparam_args) to invoke the wrapper closure.
  • In class compilation, CallIntrinsic1(SubscriptGeneric) and the subsequent CallFunctionKeyword/Positional for __build_class__ are updated mechanically to tuple-style with the same argument counts.

Across these sites the stack comments and arg counts still line up with the design; I don’t see any regressions in how generics are threaded through.

Also applies to: 2550-2640, 2656-2760


1958-2061: Try/except/finally, async-with, async-for, and async comprehensions remain structurally equivalent

The conversions to:

  • SetupFinally(finally_block), SetupExcept(handler_block/after_block), Jump(else_block/finally_block), Raise(Reraise) in compile_try_statement,
  • SetupAsyncWith(final_block) / SetupWith(final_block) plus Resume(AfterAwait as u32) in compile_with,
  • async-for’s SetupExcept(else_block), Resume(AfterAwait as u32), and EndAsyncFor,
  • async comprehensions’ SetupExcept(after_block), Resume(AfterAwait as u32), and EndAsyncFor,

all keep the same block structure and stack behaviour used previously, just through tuple-style variants. Exception handling, finally unwinding, and async control flow should behave identically.

Also applies to: 2818-2894, 2896-2965, 4990-5003


3096-3570: Pattern-matching opcodes (Unpack*, Match*, mapping/sequence/or-pattern helpers) are consistently updated

In the structural pattern-matching helpers you’ve migrated to tuple-style opcodes:

  • UnpackEx(args) / UnpackSequence(n) in sequence unpacking and star patterns,
  • MatchClass(nargs), MatchMapping, MatchSequence,
  • mapping patterns’ BuildTuple(size) for keys, BuildMap(0), DictUpdate(2), UnpackSequence(size), and the CopyItem/Swap choreography for deleting keys into DeleteSubscript,
  • BinaryOperation(BinaryOperator::Subtract) for index arithmetic,
  • various CopyItem/Swap calls in or-pattern store alignment and sequence subscripting.

The updated instructions mirror the CPython algorithm and the prior Rust implementation; counts are still derived from elts.len(), nargs + n_attrs, or size checks that guard against overflow. I don’t see off-by-one issues or stack mismanagement introduced by the tuple-style transition here.

Also applies to: 3680-3769


3822-3890: match case dispatch and guards use the new jump opcodes correctly

For match statements:

  • Each non-default case now uses CopyItem(1) to duplicate the subject (except the last checked case), compiles the pattern, then jumps with Jump(end) after the case body.
  • Guards use PopJumpIfFalse(pattern_context.fail_pop[0]) to fall back to the failure label, and you still call emit_and_reset_fail_pop between cases.
  • The final default case, when present, uses JumpIfFalseOrPop(end) for its guard.

The tuple-style Jump/PopJumpIfFalse/JumpIfFalseOrPop opcodes target the same blocks as before, and the subject clean-up (Pop) and store binding logic remain intact. Control flow through cases is unchanged.


3898-3973: Chained comparisons and boolean short-circuiting preserve previous control-flow structure

For chained comparisons:

  • CompareOperation(Equal/Less/LessOrEqual/Greater/GreaterOrEqual) is now emitted via the tuple-style variant.
  • The Swap(2) + CopyItem(2) pattern to retain the RHS for the next comparison, and JumpIfFalseOrPop(break_block) for early exit, are unchanged in logic.

For boolean expressions:

  • compile_jump_if’s fallback path now uses PopJumpIfTrue/False(target_block),
  • compile_bool_op uses JumpIfFalseOrPop / JumpIfTrueOrPop with the same block layout (after_block).

Only the constructor syntax changed; the jump conditions, labels, and stack effects remain as before.

Also applies to: 4207-4277, 4281-4303


4350-4366: Expression-level opcodes (unary, slice, yield/await, comprehensions, f-strings) are refactored mechanically

Key expression sites now use tuple-style variants:

  • UnaryOperation(op) after unary exprs, with op unchanged.
  • BuildSlice(step) in slice expressions (bool step flag).
  • YieldValue + Resume(AfterYield/AfterYieldFrom) and GetAwaitable + YieldFrom + Resume(AfterAwait) for yield, yield from, await.
  • Comprehensions’ initializers BuildList/BuildSet/BuildMap(OpArgMarker::marker()) and per-element ops ListAppend(len), SetAdd(len), MapAdd(len) in compile_comprehension.
  • f-strings using FormatValue(conversion) and BuildString(count) for both nested and top-level formatting.

Given the counts and enums are unchanged, these are straightforward enum-shape updates without semantic changes.

Also applies to: 4406-4407, 4417-4454, 4538-4584, 4593-4612, 5340-5341, 5436-5448


4720-4834: Call and keyword handling (CallFunction* / CallMethod* / BuildMap*) align with the new API

In the call path:

  • compile_call now uses LoadMethod(idx) for method calls, then dispatches through compile_method_call vs compile_normal_call.
  • compile_normal_call and compile_method_call cleanly map CallType::{Positional,Keyword,Ex} to CallFunctionPositional/Keyword/Ex and their method counterparts.
  • compile_call_inner uses BuildTupleFromTuples(size) vs BuildTuple(size) for positional args, and CallFunctionKeyword(count) when keyword names are packed into a const tuple.
  • compile_keywords emits BuildMap(sub_size) for each group of named keywords and BuildMapForCall(size) when multiple mapping segments are to be merged.

All argument counts are derived from arguments.len(), size, or keyword group sizes as before, so the switch to tuple-style opcodes should not affect call semantics.


5130-5136: Const-return peephole still works with marker-based LoadConst / ReturnConst

emit_load_const / emit_return_const now create LoadConst(idx) / ReturnConst(idx) using emit_arg, and emit_return_value rewrites a terminal LoadConst(idx) into ReturnConst(idx) by pattern-matching on the new tuple-style variant.

Because the OpArg payload is untouched when you mutate inst.instr, the encoded constant index is preserved. This keeps the peephole optimization intact under the new Instruction representation.

Also applies to: 5140-5143

crates/vm/src/frame.rs (2)

882-885: TODO placeholder always returns true.

The gen_is_coroutine closure always returns true, which may not correctly identify whether a generator is actually a coroutine. This could affect the behavior of async iteration. The TODO is acknowledged, but consider adding an issue reference to track this.


533-1543: The execute_instruction match arms are correctly updated.

The transition to tuple-style instruction variants with .get(arg) accessor pattern is consistently applied throughout the execution logic. The alphabetical ordering of match arms aligns with the bytecode definition (with the noted exception of EnterFinally/ExtendedArg order in bytecode.rs).

crates/compiler-core/src/bytecode.rs (1)

551-691: Tuple-style variant refactoring looks correct.

The conversion from named-field variants (e.g., Foo { index: Arg<T> }) to tuple-style variants (e.g., Foo(Arg<T>)) is consistently applied. The LAST_INSTRUCTION sentinel is correctly updated to YieldValue (the last variant alphabetically after fixing the EnterFinally/ExtendedArg order).

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/compiler-core/src/bytecode.rs (1)

1316-1326: Critical: Pattern matching not updated for tuple-style variants.

The unconditional_branch() method still uses struct-style patterns (Jump { .. }) but the Instruction enum variants were changed to tuple-style (Jump(Arg<Label>)). This mismatch will cause a compilation error.

Apply this diff to fix the pattern matching:

     pub const fn unconditional_branch(&self) -> bool {
         matches!(
             self,
-            Jump { .. }
-                | Continue { .. }
-                | Break { .. }
+            Jump(..)
+                | Continue(..)
+                | Break(..)
                 | ReturnValue
-                | ReturnConst { .. }
-                | Raise { .. }
+                | ReturnConst(..)
+                | Raise(..)
         )
     }
📜 Review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d917dbc and 6579723.

📒 Files selected for processing (5)
  • crates/codegen/src/compile.rs (114 hunks)
  • crates/compiler-core/src/bytecode.rs (5 hunks)
  • crates/jit/src/instructions.rs (4 hunks)
  • crates/jit/tests/common.rs (4 hunks)
  • crates/vm/src/frame.rs (8 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style in Rust code (cargo fmt to format)
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management

Files:

  • crates/jit/src/instructions.rs
  • crates/jit/tests/common.rs
  • crates/codegen/src/compile.rs
  • crates/vm/src/frame.rs
  • crates/compiler-core/src/bytecode.rs
**/src/**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/jit/src/instructions.rs
  • crates/codegen/src/compile.rs
  • crates/vm/src/frame.rs
  • crates/compiler-core/src/bytecode.rs
🧬 Code graph analysis (2)
crates/codegen/src/compile.rs (1)
crates/compiler-core/src/bytecode.rs (1)
  • marker (405-407)
crates/compiler-core/src/bytecode.rs (2)
crates/vm/src/frame.rs (2)
  • arg (2449-2450)
  • jump (2002-2006)
crates/codegen/src/ir.rs (1)
  • idx (35-37)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: Run tests under miri
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Check Rust code with clippy
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Ensure compilation on various targets
🔇 Additional comments (13)
crates/jit/tests/common.rs (1)

95-173: StackMachine opcode updates align with new compiler emission

The new handlers for BuildMap, ExtendedArg, LoadConst, LoadNameAny, ReturnConst, ReturnValue, SetFunctionAttribute, StoreAttr, and StoreLocal are consistent with how compile.rs now emits bytecode (especially for annotations, function creation, and locals). Stack shapes for annotations (LoadNameAny + BuildMap + SetFunctionAttribute(ANNOTATIONS)) and for storing functions into locals via StoreLocal look correct for the jit_function! test harness.

No blocking issues here from a JIT test perspective.

crates/jit/src/instructions.rs (4)

198-208: Unreachable-code tracking correctly extended to ReturnConst

Marking in_unreachable_code = true for both Instruction::ReturnValue and Instruction::ReturnConst(_) ensures no further IR is emitted in the same block after any return. This matches how ReturnConst now behaves and avoids malformed Cranelift blocks.


428-480: CompareOperation handling is type‑sound for int/bool/float combinations

The new Instruction::CompareOperation arm correctly:

  • Pops b then a (rhs, lhs) and inspects their JitType.
  • Widens Bool to I64 before integer comparisons, treating bools as 0/1.
  • Maps all ComparisonOperator variants to the appropriate IntCC/FloatCC.
  • Rejects unsupported type pairs with JitCompileError::NotSupported.

This keeps the JIT’s comparison semantics aligned with Python’s for ints, bools, and floats.


514-548: Branch lowering for PopJumpIfFalse/True matches bytecode semantics

The new PopJumpIfFalse / PopJumpIfTrue arms:

  • Pop the condition, coerce it via boolean_val, and create a then_block for the jump target and a fresh fall‑through block.
  • Use brif with the target and fall‑through blocks wired in the correct order (false → target for PopJumpIfFalse, true → target for PopJumpIfTrue), then switch the builder to the fall‑through block.

This matches the CPython POP_JUMP_IF_* behavior while integrating cleanly with the label‑based block mapping.


553-618: New control‑flow and stack ops (ExtendedArg, Jump, Load*, Pop, PopBlock, Resume, ReturnConst/Value, SetupLoop, StoreFast, UnpackSequence) look consistent

The added/updated cases in add_instruction for:

  • ExtendedArg (no‑op),
  • Jump,
  • LoadConst / LoadFast / LoadGlobal,
  • Pop / PopBlock,
  • Resume (placeholder),
  • ReturnConst / ReturnValue,
  • SetupLoop,
  • StoreFast,
  • UnpackSequence,

all use reasonable stack discipline and error out with BadBytecode / NotSupported on malformed input. In particular:

  • LoadGlobal only supports the function’s own name, pushing a FuncRef, which is a clear, safe restriction.
  • UnpackSequence checks that the tuple length matches size.get(arg) before extending the stack in reverse, matching Python’s UNPACK_SEQUENCE behavior. (docs.python.org)

These additions are fine for the current, limited JIT surface.

crates/codegen/src/compile.rs (6)

441-461: Slice emission with BuildSlice(n == 3) matches 2/3‑arg slice semantics

compile_subscript now calls:

let n = self.compile_slice(s)?;
match ctx {
    ExprContext::Load => {
        emit!(self, Instruction::BuildSlice(n == 3));
        emit!(self, Instruction::Subscript);
    }
    ExprContext::Store => {
        emit!(self, Instruction::BuildSlice(n == 3));
        emit!(self, Instruction::StoreSubscript);
    }
    ...
}

With compile_slice returning 3 when a step is present and 2 otherwise, this lines up with Python 3.12’s BUILD_SLICE taking a boolean “has step” flag rather than an explicit count, while keeping the existing stack discipline.


150-217: Function annotations and SetFunctionAttribute(ANNOTATIONS) are wired correctly

The combination of:

  • visit_annotations pushing name/annotation pairs and then emit!(self, Instruction::BuildMap(num_annotations)), and
  • compile_function_def OR’ing MakeFunctionFlags::ANNOTATIONS into funcflags and passing it into compile_function_body / make_closure,

ensures that when make_closure runs, the annotations dict is already on the stack beneath the freshly created function, so SetFunctionAttribute(ANNOTATIONS) can consume [annotations_dict, func] and attach the annotations. This matches the new attribute‑based MAKE_FUNCTION design and the JIT test harness’ expectation that annotations live in a single map per function.


2091-2132: Default/kw‑default handling via BuildTuple and BuildMap is consistent

compile_default_arguments now:

  • Compiles positional defaults and wraps them in BuildTuple(defaults.len().to_u32()), setting DEFAULTS in MakeFunctionFlags.
  • Compiles keyword‑only defaults into key/value pairs and wraps them in BuildMap(kw_with_defaults.len().to_u32()), setting KW_ONLY_DEFAULTS.

These opcodes match Python’s BUILD_TUPLE / BUILD_MAP(count) semantics (tuple of defaults, dict of kwdefaults), and make_closure later consumes them via SetFunctionAttribute(DEFAULTS) and SetFunctionAttribute(KW_ONLY_DEFAULTS) in the right stack order.


5122-5140: emit_return_value peephole to ReturnConst is safe and reduces bytecode

The new emit_return_value:

fn emit_return_value(&mut self) {
    if let Some(inst) = self.current_block().instructions.last_mut()
        && let Instruction::LoadConst(idx) = inst.instr
    {
        inst.instr = Instruction::ReturnConst(idx);
        return;
    }
    emit!(self, Instruction::ReturnValue)
}

reuses the existing OpArg payload and only changes the opcode variant, so the op‑arg encoding stays correct while shrinking a LOAD_CONST; RETURN_VALUE pair into a single RETURN_CONST. This matches the new RETURN_CONST opcode and is a straightforward win.


4303-4314: Dict literal emission (BuildMap + DictUpdate) matches CPython 3.9+

compile_dict now:

  • Emits key/value pairs for non‑unpacked entries followed by BuildMap(size).
  • For each **mapping entry, evaluates the mapping expression then emits DictUpdate(1).

This mirrors CPython’s BUILD_MAP(count) followed by DICT_UPDATE(i) semantics for unpacked mappings, with size as the number of packed key/value pairs. (docs.python.org)


3033-3045: Based on my verification through web searches, I can confirm key claims from the review comment:

✓ Verified:

  • CPython's SWAP(i) does use 1-based indexing: SWAP(i) swaps STACK[-i] with STACK[-1]
  • SWAP(1) is indeed a no-op (swaps TOS with itself)
  • RustPython implements SWAP matching CPython's semantics

✗ Cannot Verify (Repository access unavailable):

  • The claimed off-by-one in the JIT implementation
  • Whether pattern_helper_rotate actually works correctly with RustPython's implementation of Instruction::Swap
  • The actual behavior of Instruction::CopyItem and its semantic alignment with COPY(i)

The review comment's core assumption about CPython-style 1-based depth semantics is correct; however, verification of the specific JIT off-by-one and correctness of pattern_helper_rotate requires repository access.

The reviewer's request to confirm that Instruction::Swap and Instruction::CopyItem use CPython's 1-based depth semantics is reasonable and well-founded, given that RustPython is documented to follow CPython's semantics. The suggestion to verify this alignment is sound, as semantic inconsistencies between VM components would indeed cause bugs. However, without access to the RustPython repository to inspect the actual JIT implementation and the claimed off-by-one error, the verification cannot be completed.

crates/vm/src/frame.rs (1)

534-1543: LGTM! Mechanical conversion is correct and complete.

All instruction handlers have been correctly updated from struct-style to tuple-style syntax. The pattern is consistent throughout: each instruction variant uses Foo(marker) followed by marker.get(arg) to extract the actual argument value. The unsafe operations are appropriately documented and follow the established "trust compiler" pattern.

crates/compiler-core/src/bytecode.rs (1)

552-691: Enum refactoring looks good overall.

The Instruction enum variants have been successfully:

  1. Converted from struct-style to tuple-style (e.g., Foo { index: Arg<T> }Foo(Arg<T>))
  2. Alphabetically sorted as intended
  3. Properly reflected in LAST_INSTRUCTION constant

The conversion is complete and correct for the enum definition itself.

Comment on lines +568 to 577
Instruction::StoreFast(idx) => {
let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(idx.get(arg), val)
}
Instruction::Swap(index) => {
let len = self.stack.len();
let i = len - 1;
let j = len - 1 - index.get(arg) as usize;
self.stack.swap(i, j);
Ok(())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, let's examine the current Swap implementation and related code
cat -n crates/jit/src/instructions.rs | sed -n '565,580p'

Repository: RustPython/RustPython

Length of output: 186


🏁 Script executed:

# Search for compile.rs to see how SWAP is emitted
fd -t f "compile.rs" --exec cat -n {} +

Repository: RustPython/RustPython

Length of output: 186


🏁 Script executed:

# Search for tests related to Swap instruction
rg -n "Swap" crates/jit/src --type=rs -A 3 -B 3

Repository: RustPython/RustPython

Length of output: 186


🏁 Script executed:

# Look for any test files that exercise SWAP patterns
fd -t f -e rs --path "*/test*" | xargs rg -l "Swap\|SWAP" 2>/dev/null | head -20

Repository: RustPython/RustPython

Length of output: 186


🏁 Script executed:

# Check for any bytecode or stack trace comments explaining SWAP
rg -n "STACK\[" crates/jit/src --type=rs -A 2 -B 2

Repository: RustPython/RustPython

Length of output: 186


🌐 Web query:

CPython 3.12 SWAP bytecode instruction semantics documentation

💡 Result:

SWAP(i) in CPython 3.12 swaps the top-of-stack with the i‑th-from-top element:

STACK[-i], STACK[-1] = STACK[-1], STACK[-i]. [1][2]

(Opcode documented in the Python dis module; added in 3.11 and present in 3.12.) [1][2]

Sources:
[1] Python 3.12 dis — Disassembler for Python bytecode. [2] Python documentation (dis) (same opcode entry).


🌐 Web query:

Python dis SWAP instruction stack operations 1-based depth TOS

💡 Result:

SWAP(i) (added in CPython 3.11) swaps the top-of-stack (TOS) with the i‑th element below it, where i is 1‑based. Formally:

STACK[-i], STACK[-1] = STACK[-1], STACK[-i]. [1][2]

Example: stack = [A, B, C, D] (TOS = D)

  • SWAP(1) leaves it [A, B, C, D] (swap TOS with itself)
  • SWAP(2) → [A, B, D, C] (swap D and C)
  • SWAP(4) → [D, B, C, A] (swap D and A). [1][2]

Sources: Python dis module documentation. [1][2]


🌐 Web query:

CPython 3.12 augmented assignment s[a] += b bytecode SWAP pattern

💡 Result:

Short answer: CPython 3.12 uses two SWAPs to reorder the stack so STORE_SUBSCR sees (key, container, value) as it expects after the in-place BINARY_OP.

Why: BINARY_SUBSCR (earlier) leaves the stack as: ..., s, a, s[a]. After loading b and executing BINARY_OP (+=) the stack is ..., s, a, (s[a] + b). STORE_SUBSCR implements
key = STACK.pop()
container = STACK.pop()
value = STACK.pop()
container[key] = value
so the top three stack items must be (value, container, key) before STORE_SUBSCR runs. The two SWAP ops move the computed value down into the right position (first SWAP 3 moves the value below s and a; SWAP 2 then places key and container in the order STORE_SUBSCR expects). See the dis/bytecode semantics and an example disassembly of s[a] += b. [1][2]

Sources:

  • dis docs (BINARY_SUBSCR, BINARY_OP, STORE_SUBSCR semantics). [1]
  • Example disassembly and stack-walk explanation for s[a] += b showing the COPY/BINARY_SUBSCR → BINARY_OP → SWAP, SWAP → STORE_SUBSCR sequence. [2]

Swap instruction has a confirmed off‑by‑one error in JIT compilation

The web search confirms that CPython 3.12's SWAP(i) bytecode uses 1-based indexing and swaps STACK[-i] with STACK[-1] (the top of stack). For example, with stack [A, B, C, D] (TOS = D), SWAP(2) produces [A, B, D, C] by swapping D and C.

Tracing through the current implementation:

  • For SWAP(2) on a stack of length 4: i = 3, j = 4 - 1 - 2 = 1 → swaps stack[3] with stack[1] (D with B), yielding [A, D, C, B]
  • Correct formula should be: j = len - index.get(arg) as usizej = 4 - 2 = 2 → swaps stack[3] with stack[2] (D with C), yielding [A, B, D, C]

This off-by-one error will cause incorrect stack reordering for any JIT-compiled code using Swap, including augmented assignment patterns like s[a] += b which use SWAP 3; SWAP 2 in CPython 3.12+.

Recommended fix:

- let j = len - 1 - index.get(arg) as usize;
+ let j = len - index.get(arg) as usize;

Add a unit test exercising SWAP 2 and SWAP 3 patterns to lock in correct semantics.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Instruction::StoreFast(idx) => {
let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(idx.get(arg), val)
}
Instruction::Swap(index) => {
let len = self.stack.len();
let i = len - 1;
let j = len - 1 - index.get(arg) as usize;
self.stack.swap(i, j);
Ok(())
Instruction::StoreFast(idx) => {
let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?;
self.store_variable(idx.get(arg), val)
}
Instruction::Swap(index) => {
let len = self.stack.len();
let i = len - 1;
let j = len - index.get(arg) as usize;
self.stack.swap(i, j);
Ok(())
🤖 Prompt for AI Agents
In crates/jit/src/instructions.rs around lines 568 to 577, the SWAP
implementation uses an off-by-one index calculation (j = len - 1 -
index.get(arg) as usize) which incorrectly treats the operand as 0-based; change
the calculation to use 1-based semantics (j = len - index.get(arg) as usize) so
SWAP(i) swaps STACK[-i] with STACK[-1], and update any comments accordingly;
also add unit tests exercising SWAP 2 and SWAP 3 (and combined patterns like
SWAP 3; SWAP 2 used in augmented assignment) to validate correct stack
reordering.

SetupLoop,

SetupAsyncWith(Arg<Label>),
SetupExcept(Arg<Label>),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about at least keeping the name as comment?

e.g.

Suggested change
SetupExcept(Arg<Label>),
SetupExcept(Arg<Label>), // handler

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can do that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also tell me about the motivation to remove the label? They compiled to the same bytecode, but having label is easier to read the code

Copy link
Collaborator Author

@ShaharNaveh ShaharNaveh Nov 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The motivation is the inconsistenty between opcodes that holds the same OpArgType, some holds it as an associated value and some as a named field, which is a bit confusing imo. also, I find it very tedious and just more boilerplate for having different an enum that it's variants hold nothing or a single named field.

I'm not so good with words so here's what I mean:

I can justify having enums like this where different variants hold different fields, even if some overlapping:

enum Foo {
    A { name: String, age: u8 },
    B { name: String, job: String, married: bool },
    ...
}

But I do find it redundant and overly verbose to have something like this:

enum Foo {
    A { name: String },
    B { name: String },
    C { name: String },
    ...
}

Not to mention that we have cases like this:

enum Foo {
    A { target: Label },
    B { end: Label },
    C { handler: Label },
    ...
}

Because Label is a newtype for u32 already, it's just confusing as to why the same newtype is being "treated" differently. I'd much rather to go with newtyprs all the way like:

enum Foo {
    A(TargetLabel),
    B(EndLabel),
    C(HandlerLabel),
    ...
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@youknowone I can revert that change if that's a deal breaker for you:)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the new enum idea on #6313. Not sure about Label. Because all the labels are actually label.

If the consistency matters, giving label for every instruction looks better to me. It feels like redundancy while writing code, but we read code more than writing code. Getting information from code is easier than checking document.

The situation is also a bit different to your example. It is more like:

enum Foo {
    CallMethodPositional { nargs: String },
    ReturnConst { idx: String },
    ...
}

Maybe giving different types for them will also make sense though.

cc @coolreader18

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the new enum idea on #6313. Not sure about Label. Because all the labels are actually label.

If the consistency matters, giving label for every instruction looks better to me. It feels like redundancy while writing code, but we read code more than writing code. Getting information from code is easier than checking document.

IMO consistency matters, I can make them consistent in the other way around by having each Oparg to be stored as a named field if that's desired.

The situation is also a bit different to your example. It is more like:

enum Foo {
    CallMethodPositional { nargs: String },
    ReturnConst { idx: String },
    ...
}

Maybe giving different types for them will also make sense though.

I think that it will, but that is out of scope for this PR, I'll do it in a separate one

cc @coolreader18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants