-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Sort Instruction variants + Ensure each holds a singular oparg type
#6312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
WalkthroughRefactors many Instruction enum variants from struct-style to tuple-style, rewrites the Instruction set with Arg payloads, and updates codegen, bytecode, VM frame execution, JIT, and tests to match the new instruction shapes and dispatch. Changes
Sequence Diagram(s)mermaid Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes
Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)**/*.rs📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
🧬 Code graph analysis (1)crates/compiler-core/src/bytecode.rs (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
🔇 Additional comments (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (3)
crates/codegen/src/compile.rs (1)
783-799: Optional: consider makingResumeTypeanOpArgTypeto drop theas u32castsYou currently encode
Resume’s argument asbytecode::ResumeType::* as u32at all sites (function start,yield,await,yield from, async-with/for, async comprehensions). IfResumeTypeconceptually is the logical op-arg type, implementingOpArgTypefor it and definingInstruction::Resume(OpArgMarker<ResumeType>)would let you writeInstruction::Resume(ResumeType::AfterAwait)directly, avoiding the repeated casts and making the intent clearer at the call sites.Also applies to: 2836-2845, 2886-2888, 2922-2929, 4417-4454, 4529-4539, 4993-5003, 5058-5064
crates/vm/src/frame.rs (1)
616-625: Remove or document the commented-out code block.This commented-out
ToBoolimplementation is dead code. Since there's already a workingToBoolhandler at lines 1479-1484, this block should be removed to reduce noise. If it's meant as a reference for future changes, consider moving it to documentation or a TODO comment.- /* - bytecode::Instruction::ToBool => { - dbg!("Shouldn't be called outside of match statements for now") - let value = self.pop_value(); - // call __bool__ - let result = value.try_to_bool(vm)?; - self.push_value(vm.ctx.new_bool(result).into()); - Ok(None) - } - */crates/compiler-core/src/bytecode.rs (1)
1413-1414: Misleading comment forMatchKeysstack effect.The comment states "Pop 2 (subject, keys), push 3 (subject, keys_or_none, values_or_none)" but the implementation in
frame.rs(lines 1229-1296) shows thatMatchKeysdoesn't pop subject and keys—it only reads them and pushes one value (the matched values tuple or None). The stack effect of1is correct, but the comment is inaccurate.- MatchKeys => 1, // Pop 2 (subject, keys), push 3 (subject, keys_or_none, values_or_none) + MatchKeys => 1, // Reads subject and keys from stack, pushes values tuple or None
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
crates/codegen/src/compile.rs(114 hunks)crates/compiler-core/src/bytecode.rs(4 hunks)crates/vm/src/frame.rs(8 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Follow the default rustfmt code style in Rust code (cargo fmtto format)
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
crates/codegen/src/compile.rscrates/vm/src/frame.rscrates/compiler-core/src/bytecode.rs
**/src/**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use the macro system (
pyclass,pymodule,pyfunction, etc.) when implementing Python functionality in Rust
Files:
crates/codegen/src/compile.rscrates/vm/src/frame.rscrates/compiler-core/src/bytecode.rs
🧬 Code graph analysis (1)
crates/codegen/src/compile.rs (1)
crates/compiler-core/src/bytecode.rs (1)
marker(405-407)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Run rust tests (windows-latest)
- GitHub Check: Check the WASM package and demo
- GitHub Check: Run snippets and cpython tests (windows-latest)
- GitHub Check: Run snippets and cpython tests (macos-latest)
- GitHub Check: Run tests under miri
- GitHub Check: Check Rust code with clippy
- GitHub Check: Run snippets and cpython tests on wasm-wasi
- GitHub Check: Ensure compilation on various targets
🔇 Additional comments (19)
crates/codegen/src/compile.rs (16)
238-251: Tuple-styleemit!macro,EmitArg, and const-load helpers look correctThe new
emit!arm forInstruction::$op(arg)correctly routes throughemit_arg, and theEmitArgimpls forOpArgTypeandir::BlockIdxensure that:
- data-bearing opcodes get their
OpArgfromOpArgMarker::new(...), and- label-based opcodes (e.g.
Jump,Break,Continue) store the label inInstructionInfo::targetwith a marker created viaOpArgMarker::marker().
emit_load_const/emit_return_constand theemit_return_valuepeephole now construct and rewriteLoadConst(idx)/ReturnConst(idx)using the marker form, reusing the existing encodedOpArgunder the hood. This matches the new enum shape and keeps indices and labels type-safe. Based on the providedmarker()definition, this use ofOpArgMarker::marker()as a zero-sized type marker is consistent.Also applies to: 5091-5120, 5454-5475
430-487:BuildSlicerefactor keeps 2-/3-argument slice semantics intactIn
compile_subscriptand the standalone slice expression,Instruction::BuildSlice(n == 3)andInstruction::BuildSlice(step)move from struct fields to tuple-style args while still using a boolean to distinguish 2 vs 3 stack operands. That matches CPython’sBINARY_SLICE/BUILD_SLICEcontract and preserves the existing control flow around 2-element vs stepped slices.Also applies to: 4390-4407
497-538: Collection-building opcodes (Build*,Unpack*) are converted mechanically and stay consistentAcross
starunpack_helper, default-argument handling, dict construction, lambda defaults,gather_elements, and f-string assembly you’ve systematically switched:
BuildTuple{...},BuildList{...},BuildSet{...},BuildMap{...},BuildString{...}→BuildTuple(size),BuildList(size),BuildSet(size),BuildMap(size),BuildString(count), withsize/countalways au32derived from lengths (to_u32(),try_from), andUnpackSequence{...},UnpackEx{...}→UnpackSequence(n),UnpackEx(args).The stack shapes (how many elements are consumed/produced) and the computed counts are unchanged; only the enum constructor syntax and arg plumbing differ. No off-by-one or overflow risks are apparent in the new uses.
Also applies to: 2079-2122, 4305-4323, 4475-4501, 4839-4881, 5340-5341, 5447-5448
783-799:Resumeopcodes now take rawu32tags; usage is consistent across scopes and suspension pointsAll
Resumesites (AtFuncStart,AfterYield,AfterAwait,AfterYieldFrom) now passbytecode::ResumeType::* as u32, which matches the new tuple-style constructor and appears consistent across:
- function/module entry (
enter_scope),- async with / async for,
yield,await,yield from,- generator-based comprehensions.
Every
Resumeis still paired with the correct precedingYield*/GetAwaitable/YieldFrom. As long asResume’s oparg type inInstructionis indeedu32(or a type alias thereof), this is semantically equivalent.Also applies to: 2836-2845, 2886-2888, 2922-2929, 4417-4454, 4529-4539, 4993-5003, 5058-5064
1062-1075:CopyItem/Swapdepth indices remain aligned with previous stack disciplineYou’ve replaced legacy dup/rot instructions with tuple-style
CopyItem(depth)andSwap(depth)in several stack-sensitive places:
- REPL last-expression printing and block expressions,
- multi-target assignments and named expressions,
__classcell__capture in class bodies,- pattern-matching helpers (sequence/mapping/or patterns),
- chained comparisons and guard handling,
- augmented assignment for subscripts/attributes.
All these call sites preserve the old constant depths (
1_u32,2_u32,3,1 + remaining, etc.), and the surrounding comments/logic still match the expected stack shapes (e.g., copying the subject before pattern attempts, reordering container/slice/result in augassign). I don’t see any depth mismatches or regressions here.Also applies to: 1087-1098, 1636-1644, 2626-2633, 3163-3175, 3230-3233, 3331-3337, 3492-3551, 3595-3603, 3838-3850, 3947-3954, 4128-4145, 4162-4170, 4648-4651
1305-1391: Import and attribute opcodes (Import*,Load*/Store*/DeleteAttr,LoadMethod) updated cleanlyThe various import and attribute-related instructions now use tuple-style constructors:
ImportName(idx),ImportFrom(idx),ImportNameless,LoadAttr(idx),StoreAttr(idx),DeleteAttr(idx),LoadMethod(idx).Each
idxis consistently obtained viaself.name(...)(aNameIdx), and the surrounding control flow (from-import *, chained attribute loads, delete-attribute, method calls) is unchanged. This is a straightforward enum-shape refactor with no semantic change.Also applies to: 1721-1735, 4041-4053, 4362-4366, 4742-4751
1408-1463: Control-flow opcodes with label operands correctly leverage the newLabelplumbingAll structured control flow now routes label targets through
emit!(..., Instruction::<Op>(block_idx)), relying on the newEmitArg<bytecode::Label> for ir::BlockIdx:
Jump(...)in if/elif chains, while/for loops, try/except/else/finally,match, and comprehensions,- loop control (
Break(exit_block),Continue(loop_block),ForIter(else_block)),- exception setup (
SetupExcept(handler_block),SetupFinally(finally_block)),- boolean / guard jumps (
PopJumpIfTrue/False,JumpIfFalseOrPop,JumpIfTrueOrPop).Targets are always the appropriate
BlockIdx(e.g., after/else blocks, handler blocks, loop heads/tails), and there’s no sign of miswired labels. This matches the new label-basedInstructiondesign without altering high-level control flow.Also applies to: 1553-1607, 1958-2061, 2791-2816, 2896-2965, 2999-3015, 3812-3890, 4207-4277, 4281-4303, 4990-5003
1465-1479:Raiseand comparison-related opcodes migrated to tuple-style with unchanged semanticsThe refactors:
Raise(kind)for the threeRaiseKindcases,CompareOperation(ComparisonOperator::...)in mapping/sequence patterns and chained comparisons,ContainsOp(Invert::...),IsOp(Invert::...)in comparisons and pattern checks,are all one-to-one replacements of the prior struct-style variants. The chosen
ComparisonOperatorvalues andInvertflags still align with the surrounding logic (>=for mapping-length checks,==for value patterns,==/>=in chained comparisons, etc.). No behavioural differences are introduced.Also applies to: 3421-3443, 3720-3739, 3756-3765, 3898-3973
1806-1955: Type-parameter and generic handling matches the new intrinsic and function-attribute opcodesIn the type-params pipeline:
compile_type_param_bound_or_defaultusesUnpackSequence(1)only when allowed and wraps evaluation in a tiny function called viaCallFunctionPositional(0), as before.compile_type_paramsnow emitsCallIntrinsic1(TypeVar/ParamSpec/TypeVarTuple)andCallIntrinsic2(TypeVarWithBound/Constraint/SetTypeparamDefault)with tuple-style args, plusCopyItem(1)+store_nameto bind each type param. The counts passed toBuildTuple(...)are still derived fromtype_params.len().For generic functions/classes:
compile_function_defandcompile_class_defuseSwap(2)/Swap(num_typeparam_args + 1)andCallIntrinsic2(SetFunctionTypeParams)to attach type params, and laterCallFunctionPositional(num_typeparam_args)to invoke the wrapper closure.- In class compilation,
CallIntrinsic1(SubscriptGeneric)and the subsequentCallFunctionKeyword/Positionalfor__build_class__are updated mechanically to tuple-style with the same argument counts.Across these sites the stack comments and arg counts still line up with the design; I don’t see any regressions in how generics are threaded through.
Also applies to: 2550-2640, 2656-2760
1958-2061: Try/except/finally, async-with, async-for, and async comprehensions remain structurally equivalentThe conversions to:
SetupFinally(finally_block),SetupExcept(handler_block/after_block),Jump(else_block/finally_block),Raise(Reraise)incompile_try_statement,SetupAsyncWith(final_block)/SetupWith(final_block)plusResume(AfterAwait as u32)incompile_with,- async-for’s
SetupExcept(else_block),Resume(AfterAwait as u32), andEndAsyncFor,- async comprehensions’
SetupExcept(after_block),Resume(AfterAwait as u32), andEndAsyncFor,all keep the same block structure and stack behaviour used previously, just through tuple-style variants. Exception handling, finally unwinding, and async control flow should behave identically.
Also applies to: 2818-2894, 2896-2965, 4990-5003
3096-3570: Pattern-matching opcodes (Unpack*,Match*, mapping/sequence/or-pattern helpers) are consistently updatedIn the structural pattern-matching helpers you’ve migrated to tuple-style opcodes:
UnpackEx(args)/UnpackSequence(n)in sequence unpacking and star patterns,MatchClass(nargs),MatchMapping,MatchSequence,- mapping patterns’
BuildTuple(size)for keys,BuildMap(0),DictUpdate(2),UnpackSequence(size), and theCopyItem/Swapchoreography for deleting keys intoDeleteSubscript,BinaryOperation(BinaryOperator::Subtract)for index arithmetic,- various
CopyItem/Swapcalls in or-pattern store alignment and sequence subscripting.The updated instructions mirror the CPython algorithm and the prior Rust implementation; counts are still derived from
elts.len(),nargs + n_attrs, orsizechecks that guard against overflow. I don’t see off-by-one issues or stack mismanagement introduced by the tuple-style transition here.Also applies to: 3680-3769
3822-3890:matchcase dispatch and guards use the new jump opcodes correctlyFor
matchstatements:
- Each non-default case now uses
CopyItem(1)to duplicate the subject (except the last checked case), compiles the pattern, then jumps withJump(end)after the case body.- Guards use
PopJumpIfFalse(pattern_context.fail_pop[0])to fall back to the failure label, and you still callemit_and_reset_fail_popbetween cases.- The final default case, when present, uses
JumpIfFalseOrPop(end)for its guard.The tuple-style
Jump/PopJumpIfFalse/JumpIfFalseOrPopopcodes target the same blocks as before, and the subject clean-up (Pop) and store binding logic remain intact. Control flow through cases is unchanged.
3898-3973: Chained comparisons and boolean short-circuiting preserve previous control-flow structureFor chained comparisons:
CompareOperation(Equal/Less/LessOrEqual/Greater/GreaterOrEqual)is now emitted via the tuple-style variant.- The
Swap(2)+CopyItem(2)pattern to retain the RHS for the next comparison, andJumpIfFalseOrPop(break_block)for early exit, are unchanged in logic.For boolean expressions:
compile_jump_if’s fallback path now usesPopJumpIfTrue/False(target_block),compile_bool_opusesJumpIfFalseOrPop/JumpIfTrueOrPopwith the same block layout (after_block).Only the constructor syntax changed; the jump conditions, labels, and stack effects remain as before.
Also applies to: 4207-4277, 4281-4303
4350-4366: Expression-level opcodes (unary, slice, yield/await, comprehensions, f-strings) are refactored mechanicallyKey expression sites now use tuple-style variants:
UnaryOperation(op)after unary exprs, withopunchanged.BuildSlice(step)in slice expressions (bool step flag).YieldValue+Resume(AfterYield/AfterYieldFrom)andGetAwaitable+YieldFrom+Resume(AfterAwait)foryield,yield from,await.- Comprehensions’ initializers
BuildList/BuildSet/BuildMap(OpArgMarker::marker())and per-element opsListAppend(len),SetAdd(len),MapAdd(len)incompile_comprehension.- f-strings using
FormatValue(conversion)andBuildString(count)for both nested and top-level formatting.Given the counts and enums are unchanged, these are straightforward enum-shape updates without semantic changes.
Also applies to: 4406-4407, 4417-4454, 4538-4584, 4593-4612, 5340-5341, 5436-5448
4720-4834: Call and keyword handling (CallFunction*/CallMethod*/BuildMap*) align with the new APIIn the call path:
compile_callnow usesLoadMethod(idx)for method calls, then dispatches throughcompile_method_callvscompile_normal_call.compile_normal_callandcompile_method_callcleanly mapCallType::{Positional,Keyword,Ex}toCallFunctionPositional/Keyword/Exand their method counterparts.compile_call_innerusesBuildTupleFromTuples(size)vsBuildTuple(size)for positional args, andCallFunctionKeyword(count)when keyword names are packed into a const tuple.compile_keywordsemitsBuildMap(sub_size)for each group of named keywords andBuildMapForCall(size)when multiple mapping segments are to be merged.All argument counts are derived from
arguments.len(),size, or keyword group sizes as before, so the switch to tuple-style opcodes should not affect call semantics.
5130-5136: Const-return peephole still works with marker-basedLoadConst/ReturnConst
emit_load_const/emit_return_constnow createLoadConst(idx)/ReturnConst(idx)usingemit_arg, andemit_return_valuerewrites a terminalLoadConst(idx)intoReturnConst(idx)by pattern-matching on the new tuple-style variant.Because the
OpArgpayload is untouched when you mutateinst.instr, the encoded constant index is preserved. This keeps the peephole optimization intact under the newInstructionrepresentation.Also applies to: 5140-5143
crates/vm/src/frame.rs (2)
882-885: TODO placeholder always returnstrue.The
gen_is_coroutineclosure always returnstrue, which may not correctly identify whether a generator is actually a coroutine. This could affect the behavior of async iteration. The TODO is acknowledged, but consider adding an issue reference to track this.
533-1543: Theexecute_instructionmatch arms are correctly updated.The transition to tuple-style instruction variants with
.get(arg)accessor pattern is consistently applied throughout the execution logic. The alphabetical ordering of match arms aligns with the bytecode definition (with the noted exception ofEnterFinally/ExtendedArgorder in bytecode.rs).crates/compiler-core/src/bytecode.rs (1)
551-691: Tuple-style variant refactoring looks correct.The conversion from named-field variants (e.g.,
Foo { index: Arg<T> }) to tuple-style variants (e.g.,Foo(Arg<T>)) is consistently applied. TheLAST_INSTRUCTIONsentinel is correctly updated toYieldValue(the last variant alphabetically after fixing theEnterFinally/ExtendedArgorder).
533911c to
d5abcdb
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/compiler-core/src/bytecode.rs (1)
1316-1326: Critical: Pattern matching not updated for tuple-style variants.The
unconditional_branch()method still uses struct-style patterns (Jump { .. }) but theInstructionenum variants were changed to tuple-style (Jump(Arg<Label>)). This mismatch will cause a compilation error.Apply this diff to fix the pattern matching:
pub const fn unconditional_branch(&self) -> bool { matches!( self, - Jump { .. } - | Continue { .. } - | Break { .. } + Jump(..) + | Continue(..) + | Break(..) | ReturnValue - | ReturnConst { .. } - | Raise { .. } + | ReturnConst(..) + | Raise(..) ) }
📜 Review details
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
crates/codegen/src/compile.rs(114 hunks)crates/compiler-core/src/bytecode.rs(5 hunks)crates/jit/src/instructions.rs(4 hunks)crates/jit/tests/common.rs(4 hunks)crates/vm/src/frame.rs(8 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.rs: Follow the default rustfmt code style in Rust code (cargo fmtto format)
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by your changes
Follow Rust best practices for error handling and memory management
Files:
crates/jit/src/instructions.rscrates/jit/tests/common.rscrates/codegen/src/compile.rscrates/vm/src/frame.rscrates/compiler-core/src/bytecode.rs
**/src/**/*.rs
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use the macro system (
pyclass,pymodule,pyfunction, etc.) when implementing Python functionality in Rust
Files:
crates/jit/src/instructions.rscrates/codegen/src/compile.rscrates/vm/src/frame.rscrates/compiler-core/src/bytecode.rs
🧬 Code graph analysis (2)
crates/codegen/src/compile.rs (1)
crates/compiler-core/src/bytecode.rs (1)
marker(405-407)
crates/compiler-core/src/bytecode.rs (2)
crates/vm/src/frame.rs (2)
arg(2449-2450)jump(2002-2006)crates/codegen/src/ir.rs (1)
idx(35-37)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: Run tests under miri
- GitHub Check: Run snippets and cpython tests on wasm-wasi
- GitHub Check: Check Rust code with clippy
- GitHub Check: Check the WASM package and demo
- GitHub Check: Run snippets and cpython tests (macos-latest)
- GitHub Check: Run snippets and cpython tests (windows-latest)
- GitHub Check: Run snippets and cpython tests (ubuntu-latest)
- GitHub Check: Run rust tests (ubuntu-latest)
- GitHub Check: Run rust tests (windows-latest)
- GitHub Check: Run rust tests (macos-latest)
- GitHub Check: Ensure compilation on various targets
🔇 Additional comments (13)
crates/jit/tests/common.rs (1)
95-173: StackMachine opcode updates align with new compiler emissionThe new handlers for
BuildMap,ExtendedArg,LoadConst,LoadNameAny,ReturnConst,ReturnValue,SetFunctionAttribute,StoreAttr, andStoreLocalare consistent with howcompile.rsnow emits bytecode (especially for annotations, function creation, and locals). Stack shapes for annotations (LoadNameAny+BuildMap+SetFunctionAttribute(ANNOTATIONS)) and for storing functions intolocalsviaStoreLocallook correct for thejit_function!test harness.No blocking issues here from a JIT test perspective.
crates/jit/src/instructions.rs (4)
198-208: Unreachable-code tracking correctly extended toReturnConstMarking
in_unreachable_code = truefor bothInstruction::ReturnValueandInstruction::ReturnConst(_)ensures no further IR is emitted in the same block after any return. This matches howReturnConstnow behaves and avoids malformed Cranelift blocks.
428-480:CompareOperationhandling is type‑sound for int/bool/float combinationsThe new
Instruction::CompareOperationarm correctly:
- Pops
bthena(rhs, lhs) and inspects theirJitType.- Widens
BooltoI64before integer comparisons, treating bools as 0/1.- Maps all
ComparisonOperatorvariants to the appropriateIntCC/FloatCC.- Rejects unsupported type pairs with
JitCompileError::NotSupported.This keeps the JIT’s comparison semantics aligned with Python’s for ints, bools, and floats.
514-548: Branch lowering forPopJumpIfFalse/Truematches bytecode semanticsThe new
PopJumpIfFalse/PopJumpIfTruearms:
- Pop the condition, coerce it via
boolean_val, and create athen_blockfor the jump target and a fresh fall‑through block.- Use
brifwith the target and fall‑through blocks wired in the correct order (false → target forPopJumpIfFalse, true → target forPopJumpIfTrue), then switch the builder to the fall‑through block.This matches the CPython POP_JUMP_IF_* behavior while integrating cleanly with the label‑based block mapping.
553-618: New control‑flow and stack ops (ExtendedArg,Jump,Load*,Pop,PopBlock,Resume,ReturnConst/Value,SetupLoop,StoreFast,UnpackSequence) look consistentThe added/updated cases in
add_instructionfor:
ExtendedArg(no‑op),Jump,LoadConst/LoadFast/LoadGlobal,Pop/PopBlock,Resume(placeholder),ReturnConst/ReturnValue,SetupLoop,StoreFast,UnpackSequence,all use reasonable stack discipline and error out with
BadBytecode/NotSupportedon malformed input. In particular:
LoadGlobalonly supports the function’s own name, pushing aFuncRef, which is a clear, safe restriction.UnpackSequencechecks that the tuple length matchessize.get(arg)before extending the stack in reverse, matching Python’s UNPACK_SEQUENCE behavior. (docs.python.org)These additions are fine for the current, limited JIT surface.
crates/codegen/src/compile.rs (6)
441-461: Slice emission withBuildSlice(n == 3)matches 2/3‑arg slice semantics
compile_subscriptnow calls:let n = self.compile_slice(s)?; match ctx { ExprContext::Load => { emit!(self, Instruction::BuildSlice(n == 3)); emit!(self, Instruction::Subscript); } ExprContext::Store => { emit!(self, Instruction::BuildSlice(n == 3)); emit!(self, Instruction::StoreSubscript); } ... }With
compile_slicereturning 3 when a step is present and 2 otherwise, this lines up with Python 3.12’sBUILD_SLICEtaking a boolean “has step” flag rather than an explicit count, while keeping the existing stack discipline.
150-217: Function annotations andSetFunctionAttribute(ANNOTATIONS)are wired correctlyThe combination of:
visit_annotationspushing name/annotation pairs and thenemit!(self, Instruction::BuildMap(num_annotations)), andcompile_function_defOR’ingMakeFunctionFlags::ANNOTATIONSintofuncflagsand passing it intocompile_function_body/make_closure,ensures that when
make_closureruns, the annotations dict is already on the stack beneath the freshly created function, soSetFunctionAttribute(ANNOTATIONS)can consume[annotations_dict, func]and attach the annotations. This matches the new attribute‑based MAKE_FUNCTION design and the JIT test harness’ expectation that annotations live in a single map per function.
2091-2132: Default/kw‑default handling viaBuildTupleandBuildMapis consistent
compile_default_argumentsnow:
- Compiles positional defaults and wraps them in
BuildTuple(defaults.len().to_u32()), settingDEFAULTSinMakeFunctionFlags.- Compiles keyword‑only defaults into key/value pairs and wraps them in
BuildMap(kw_with_defaults.len().to_u32()), settingKW_ONLY_DEFAULTS.These opcodes match Python’s
BUILD_TUPLE/BUILD_MAP(count)semantics (tuple of defaults, dict of kwdefaults), andmake_closurelater consumes them viaSetFunctionAttribute(DEFAULTS)andSetFunctionAttribute(KW_ONLY_DEFAULTS)in the right stack order.
5122-5140:emit_return_valuepeephole toReturnConstis safe and reduces bytecodeThe new
emit_return_value:fn emit_return_value(&mut self) { if let Some(inst) = self.current_block().instructions.last_mut() && let Instruction::LoadConst(idx) = inst.instr { inst.instr = Instruction::ReturnConst(idx); return; } emit!(self, Instruction::ReturnValue) }reuses the existing
OpArgpayload and only changes the opcode variant, so the op‑arg encoding stays correct while shrinking aLOAD_CONST; RETURN_VALUEpair into a singleRETURN_CONST. This matches the newRETURN_CONSTopcode and is a straightforward win.
4303-4314: Dict literal emission (BuildMap+DictUpdate) matches CPython 3.9+
compile_dictnow:
- Emits key/value pairs for non‑unpacked entries followed by
BuildMap(size).- For each
**mappingentry, evaluates the mapping expression then emitsDictUpdate(1).This mirrors CPython’s
BUILD_MAP(count)followed byDICT_UPDATE(i)semantics for unpacked mappings, withsizeas the number of packed key/value pairs. (docs.python.org)
3033-3045: Based on my verification through web searches, I can confirm key claims from the review comment:✓ Verified:
- CPython's
SWAP(i)does use 1-based indexing:SWAP(i)swapsSTACK[-i]withSTACK[-1]SWAP(1)is indeed a no-op (swaps TOS with itself)- RustPython implements
SWAPmatching CPython's semantics✗ Cannot Verify (Repository access unavailable):
- The claimed off-by-one in the JIT implementation
- Whether
pattern_helper_rotateactually works correctly with RustPython's implementation ofInstruction::Swap- The actual behavior of
Instruction::CopyItemand its semantic alignment withCOPY(i)
The review comment's core assumption about CPython-style 1-based depth semantics is correct; however, verification of the specific JIT off-by-one and correctness of
pattern_helper_rotaterequires repository access.The reviewer's request to confirm that
Instruction::SwapandInstruction::CopyItemuse CPython's 1-based depth semantics is reasonable and well-founded, given that RustPython is documented to follow CPython's semantics. The suggestion to verify this alignment is sound, as semantic inconsistencies between VM components would indeed cause bugs. However, without access to the RustPython repository to inspect the actual JIT implementation and the claimed off-by-one error, the verification cannot be completed.crates/vm/src/frame.rs (1)
534-1543: LGTM! Mechanical conversion is correct and complete.All instruction handlers have been correctly updated from struct-style to tuple-style syntax. The pattern is consistent throughout: each instruction variant uses
Foo(marker)followed bymarker.get(arg)to extract the actual argument value. The unsafe operations are appropriately documented and follow the established "trust compiler" pattern.crates/compiler-core/src/bytecode.rs (1)
552-691: Enum refactoring looks good overall.The
Instructionenum variants have been successfully:
- Converted from struct-style to tuple-style (e.g.,
Foo { index: Arg<T> }→Foo(Arg<T>))- Alphabetically sorted as intended
- Properly reflected in
LAST_INSTRUCTIONconstantThe conversion is complete and correct for the enum definition itself.
| Instruction::StoreFast(idx) => { | ||
| let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?; | ||
| self.store_variable(idx.get(arg), val) | ||
| } | ||
| Instruction::Swap(index) => { | ||
| let len = self.stack.len(); | ||
| let i = len - 1; | ||
| let j = len - 1 - index.get(arg) as usize; | ||
| self.stack.swap(i, j); | ||
| Ok(()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🏁 Script executed:
# First, let's examine the current Swap implementation and related code
cat -n crates/jit/src/instructions.rs | sed -n '565,580p'Repository: RustPython/RustPython
Length of output: 186
🏁 Script executed:
# Search for compile.rs to see how SWAP is emitted
fd -t f "compile.rs" --exec cat -n {} +Repository: RustPython/RustPython
Length of output: 186
🏁 Script executed:
# Search for tests related to Swap instruction
rg -n "Swap" crates/jit/src --type=rs -A 3 -B 3Repository: RustPython/RustPython
Length of output: 186
🏁 Script executed:
# Look for any test files that exercise SWAP patterns
fd -t f -e rs --path "*/test*" | xargs rg -l "Swap\|SWAP" 2>/dev/null | head -20Repository: RustPython/RustPython
Length of output: 186
🏁 Script executed:
# Check for any bytecode or stack trace comments explaining SWAP
rg -n "STACK\[" crates/jit/src --type=rs -A 2 -B 2Repository: RustPython/RustPython
Length of output: 186
🌐 Web query:
CPython 3.12 SWAP bytecode instruction semantics documentation
💡 Result:
SWAP(i) in CPython 3.12 swaps the top-of-stack with the i‑th-from-top element:
STACK[-i], STACK[-1] = STACK[-1], STACK[-i]. [1][2]
(Opcode documented in the Python dis module; added in 3.11 and present in 3.12.) [1][2]
Sources:
[1] Python 3.12 dis — Disassembler for Python bytecode. [2] Python documentation (dis) (same opcode entry).
🌐 Web query:
Python dis SWAP instruction stack operations 1-based depth TOS
💡 Result:
SWAP(i) (added in CPython 3.11) swaps the top-of-stack (TOS) with the i‑th element below it, where i is 1‑based. Formally:
STACK[-i], STACK[-1] = STACK[-1], STACK[-i]. [1][2]
Example: stack = [A, B, C, D] (TOS = D)
- SWAP(1) leaves it [A, B, C, D] (swap TOS with itself)
- SWAP(2) → [A, B, D, C] (swap D and C)
- SWAP(4) → [D, B, C, A] (swap D and A). [1][2]
Sources: Python dis module documentation. [1][2]
🌐 Web query:
CPython 3.12 augmented assignment s[a] += b bytecode SWAP pattern
💡 Result:
Short answer: CPython 3.12 uses two SWAPs to reorder the stack so STORE_SUBSCR sees (key, container, value) as it expects after the in-place BINARY_OP.
Why: BINARY_SUBSCR (earlier) leaves the stack as: ..., s, a, s[a]. After loading b and executing BINARY_OP (+=) the stack is ..., s, a, (s[a] + b). STORE_SUBSCR implements
key = STACK.pop()
container = STACK.pop()
value = STACK.pop()
container[key] = value
so the top three stack items must be (value, container, key) before STORE_SUBSCR runs. The two SWAP ops move the computed value down into the right position (first SWAP 3 moves the value below s and a; SWAP 2 then places key and container in the order STORE_SUBSCR expects). See the dis/bytecode semantics and an example disassembly of s[a] += b. [1][2]
Sources:
- dis docs (BINARY_SUBSCR, BINARY_OP, STORE_SUBSCR semantics). [1]
- Example disassembly and stack-walk explanation for s[a] += b showing the COPY/BINARY_SUBSCR → BINARY_OP → SWAP, SWAP → STORE_SUBSCR sequence. [2]
Swap instruction has a confirmed off‑by‑one error in JIT compilation
The web search confirms that CPython 3.12's SWAP(i) bytecode uses 1-based indexing and swaps STACK[-i] with STACK[-1] (the top of stack). For example, with stack [A, B, C, D] (TOS = D), SWAP(2) produces [A, B, D, C] by swapping D and C.
Tracing through the current implementation:
- For
SWAP(2)on a stack of length 4:i = 3,j = 4 - 1 - 2 = 1→ swapsstack[3]withstack[1](D with B), yielding[A, D, C, B]✗ - Correct formula should be:
j = len - index.get(arg) as usize→j = 4 - 2 = 2→ swapsstack[3]withstack[2](D with C), yielding[A, B, D, C]✓
This off-by-one error will cause incorrect stack reordering for any JIT-compiled code using Swap, including augmented assignment patterns like s[a] += b which use SWAP 3; SWAP 2 in CPython 3.12+.
Recommended fix:
- let j = len - 1 - index.get(arg) as usize;
+ let j = len - index.get(arg) as usize;Add a unit test exercising SWAP 2 and SWAP 3 patterns to lock in correct semantics.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| Instruction::StoreFast(idx) => { | |
| let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?; | |
| self.store_variable(idx.get(arg), val) | |
| } | |
| Instruction::Swap(index) => { | |
| let len = self.stack.len(); | |
| let i = len - 1; | |
| let j = len - 1 - index.get(arg) as usize; | |
| self.stack.swap(i, j); | |
| Ok(()) | |
| Instruction::StoreFast(idx) => { | |
| let val = self.stack.pop().ok_or(JitCompileError::BadBytecode)?; | |
| self.store_variable(idx.get(arg), val) | |
| } | |
| Instruction::Swap(index) => { | |
| let len = self.stack.len(); | |
| let i = len - 1; | |
| let j = len - index.get(arg) as usize; | |
| self.stack.swap(i, j); | |
| Ok(()) |
🤖 Prompt for AI Agents
In crates/jit/src/instructions.rs around lines 568 to 577, the SWAP
implementation uses an off-by-one index calculation (j = len - 1 -
index.get(arg) as usize) which incorrectly treats the operand as 0-based; change
the calculation to use 1-based semantics (j = len - index.get(arg) as usize) so
SWAP(i) swaps STACK[-i] with STACK[-1], and update any comments accordingly;
also add unit tests exercising SWAP 2 and SWAP 3 (and combined patterns like
SWAP 3; SWAP 2 used in augmented assignment) to validate correct stack
reordering.
crates/compiler-core/src/bytecode.rs
Outdated
| SetupLoop, | ||
|
|
||
| SetupAsyncWith(Arg<Label>), | ||
| SetupExcept(Arg<Label>), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how about at least keeping the name as comment?
e.g.
| SetupExcept(Arg<Label>), | |
| SetupExcept(Arg<Label>), // handler |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I can do that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you also tell me about the motivation to remove the label? They compiled to the same bytecode, but having label is easier to read the code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The motivation is the inconsistenty between opcodes that holds the same OpArgType, some holds it as an associated value and some as a named field, which is a bit confusing imo. also, I find it very tedious and just more boilerplate for having different an enum that it's variants hold nothing or a single named field.
I'm not so good with words so here's what I mean:
I can justify having enums like this where different variants hold different fields, even if some overlapping:
enum Foo {
A { name: String, age: u8 },
B { name: String, job: String, married: bool },
...
}But I do find it redundant and overly verbose to have something like this:
enum Foo {
A { name: String },
B { name: String },
C { name: String },
...
}Not to mention that we have cases like this:
enum Foo {
A { target: Label },
B { end: Label },
C { handler: Label },
...
}Because Label is a newtype for u32 already, it's just confusing as to why the same newtype is being "treated" differently. I'd much rather to go with newtyprs all the way like:
enum Foo {
A(TargetLabel),
B(EndLabel),
C(HandlerLabel),
...
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@youknowone I can revert that change if that's a deal breaker for you:)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the new enum idea on #6313. Not sure about Label. Because all the labels are actually label.
If the consistency matters, giving label for every instruction looks better to me. It feels like redundancy while writing code, but we read code more than writing code. Getting information from code is easier than checking document.
The situation is also a bit different to your example. It is more like:
enum Foo {
CallMethodPositional { nargs: String },
ReturnConst { idx: String },
...
}Maybe giving different types for them will also make sense though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the new enum idea on #6313. Not sure about Label. Because all the labels are actually label.
If the consistency matters, giving label for every instruction looks better to me. It feels like redundancy while writing code, but we read code more than writing code. Getting information from code is easier than checking document.
IMO consistency matters, I can make them consistent in the other way around by having each Oparg to be stored as a named field if that's desired.
The situation is also a bit different to your example. It is more like:
enum Foo { CallMethodPositional { nargs: String }, ReturnConst { idx: String }, ... }Maybe giving different types for them will also make sense though.
I think that it will, but that is out of scope for this PR, I'll do it in a separate one
One of the changes made is to sort all variants alphabetically, this will makes it easier for finding if an instruction exist, especially when comparing it to https://github.com/python/cpython/blob/cfcd52490d6531f3b0e8ddd4bb2b40fb6baee854/Include/opcode_ids.h (which is sorted alphabetically).
The second change I did was to change each named field to an associated value because we are only holding the oparg type.
For example:
to:
IMO this has the following benefits:
Instructiononly holds the oparg type not the actual value.Instructioneasier to auto-generate (what Opcode metadata auto generation #6174 tried to do).The reason why we had named fields was because in the first iterations of the
Instructionenum we held the actual oparg value as a named field inside the Instruction, which is no longer the case.IMO this enum should be renamed to
Opcodeand have a method calledopargthat looks like:but that's a different discussion
Summary by CodeRabbit
Refactor
New Features
Bug Fixes
✏️ Tip: You can customize this high-level summary in your review settings.