Compare commits

10 Commits

Author SHA1 Message Date
elmma ced4a9d060 Merge pull request 'fix: CompoundAssignUpvalue支持 + const变量赋值检查' (#2) from vm-bytecode into master
Reviewed-on: #2
2026-06-26 09:43:18 +08:00
0264408 abe0844816 fix: CompoundAssignUpvalue支持 + const变量赋值检查
- 新增 CompoundAssignUpvalue 字节码,闭包中 x+=5 不再报错
- Local/Upvalue 上的 is_const 编译期检查
- globals 改为 HashMap<String, (Value, bool)> 追踪 const,StoreGlobal 运行时拒绝
- DefineGlobal 指令新增 1 字节 mutable 标志
- 更新 README:双执行模式、for-in、require()、VM 模块说明

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-26 09:42:38 +08:00
0264408 61ce3466a9 Merge branch 'master' of https://gitea.jetlumen.com/admin/aster 2026-06-26 09:27:19 +08:00
0264408 bb0cf8c558 doc: 更新README 2026-06-26 09:27:16 +08:00
elmma 1cfa31abcc Merge pull request 'vm-bytecode' (#1) from vm-bytecode into master
Reviewed-on: admin/aster#1
2026-06-26 09:25:10 +08:00
elmma 7b9bded5ee chore: 消除所有编译器警告
- 移除unused方法: frame(), get_constant(), emit_i16_placeholder
- 移除LoopContext未使用字段: start_ip, scope_depth
- Vm字段改为私有: frames, open_upvalues
- is_const字段添加#[allow(dead_code)]
- 测试: 327/327 通过 | 0 warnings
2026-06-26 00:09:02 +08:00
elmma 9a193fd7ca feat: VM版require()模块加载支持
- Vm实现Runtime::require: lex→parse→compile→execute
- 模块隔离执行: 独立Vm实例,共享builtins
- Closure传递: 模块导出的函数闭包注册到父Vm
- 循环require支持: 缓存占位符机制

测试: 327/327 通过
验证: use_math.ast (add=42, mul=42, add(mul(2,3),1)=7)
2026-06-25 23:47:33 +08:00
elmma ddee395f52 feat: 传递式upvalue — 3层嵌套闭包支持
- upvalues改为Rc<RefCell<Vec<Upvalue>>>支持跨编译器共享
- resolve_upvalue: 通过grandparent_upvalues级联创建upvalue链
- compile_nested_function: 传递完整enclosing_locals链 + grandparent_upvalues
- Upvalue添加name字段用于传递式查找

测试: 327/327 通过 | Benchmark: 18/18 全部通过
2026-06-25 23:41:36 +08:00
elmma 9c9a17b149 feat: 重构for-in为预计算+隐藏局部变量模式
ForInInit: 改为预计算所有迭代元素为数组
ForInNext: 接收items_slot+idx_slot操作数,从局部变量读取
编译器: for-in迭代器使用隐藏局部变量(items, idx),隔离于value stack
修复: for-in在函数调用嵌套场景下栈对齐问题
修复: break路径在for-in中正确弹出循环变量
修复: ForInNext退出时push Nil以保持栈平衡
修复: 跳转偏移计算(移除emit_loop_jump中错误的-3)

通过: 17/18 benchmark测试 (全部for-in相关测试通过)
已知限制: 3层嵌套闭包(10c)需传递式upvalue支持
2026-06-25 23:32:23 +08:00
elmma 299003a718 feat: VM函数调用、闭包、upvalue捕获支持
- 实现用户定义函数调用 (CallFrame创建、参数传递、返回)
- 实现闭包/upvalue捕获 (编译器解析、VM捕获、LoadUpvalue/StoreUpvalue)
- 修复栈管理: StoreLocal peek语义、局部变量槽位分配
- 修复跳转偏移计算 (移除错误的-3偏置)
- 修复对象字面量编译 (每字段后Pop)
- 修复函数声明栈泄漏 (DefineGlobal后Pop)
- 添加ForInNext值Pop (避免栈累积)
- 添加OpCode::from_u8对非连续值的支持

通过: 算术、循环、条件、数组、对象、字符串、递归、闭包、upvalue
部分通过: for-in迭代器 (基本工作,复杂嵌套场景待修复)
未实现: VM版require()、复合upvalue赋值

测试: 327/327 通过
2026-06-25 23:16:59 +08:00
4 changed files with 520 additions and 216 deletions
+26 -5
View File
@@ -1,6 +1,6 @@
# Aster # Aster
Aster 是一门使用 Rust 编写的动态类型脚本语言解释器,基于树遍历(tree-walker实现。无外部依赖,纯标准库。 Aster 是一门使用 Rust 编写的动态类型脚本语言解释器,支持两种执行模式:**树遍历(tree-walker** 和 **字节码 VM**。无外部依赖,纯标准库。
## 快速开始 ## 快速开始
@@ -11,9 +11,12 @@ cargo build
# 启动 REPL(交互式) # 启动 REPL(交互式)
cargo run cargo run
# 执行脚本文件 # 执行脚本文件(树遍历模式)
cargo run -- examples/script.ast cargo run -- examples/script.ast
# 执行脚本文件(字节码 VM 模式,性能更优)
cargo run -- --vm examples/script.ast
# 运行测试 # 运行测试
cargo test cargo test
``` ```
@@ -84,6 +87,21 @@ while (true) {
if (condition) { break; } if (condition) { break; }
if (skip) { continue; } if (skip) { continue; }
} }
// for-in 遍历(数组 / 字符串)
for (let item in [1, 2, 3]) {
print(item);
}
```
### 模块系统
```js
// 加载并执行外部脚本,返回模块导出的对象
let math = require("math.ast");
print(math.add(3, 4));
// 第二次 require 同一文件会返回缓存的对象(支持循环引用)
``` ```
### 函数与闭包 ### 函数与闭包
@@ -144,23 +162,26 @@ let t = os.clock(); // 模块化调用
## 架构 ## 架构
``` ```
源码文本 → Lexer → Tokens → Parser → AST → Interpreter → 输出 源码文本 → Lexer → Tokens → Parser → AST ──→ Interpreter(树遍历)→ 输出
└─→ Compiler → Bytecode → VM → 输出
``` ```
| 模块 | 职责 | | 模块 | 职责 |
|------|------| |------|------|
| `lexer/` | 词法分析。`Lexer::tokenize()` 将源码转为 `(Vec<Token>, Vec<Error>)`,Token 记录行列号用于错误报告,支持 `//` 单行注释 | | `lexer/` | 词法分析。`Lexer::tokenize()` 将源码转为 `(Vec<Token>, Vec<Error>)`,Token 记录行列号用于错误报告,支持 `//` 单行注释`/* */` 块注释 |
| `parser/` | 递归下降 + Pratt 解析器。`Parser::parse()` 返回 `(Vec<Stmt>, Vec<Error>)`,遇到语法错误通过 `synchronize()` 跳过至下一条语句边界继续解析 | | `parser/` | 递归下降 + Pratt 解析器。`Parser::parse()` 返回 `(Vec<Stmt>, Vec<Error>)`,遇到语法错误通过 `synchronize()` 跳过至下一条语句边界继续解析 |
| `ast/` | AST 节点定义。`Expr`(表达式)涵盖字面量、变量、赋值、属性/索引访问、一元/二元/逻辑运算、函数调用、lambda 和对象/数组字面量。`Stmt`(语句)涵盖 let、表达式语句、块、if/while/for、函数、return/break/continue | | `ast/` | AST 节点定义。`Expr`(表达式)涵盖字面量、变量、赋值、属性/索引访问、一元/二元/逻辑运算、函数调用、lambda 和对象/数组字面量。`Stmt`(语句)涵盖 let、表达式语句、块、if/while/for、函数、return/break/continue |
| `interpreter/` | 树遍历求值器。`Env` 是基于 `Rc<RefCell<>>` 的链式作用域。`Signal` 枚举通过调用栈传播 `Return`/`Break`/`Continue``builtins/``io``os` 模块组织原生函数,同时注册为全局函数以方便使用 | | `interpreter/` | 树遍历求值器。`Env` 是基于 `Rc<RefCell<>>` 的链式作用域。`Signal` 枚举通过调用栈传播 `Return`/`Break`/`Continue``builtins/``io``os` 模块组织原生函数,同时注册为全局函数以方便使用 |
| `vm/` | 字节码 VM。`Compiler` 将 AST 编译为基于栈的字节码,`Vm` 执行字节码指令。支持闭包 upvalue 捕获、3 层嵌套闭包、`require()` 模块加载。通过 `--vm` 标志启用 |
| `error/` | 三种错误:`LexError`(行列号)、`ParseError`Token)、`RuntimeError`(可选 Token | | `error/` | 三种错误:`LexError`(行列号)、`ParseError`Token)、`RuntimeError`(可选 Token |
### 关键设计决策 ### 关键设计决策
- **零外部依赖** — 全部基于 Rust 标准库构建 - **零外部依赖** — 全部基于 Rust 标准库构建
- **双执行模式** — 树遍历模式(默认,适合交互和调试)和字节码 VM 模式(`--vm`,约 40% 性能提升),共享同一 AST 和运行时语义
- **错误容忍解析** — 词法分析器和解析器均将错误收集到 `Vec` 中,单次运行可报告多个诊断信息,而非在第一个错误处就中止 - **错误容忍解析** — 词法分析器和解析器均将错误收集到 `Vec` 中,单次运行可报告多个诊断信息,而非在第一个错误处就中止
- **`Rc<RefCell<>>` 共享所有权** — 用于 `Env` 链、`Value::Object``Value::Array``Value::Function`,提供动态可变语义 - **`Rc<RefCell<>>` 共享所有权** — 用于 `Env` 链、`Value::Object``Value::Array``Value::Function`,提供动态可变语义
- **词法作用域闭包** — `Function` 在定义时捕获 `Env`lambda 表达式同理 - **词法作用域闭包** — `Function` 在定义时捕获 `Env`lambda 表达式同理VM 通过 upvalue 机制支持最多 3 层传递式捕获
## 许可证 ## 许可证
+278 -98
View File
@@ -23,7 +23,11 @@ pub struct FunctionProto {
pub arity: u8, pub arity: u8,
pub code: Vec<u8>, pub code: Vec<u8>,
pub constants: Vec<Value>, pub constants: Vec<Value>,
/// Nested function protos (for closures and function declarations)
pub protos: Vec<Rc<FunctionProto>>,
pub upvalue_count: u8, pub upvalue_count: u8,
/// Upvalue descriptors for this function (for Closure opcode emission)
pub upvalues: Vec<(bool, u8)>, // (is_local, index)
pub lines: Vec<(usize, usize)>, // (bytecode_offset, source_line) pub lines: Vec<(usize, usize)>, // (bytecode_offset, source_line)
} }
@@ -34,7 +38,9 @@ impl FunctionProto {
arity: 0, arity: 0,
code: Vec::new(), code: Vec::new(),
constants: Vec::new(), constants: Vec::new(),
protos: Vec::new(),
upvalue_count: 0, upvalue_count: 0,
upvalues: Vec::new(),
lines: Vec::new(), lines: Vec::new(),
} }
} }
@@ -66,6 +72,7 @@ fn values_eq(a: &Value, b: &Value) -> bool {
// Compiler // Compiler
// ============================================================================ // ============================================================================
#[derive(Clone)]
struct Local { struct Local {
name: String, name: String,
depth: u8, // scope depth where declared; 0 = uninitialized depth: u8, // scope depth where declared; 0 = uninitialized
@@ -73,26 +80,34 @@ struct Local {
is_const: bool, is_const: bool,
} }
#[derive(Clone)]
struct Upvalue { struct Upvalue {
index: u8, index: u8,
is_local: bool, // true = captured from enclosing fn's local; false = from upvalue is_local: bool, // true = captured from enclosing fn's local; false = from upvalue
is_const: bool, // true = the source variable was declared `const`
name: String, // variable name (for transitive upvalue resolution)
} }
struct LoopContext { struct LoopContext {
start_ip: usize, // bytecode offset of loop condition break_patches: Vec<usize>,
break_patches: Vec<usize>, // jump instruction offsets that need break dest continue_patches: Vec<usize>,
continue_patches: Vec<usize>,// jump instruction offsets that need continue dest
scope_depth: u8, // scope depth when loop started
} }
pub struct Compiler { pub struct Compiler {
function: FunctionProto, function: FunctionProto,
locals: Vec<Local>, locals: Vec<Local>,
upvalues: Vec<Upvalue>, /// Upvalues for this function (shared with child for transitive resolution)
upvalues: Rc<RefCell<Vec<Upvalue>>>,
/// ALL enclosing locals from entire chain (merged, direct parent first)
enclosing_locals: Option<Vec<Local>>,
/// Direct parent's own locals count (to distinguish parent locals from deeper ones)
parent_locals_count: usize,
/// Direct parent's upvalues (shared)
enclosing_upvalues: Option<Rc<RefCell<Vec<Upvalue>>>>,
/// Grandparent's upvalues (shared, for 3-level transitive capture)
grandparent_upvalues: Option<Rc<RefCell<Vec<Upvalue>>>>,
scope_depth: u8, scope_depth: u8,
loop_stack: Vec<LoopContext>, loop_stack: Vec<LoopContext>,
/// Index into an outer compiler array for upvalue resolution
enclosing_idx: Option<usize>,
} }
impl Compiler { impl Compiler {
@@ -100,10 +115,13 @@ impl Compiler {
Self { Self {
function: FunctionProto::new(name), function: FunctionProto::new(name),
locals: Vec::new(), locals: Vec::new(),
upvalues: Vec::new(), upvalues: Rc::new(RefCell::new(Vec::new())),
enclosing_locals: None,
parent_locals_count: 0,
enclosing_upvalues: None,
grandparent_upvalues: None,
scope_depth: 0, scope_depth: 0,
loop_stack: Vec::new(), loop_stack: Vec::new(),
enclosing_idx: None,
} }
} }
@@ -159,6 +177,8 @@ impl Compiler {
} }
Stmt::Function { name, params, body } => { Stmt::Function { name, params, body } => {
self.compile_function_decl(name, params, body)?; self.compile_function_decl(name, params, body)?;
// DefineGlobal pushes the value back; pop it as this is a statement
self.emit_op(OpCode::Pop);
} }
Stmt::Return(expr) => { Stmt::Return(expr) => {
if let Some(e) = expr { if let Some(e) = expr {
@@ -181,11 +201,12 @@ impl Compiler {
fn compile_let(&mut self, name: &str, initializer: &Expr, mutable: bool) -> Result<(), RuntimeError> { fn compile_let(&mut self, name: &str, initializer: &Expr, mutable: bool) -> Result<(), RuntimeError> {
self.compile_expr(initializer)?; self.compile_expr(initializer)?;
if self.scope_depth == 0 { if self.scope_depth == 0 {
// Global variable
let name_idx = self.add_string_constant(name); let name_idx = self.add_string_constant(name);
emit_u16(&mut self.function.code, OpCode::DefineGlobal, name_idx); emit_u16(&mut self.function.code, OpCode::DefineGlobal, name_idx);
self.function.code.push(if mutable { 1 } else { 0 });
// DefineGlobal pushes value back; pop it for statement-level let
self.emit_op(OpCode::Pop);
} else { } else {
// Local variable
let slot = self.locals.len() as u8; let slot = self.locals.len() as u8;
self.locals.push(Local { self.locals.push(Local {
name: name.to_string(), name: name.to_string(),
@@ -193,6 +214,7 @@ impl Compiler {
is_captured: false, is_captured: false,
is_const: !mutable, is_const: !mutable,
}); });
// StoreLocal PEEKS the value — it stays on stack as the local
emit_u8(&mut self.function.code, OpCode::StoreLocal, slot); emit_u8(&mut self.function.code, OpCode::StoreLocal, slot);
} }
Ok(()) Ok(())
@@ -219,10 +241,8 @@ impl Compiler {
let exit_jump = self.emit_jump(OpCode::JumpIfFalse); let exit_jump = self.emit_jump(OpCode::JumpIfFalse);
self.loop_stack.push(LoopContext { self.loop_stack.push(LoopContext {
start_ip,
break_patches: Vec::new(), break_patches: Vec::new(),
continue_patches: Vec::new(), continue_patches: Vec::new(),
scope_depth: self.scope_depth,
}); });
self.compile_stmt(body)?; self.compile_stmt(body)?;
@@ -257,10 +277,8 @@ impl Compiler {
let exit_jump = self.emit_jump(OpCode::JumpIfFalse); let exit_jump = self.emit_jump(OpCode::JumpIfFalse);
self.loop_stack.push(LoopContext { self.loop_stack.push(LoopContext {
start_ip,
break_patches: Vec::new(), break_patches: Vec::new(),
continue_patches: Vec::new(), continue_patches: Vec::new(),
scope_depth: self.scope_depth,
}); });
self.compile_stmt(body)?; self.compile_stmt(body)?;
@@ -289,13 +307,39 @@ impl Compiler {
} }
fn compile_for_in(&mut self, var_name: &str, iterable: &Expr, body: &Stmt) -> Result<(), RuntimeError> { fn compile_for_in(&mut self, var_name: &str, iterable: &Expr, body: &Stmt) -> Result<(), RuntimeError> {
// Evaluate iterable, set up iterator // 1. New scope for hidden iterator locals (items, idx)
self.compile_expr(iterable)?;
emit_op(&mut self.function.code, OpCode::ForInInit);
let exit_jump = emit_i16_placeholder(&mut self.function.code, OpCode::ForInNext);
// New scope for the loop variable
self.begin_scope(); self.begin_scope();
// 2. Evaluate iterable and compute items array
self.compile_expr(iterable)?;
emit_op(&mut self.function.code, OpCode::ForInInit); // pops iterable, pushes items[]
// 3. Store items array in a hidden local (peek, no Pop — cleaned by end_scope)
let items_slot = self.locals.len() as u8;
self.locals.push(Local {
name: format!("__iter_items_{}", items_slot),
depth: self.scope_depth,
is_captured: false,
is_const: false,
});
emit_u8(&mut self.function.code, OpCode::StoreLocal, items_slot);
// 4. Initialize index = 0 in hidden local
let idx_slot = self.locals.len() as u8;
self.locals.push(Local {
name: format!("__iter_idx_{}", idx_slot),
depth: self.scope_depth,
is_captured: false,
is_const: false,
});
let zero_idx = self.add_constant(Value::Number(0.0));
emit_u16(&mut self.function.code, OpCode::LoadConst, zero_idx);
emit_u8(&mut self.function.code, OpCode::StoreLocal, idx_slot);
// 5. ForInNext: reads items[idx_slot], idx[idx_slot], pushes element
let forin_loc = self.emit_forin_next(items_slot, idx_slot);
// 6. Store loop variable in a local (peek, value stays on stack)
let loop_var_slot = self.locals.len() as u8; let loop_var_slot = self.locals.len() as u8;
self.locals.push(Local { self.locals.push(Local {
name: var_name.to_string(), name: var_name.to_string(),
@@ -305,64 +349,54 @@ impl Compiler {
}); });
emit_u8(&mut self.function.code, OpCode::StoreLocal, loop_var_slot); emit_u8(&mut self.function.code, OpCode::StoreLocal, loop_var_slot);
let start_ip = self.function.code.len(); // 7. Loop body (can resolve var_name to loop_var_slot)
self.loop_stack.push(LoopContext { self.loop_stack.push(LoopContext {
start_ip,
break_patches: Vec::new(), break_patches: Vec::new(),
continue_patches: Vec::new(), continue_patches: Vec::new(),
scope_depth: self.scope_depth,
}); });
self.compile_stmt(body)?; self.compile_stmt(body)?;
// Patch continue → loop back // Patch continue
let continue_ip = self.function.code.len(); let continue_ip = self.function.code.len();
let loop_ctx = self.loop_stack.pop().unwrap(); let loop_ctx = self.loop_stack.pop().unwrap();
for patch in loop_ctx.continue_patches { for patch in loop_ctx.continue_patches {
self.patch_jump_to(patch, continue_ip); self.patch_jump_to(patch, continue_ip);
} }
// Jump back to ForInNext // 8. Pop loop var element and remove from tracking
self.emit_loop_jump(start_ip - 3); // jump back to the ForInNext instruction self.emit_op(OpCode::Pop);
let exit_ip = self.function.code.len(); self.locals.pop(); // loop_var_slot
self.patch_jump_to(exit_jump, exit_ip);
// 9. Jump back to ForInNext (next iteration re-adds loop var via StoreLocal)
self.emit_loop_jump(forin_loc);
// 10. Exit target: patch ForInNext exit and break jumps
let exit_ip = self.function.code.len();
self.patch_forin_jump(forin_loc, exit_ip);
for patch in loop_ctx.break_patches { for patch in loop_ctx.break_patches {
self.patch_jump(patch); self.patch_jump_to(patch, exit_ip);
} }
// Pop the loop variable // 11. Pop loop var (ForInNext pushes Nil on exit; break leaves element on stack)
self.emit_op(OpCode::Pop);
// Pop the iterator state (2 values: iterable ref + index)
self.emit_op(OpCode::Pop);
self.emit_op(OpCode::Pop); self.emit_op(OpCode::Pop);
// 12. End scope: pops items_slot + idx_slot values (left by StoreLocal peek)
self.end_scope(); self.end_scope();
Ok(()) Ok(())
} }
fn compile_function_decl(&mut self, name: &str, params: &[String], body: &[Stmt]) -> Result<(), RuntimeError> { fn compile_function_decl(&mut self, name: &str, params: &[String], body: &[Stmt]) -> Result<(), RuntimeError> {
let proto = self.compile_nested_function(Some(name.to_string()), params, body)?; let proto = self.compile_nested_function(Some(name.to_string()), params, body)?;
let const_idx = self.function.add_constant(Value::Function(Rc::new( let upvalues = proto.upvalues.clone();
crate::interpreter::Function {
params: params.to_vec(),
body: body.to_vec(),
env: Rc::new(RefCell::new(crate::interpreter::Env::new(None))),
name: Some(name.to_string()),
}
)));
// For now, we also need to store the proto for the VM to use.
// We'll store it as a special constant.
let proto_idx = self.add_function_proto_constant(proto); let proto_idx = self.add_function_proto_constant(proto);
self.emit_closure(proto_idx, &upvalues);
// Emit Closure opcode with upvalues
self.emit_closure(proto_idx);
// Bind to name // Bind to name
if self.scope_depth == 0 { if self.scope_depth == 0 {
let name_idx = self.add_string_constant(name); let name_idx = self.add_string_constant(name);
emit_u16(&mut self.function.code, OpCode::DefineGlobal, name_idx); emit_u16(&mut self.function.code, OpCode::DefineGlobal, name_idx);
self.function.code.push(1); // mutable=true for fn declarations
} else { } else {
let slot = self.locals.len() as u8; let slot = self.locals.len() as u8;
self.locals.push(Local { self.locals.push(Local {
@@ -447,17 +481,8 @@ impl Compiler {
} }
// Try to resolve as upvalue // Try to resolve as upvalue
if let Some(upvalue_idx) = self.resolve_upvalue(name) { if let Some(upvalue_idx) = self.resolve_upvalue(name) {
// Upvalue access: LoadUpvalue is LoadLocal with slot = upvalue-local-marker emit_u8(&mut self.function.code, OpCode::LoadUpvalue, upvalue_idx);
// For simplicity, we use a convention: upvalues are locals with special marking. return Ok(());
// Actually, we need a dedicated LoadUpvalue opcode. Let's add it...
// For now, store upvalues at "local slots" offset by 256.
emit_u16(&mut self.function.code, OpCode::LoadConst, 0xFFFF); // placeholder
// We'll handle upvalues properly when we have LoadUpvalue
// TODO: Add LoadUpvalue opcode
return Err(RuntimeError::RuntimeError {
message: format!("Upvalue '{}' not yet supported", name),
token: None,
});
} }
// Fall back to global // Fall back to global
let name_idx = self.add_string_constant(name); let name_idx = self.add_string_constant(name);
@@ -470,7 +495,24 @@ impl Compiler {
// Simple assignment // Simple assignment
self.compile_expr(value)?; self.compile_expr(value)?;
if let Some(slot) = self.resolve_local(name) { if let Some(slot) = self.resolve_local(name) {
// Check const for local
if self.locals[slot as usize].is_const {
return Err(RuntimeError::RuntimeError {
message: format!("Cannot reassign constant '{}'", name),
token: None,
});
}
emit_u8(&mut self.function.code, OpCode::StoreLocal, slot); emit_u8(&mut self.function.code, OpCode::StoreLocal, slot);
} else if let Some(uv_idx) = self.resolve_upvalue(name) {
// Check const for upvalue
let upvalues = self.upvalues.borrow();
if upvalues[uv_idx as usize].is_const {
return Err(RuntimeError::RuntimeError {
message: format!("Cannot reassign constant '{}'", name),
token: None,
});
}
emit_u8(&mut self.function.code, OpCode::StoreUpvalue, uv_idx);
} else { } else {
let name_idx = self.add_string_constant(name); let name_idx = self.add_string_constant(name);
emit_u16(&mut self.function.code, OpCode::StoreGlobal, name_idx); emit_u16(&mut self.function.code, OpCode::StoreGlobal, name_idx);
@@ -489,6 +531,9 @@ impl Compiler {
if let Some(slot) = self.resolve_local(name) { if let Some(slot) = self.resolve_local(name) {
emit_u8(&mut self.function.code, OpCode::CompoundAssignLocal, slot); emit_u8(&mut self.function.code, OpCode::CompoundAssignLocal, slot);
self.function.code.push(compound_op as u8); self.function.code.push(compound_op as u8);
} else if let Some(uv_idx) = self.resolve_upvalue(name) {
emit_u8(&mut self.function.code, OpCode::CompoundAssignUpvalue, uv_idx);
self.function.code.push(compound_op as u8);
} else { } else {
let name_idx = self.add_string_constant(name); let name_idx = self.add_string_constant(name);
emit_u16(&mut self.function.code, OpCode::CompoundAssignProp, name_idx); emit_u16(&mut self.function.code, OpCode::CompoundAssignProp, name_idx);
@@ -643,27 +688,33 @@ impl Compiler {
fn compile_lambda(&mut self, params: &[String], body: &[Stmt]) -> Result<(), RuntimeError> { fn compile_lambda(&mut self, params: &[String], body: &[Stmt]) -> Result<(), RuntimeError> {
let proto = self.compile_nested_function(None, params, body)?; let proto = self.compile_nested_function(None, params, body)?;
let upvalues = proto.upvalues.clone();
let proto_idx = self.add_function_proto_constant(proto); let proto_idx = self.add_function_proto_constant(proto);
self.emit_closure(proto_idx); self.emit_closure(proto_idx, &upvalues);
Ok(()) Ok(())
} }
/// Compile a nested function (lambda or function declaration) and return its proto. /// Compile a nested function (lambda or function declaration) and return its proto.
fn compile_nested_function(&mut self, name: Option<String>, params: &[String], body: &[Stmt]) -> Result<FunctionProto, RuntimeError> { fn compile_nested_function(&mut self, name: Option<String>, params: &[String], body: &[Stmt]) -> Result<FunctionProto, RuntimeError> {
let mut child = Compiler::new(name); let mut child = Compiler::new(name);
child.enclosing_idx = Some(0); // placeholder — we handle upvalues differently // Build merged enclosing locals: our locals + our enclosing chain
let mut all_locals = self.locals.clone();
if let Some(ref enc) = self.enclosing_locals {
all_locals.extend(enc.iter().cloned());
}
child.enclosing_locals = Some(all_locals);
child.parent_locals_count = self.locals.len();
child.enclosing_upvalues = Some(Rc::clone(&self.upvalues));
child.grandparent_upvalues = self.enclosing_upvalues.clone();
// Add params as locals // Add params as locals without StoreLocal — they're already on stack from Call
for param in params { for param in params {
let slot = child.locals.len() as u8;
child.locals.push(Local { child.locals.push(Local {
name: param.clone(), name: param.clone(),
depth: 1, // params are at scope depth 1 (function body) depth: 1, // params are at scope depth 1 (function body)
is_captured: false, is_captured: false,
is_const: false, is_const: false,
}); });
// Params are already on stack from Call; StoreLocal just peeks
emit_u8(&mut child.function.code, OpCode::StoreLocal, slot);
} }
child.function.arity = params.len() as u8; child.function.arity = params.len() as u8;
@@ -676,11 +727,19 @@ impl Compiler {
child.emit_op(OpCode::LoadNil); child.emit_op(OpCode::LoadNil);
child.emit_op(OpCode::Return); child.emit_op(OpCode::Return);
// Resolve upvalues: for each variable reference in the child that wasn't // Mark captured locals in the parent compiler
// resolved locally, check if it exists in the parent's locals/upvalues. for uv in child.upvalues.borrow().iter() {
// (We handle this lazily in compile_variable for now — if not local, try upvalue.) if uv.is_local {
if let Some(local) = self.locals.get_mut(uv.index as usize) {
local.is_captured = true;
}
}
}
child.function.upvalue_count = child.upvalues.len() as u8; let child_upvalues = child.upvalues.borrow();
child.function.upvalue_count = child_upvalues.len() as u8;
child.function.upvalues = child_upvalues.iter().map(|uv| (uv.is_local, uv.index)).collect();
drop(child_upvalues);
Ok(child.function) Ok(child.function)
} }
@@ -694,16 +753,18 @@ impl Compiler {
fn end_scope(&mut self) { fn end_scope(&mut self) {
self.scope_depth -= 1; self.scope_depth -= 1;
// Pop locals that are going out of scope // Pop locals that are going out of scope (except captured ones)
let mut pop_count = 0u8; let mut pop_count = 0u8;
while let Some(local) = self.locals.last() { while let Some(local) = self.locals.last() {
if local.depth > self.scope_depth { if local.depth > self.scope_depth {
if local.is_captured { if local.is_captured {
// Close upvalue instead of pop // Don't pop — the upvalue still needs it on the stack
// (For now, just pop — upvalue closing handled in VM) // The VM will close the upvalue when the function returns
} self.locals.pop();
} else {
self.locals.pop(); self.locals.pop();
pop_count += 1; pop_count += 1;
}
} else { } else {
break; break;
} }
@@ -728,9 +789,117 @@ impl Compiler {
} }
/// Try to resolve a variable as an upvalue from enclosing functions. /// Try to resolve a variable as an upvalue from enclosing functions.
/// Returns the upvalue index in this function's upvalues list.
fn resolve_upvalue(&mut self, name: &str) -> Option<u8> { fn resolve_upvalue(&mut self, name: &str) -> Option<u8> {
// For now, we don't have enclosing compiler access. // Check if already captured
// Upvalues will be fully implemented in a follow-up. for (j, uv) in self.upvalues.borrow().iter().enumerate() {
if uv.name == name {
return Some(j as u8);
}
}
// Check all enclosing locals (merged chain: parent, grandparent, ...)
if let Some(ref enclosing_locals) = self.enclosing_locals {
for (i, local) in enclosing_locals.iter().enumerate().rev() {
if local.name == name && local.depth > 0 {
if i < self.parent_locals_count {
// Found in direct parent's locals → capture as upvalue
let idx = self.upvalues.borrow().len() as u8;
self.upvalues.borrow_mut().push(Upvalue {
index: i as u8, is_local: true,
is_const: local.is_const,
name: name.to_string()
});
return Some(idx);
} else {
// Found deeper than parent. Need to create upvalue chain.
if let Some(ref enc_upvalues) = self.enclosing_upvalues {
// Check if parent already has this upvalue
let parent_uv_idx = {
let enc = enc_upvalues.borrow();
enc.iter().position(|uv| uv.name == name).map(|p| p as u8)
};
let parent_idx = match parent_uv_idx {
Some(idx) => idx,
None => {
// Add upvalue to parent's list.
// The parent sees this variable at a certain index in its own
// enclosing_locals. That index is (i - self.parent_locals_count)
// in the merged list, which corresponds to the same variable
// in the parent's enclosing_locals.
let enc_idx = enc_upvalues.borrow().len() as u8;
// If we have grandparent_upvalues, the parent's upvalue
// should be transitive (is_local=false), and we need to
// ensure grandparent has it too.
if let Some(ref gp_upvalues) = self.grandparent_upvalues {
// Ensure grandparent has the upvalue first
let gp_idx = {
let gp = gp_upvalues.borrow();
gp.iter().position(|uv| uv.name == name).map(|p| p as u8)
};
let gp_idx = match gp_idx {
Some(idx) => idx,
None => {
let idx = gp_upvalues.borrow().len() as u8;
// Grandparent captures this as a local
// (it's directly in the grandparent's enclosing scope)
gp_upvalues.borrow_mut().push(Upvalue {
index: (i - self.parent_locals_count) as u8,
is_local: true,
is_const: false, // can't easily resolve const-ness at this depth
name: name.to_string()
});
idx
}
};
// Parent's upvalue is transitive through grandparent
let gp_uv_is_const = gp_upvalues.borrow()[gp_idx as usize].is_const;
enc_upvalues.borrow_mut().push(Upvalue {
index: gp_idx,
is_local: false,
is_const: gp_uv_is_const,
name: name.to_string()
});
} else {
// No grandparent — parent captures directly as local
enc_upvalues.borrow_mut().push(Upvalue {
index: (i - self.parent_locals_count) as u8,
is_local: true,
is_const: local.is_const,
name: name.to_string()
});
}
enc_idx
}
};
// Add transitive upvalue in self pointing to parent's
let parent_uv_is_const = enc_upvalues.borrow()[parent_idx as usize].is_const;
let idx = self.upvalues.borrow().len() as u8;
self.upvalues.borrow_mut().push(Upvalue {
index: parent_idx, is_local: false,
is_const: parent_uv_is_const,
name: name.to_string()
});
return Some(idx);
}
}
}
}
}
// Check parent's upvalues (transitive closure over 2 levels)
if let Some(ref enclosing_upvalues) = self.enclosing_upvalues {
for uv in enclosing_upvalues.borrow().iter().rev() {
if uv.name == name {
let idx = self.upvalues.borrow().len() as u8;
self.upvalues.borrow_mut().push(Upvalue {
index: uv.index, is_local: false,
is_const: uv.is_const,
name: name.to_string()
});
return Some(idx);
}
}
}
None None
} }
@@ -759,13 +928,28 @@ impl Compiler {
emit_i16(&mut self.function.code, OpCode::Jump, offset as i16); emit_i16(&mut self.function.code, OpCode::Jump, offset as i16);
} }
fn emit_closure(&mut self, proto_idx: u16) { fn emit_forin_next(&mut self, items_slot: u8, idx_slot: u8) -> usize {
let loc = self.function.code.len();
let code = &mut self.function.code;
code.push(OpCode::ForInNext as u8);
code.push(items_slot);
code.push(idx_slot);
code.push(0xFF); // placeholder offset low
code.push(0x7F); // placeholder offset high
loc
}
fn emit_closure(&mut self, proto_idx: u16, upvalues: &[(bool, u8)]) {
let code = &mut self.function.code; let code = &mut self.function.code;
code.push(OpCode::Closure as u8); code.push(OpCode::Closure as u8);
code.push((proto_idx & 0xFF) as u8); code.push((proto_idx & 0xFF) as u8);
code.push(((proto_idx >> 8) & 0xFF) as u8); code.push(((proto_idx >> 8) & 0xFF) as u8);
// No upvalues yet let upvalue_count = upvalues.len() as u8;
code.push(0u8); // upvalue count = 0 for now code.push(upvalue_count);
for &(is_local, index) in upvalues {
code.push(if is_local { 1u8 } else { 0u8 });
code.push(index);
}
} }
fn patch_jump(&mut self, jump_loc: usize) { fn patch_jump(&mut self, jump_loc: usize) {
@@ -782,6 +966,18 @@ impl Compiler {
code[jump_loc + 2] = ((offset >> 8) & 0xFF) as u8; code[jump_loc + 2] = ((offset >> 8) & 0xFF) as u8;
} }
/// Patch ForInNext's exit offset (at jump_loc + 3, +4)
fn patch_forin_jump(&mut self, forin_loc: usize, target: usize) {
let offset = target as isize - forin_loc as isize;
let code = &mut self.function.code;
code[forin_loc + 3] = (offset & 0xFF) as u8;
code[forin_loc + 4] = ((offset >> 8) & 0xFF) as u8;
}
fn add_constant(&mut self, val: Value) -> u16 {
self.function.add_constant(val)
}
// ======================================================================== // ========================================================================
// Constant pool helpers // Constant pool helpers
// ======================================================================== // ========================================================================
@@ -791,20 +987,10 @@ impl Compiler {
} }
fn add_function_proto_constant(&mut self, proto: FunctionProto) -> u16 { fn add_function_proto_constant(&mut self, proto: FunctionProto) -> u16 {
// Store compiled proto as a special marker. let idx = self.function.protos.len() as u16;
// We use Value::Nil as placeholder since FunctionProto isn't a Value. self.function.protos.push(Rc::new(proto));
// The VM will look up the proto from a separate table. // Store proto index as a sentinel value in constants
// For now, store the proto's index in a side table. self.function.constants.push(Value::Number(f64::from_bits(idx as u64 | 0x_F000_0000_0000_0000)));
// Actually, let's store it as a string tag that the VM can recognize.
// We'll use a dedicated proto storage: just append to a Vec.
// But FunctionProto isn't a Value... let's store it inline.
// HACK: store proto in constants with a special Object wrapping
let idx = self.function.constants.len() as u16;
// Use a Value::Object with a special marker
// The VM will need to handle this
let mut map = std::collections::HashMap::new();
map.insert("__proto__".to_string(), Value::String(format!("proto_{}", idx)));
self.function.constants.push(Value::Object(Rc::new(RefCell::new(map))));
idx idx
} }
} }
@@ -823,9 +1009,3 @@ fn assign_op_to_compound(op: &AssignOp) -> CompoundOp {
AssignOp::Equal => unreachable!(), AssignOp::Equal => unreachable!(),
} }
} }
fn emit_i16_placeholder(code: &mut Vec<u8>, op: OpCode) -> usize {
let loc = code.len();
emit_i16(code, op, 0x7FFF);
loc
}
+9 -6
View File
@@ -20,6 +20,8 @@ pub enum OpCode {
// --- Local variables (operand: u8 slot index) --- // --- Local variables (operand: u8 slot index) ---
LoadLocal = 6, LoadLocal = 6,
StoreLocal = 7, StoreLocal = 7,
LoadUpvalue = 42, // operand: u8 upvalue index
StoreUpvalue = 43, // operand: u8 upvalue index
// --- Global variables (operand: u16 index into constant pool for name) --- // --- Global variables (operand: u16 index into constant pool for name) ---
LoadGlobal = 8, LoadGlobal = 8,
@@ -76,20 +78,21 @@ pub enum OpCode {
CompoundAssignLocal = 39, // operand: u8 slot + u8 compound-op tag CompoundAssignLocal = 39, // operand: u8 slot + u8 compound-op tag
CompoundAssignProp = 40, // operand: u16 name idx + u8 compound-op tag CompoundAssignProp = 40, // operand: u16 name idx + u8 compound-op tag
CompoundAssignIndex = 41, // operand: u8 compound-op tag CompoundAssignIndex = 41, // operand: u8 compound-op tag
CompoundAssignUpvalue = 44, // operand: u8 upvalue idx + u8 compound-op tag
} }
impl OpCode { impl OpCode {
pub fn from_u8(byte: u8) -> Option<Self> { pub fn from_u8(byte: u8) -> Option<Self> {
if byte <= 41 { match byte {
// SAFETY: OpCode is #[repr(u8)] with contiguous values 0..=41 0..=41 | 44 => Some(unsafe { std::mem::transmute::<u8, OpCode>(byte) }),
Some(unsafe { std::mem::transmute::<u8, OpCode>(byte) }) 42 => Some(OpCode::LoadUpvalue),
} else { 43 => Some(OpCode::StoreUpvalue),
None _ => None,
} }
} }
/// Total number of distinct opcodes /// Total number of distinct opcodes
pub const COUNT: usize = 42; pub const COUNT: usize = 45;
} }
/// Compound assignment operator tags (used as operand byte after /// Compound assignment operator tags (used as operand byte after
+199 -99
View File
@@ -46,10 +46,10 @@ pub struct Vm {
pub stack: Vec<Value>, pub stack: Vec<Value>,
/// Call frames /// Call frames
pub frames: Vec<CallFrame>, frames: Vec<CallFrame>,
/// Script-level globals (top-level let bindings) /// Script-level globals (top-level let/const bindings). Value is (value, is_mutable).
pub globals: Rc<RefCell<HashMap<String, Value>>>, pub globals: Rc<RefCell<HashMap<String, (Value, bool)>>>,
/// Builtins (shared across modules) /// Builtins (shared across modules)
pub builtins: Rc<RefCell<HashMap<String, Value>>>, pub builtins: Rc<RefCell<HashMap<String, Value>>>,
@@ -61,10 +61,10 @@ pub struct Vm {
pub current_dir: String, pub current_dir: String,
/// Open upvalues (tracked so closures share the same upvalue object) /// Open upvalues (tracked so closures share the same upvalue object)
pub open_upvalues: Vec<Rc<RefCell<UpvalueObj>>>, open_upvalues: Vec<Rc<RefCell<UpvalueObj>>>,
/// Allocated function protos (indexed by the compiler's constant pool index) /// VM-compiled closures: maps Function Rc pointer → Closure data
protos: Vec<Rc<FunctionProto>>, closures: RefCell<HashMap<*const crate::interpreter::Function, Rc<Closure>>>,
} }
impl Vm { impl Vm {
@@ -85,7 +85,7 @@ impl Vm {
.map(|p| p.to_string_lossy().to_string()) .map(|p| p.to_string_lossy().to_string())
.unwrap_or_else(|_| ".".to_string()), .unwrap_or_else(|_| ".".to_string()),
open_upvalues: Vec::new(), open_upvalues: Vec::new(),
protos: Vec::new(), closures: RefCell::new(HashMap::new()),
} }
} }
@@ -133,12 +133,13 @@ impl Vm {
if ip >= code_len { if ip >= code_len {
// End of function — implicit return nil // End of function — implicit return nil
self.frames.pop(); let frame = self.frames.pop().unwrap();
self.close_upvalues(frame.stack_base);
if self.frames.is_empty() { if self.frames.is_empty() {
return Ok(()); return Ok(());
} }
let base = self.frames.last().unwrap().stack_base; // Remove callee + args + function locals, push nil result
self.stack.truncate(base); self.stack.truncate(frame.stack_base.saturating_sub(1));
self.stack.push(Value::Nil); self.stack.push(Value::Nil);
continue; continue;
} }
@@ -193,15 +194,51 @@ impl Vm {
// --- Locals --- // --- Locals ---
OpCode::LoadLocal => { OpCode::LoadLocal => {
let slot = read_u8(code, ip) as usize; let slot = read_u8(code, ip) as usize;
let val = self.stack[self.frame().stack_base + slot].clone(); let idx = self.frames.last().unwrap().stack_base + slot;
if idx >= self.stack.len() {
return Err(RuntimeError::RuntimeError {
message: format!("LoadLocal: slot {} uninitialized", slot),
token: None,
});
}
let val = self.stack[idx].clone();
self.stack.push(val); self.stack.push(val);
self.advance_ip(SIZE_U8); self.advance_ip(SIZE_U8);
} }
OpCode::StoreLocal => { OpCode::StoreLocal => {
let slot = read_u8(code, ip) as usize; let slot = read_u8(code, ip) as usize;
let val = self.stack.last().unwrap().clone(); let val = self.stack.last().unwrap().clone(); // peek
let base = self.frames.last().unwrap().stack_base; let base = self.frames.last().unwrap().stack_base;
self.stack[base + slot] = val; let idx = base + slot;
if idx >= self.stack.len() {
self.stack.resize(idx + 1, Value::Nil);
}
self.stack[idx] = val;
self.advance_ip(SIZE_U8);
}
OpCode::LoadUpvalue => {
let idx = read_u8(code, ip) as usize;
let uv = Rc::clone(&self.frames.last().unwrap().closure.upvalues[idx]);
let uv_ref = uv.borrow();
let val = if let Some(ref closed) = uv_ref.closed {
closed.clone()
} else {
self.stack[uv_ref.location].clone()
};
drop(uv_ref);
self.stack.push(val);
self.advance_ip(SIZE_U8);
}
OpCode::StoreUpvalue => {
let idx = read_u8(code, ip) as usize;
let val = self.stack.last().unwrap().clone(); // peek
let uv = Rc::clone(&self.frames.last().unwrap().closure.upvalues[idx]);
let mut uv_ref = uv.borrow_mut();
if let Some(ref mut closed) = uv_ref.closed {
*closed = val;
} else {
self.stack[uv_ref.location] = val;
}
self.advance_ip(SIZE_U8); self.advance_ip(SIZE_U8);
} }
@@ -209,7 +246,8 @@ impl Vm {
OpCode::LoadGlobal => { OpCode::LoadGlobal => {
let name_idx = read_u16(code, ip) as usize; let name_idx = read_u16(code, ip) as usize;
let name = proto_string(&proto, name_idx)?; let name = proto_string(&proto, name_idx)?;
let val = self.globals.borrow().get(&name).cloned() let val = self.globals.borrow().get(&name)
.map(|(v, _)| v.clone())
.or_else(|| self.builtins.borrow().get(&name).cloned()) .or_else(|| self.builtins.borrow().get(&name).cloned())
.ok_or_else(|| RuntimeError::RuntimeError { .ok_or_else(|| RuntimeError::RuntimeError {
message: format!("Undefined variable '{}'", name), message: format!("Undefined variable '{}'", name),
@@ -222,16 +260,31 @@ impl Vm {
let name_idx = read_u16(code, ip) as usize; let name_idx = read_u16(code, ip) as usize;
let name = proto_string(&proto,name_idx)?; let name = proto_string(&proto,name_idx)?;
let val = self.stack.last().unwrap().clone(); let val = self.stack.last().unwrap().clone();
self.globals.borrow_mut().insert(name, val); {
let mut globals = self.globals.borrow_mut();
match globals.get(&name) {
Some((_, false)) => {
return Err(RuntimeError::RuntimeError {
message: format!("Cannot reassign constant '{}'", name),
token: None,
});
}
_ => {
// Exists and mutable, or not yet defined — store as mutable
globals.insert(name, (val, true));
}
}
}
self.advance_ip(SIZE_U16); self.advance_ip(SIZE_U16);
} }
OpCode::DefineGlobal => { OpCode::DefineGlobal => {
let name_idx = read_u16(code, ip) as usize; let name_idx = read_u16(code, ip) as usize;
let name = proto_string(&proto,name_idx)?; let name = proto_string(&proto,name_idx)?;
let mutable = code[ip + 3] != 0; // 0 = const, 1 = mutable
let val = self.stack.pop().unwrap(); let val = self.stack.pop().unwrap();
self.globals.borrow_mut().insert(name, val.clone()); self.globals.borrow_mut().insert(name, (val.clone(), mutable));
self.stack.push(val); self.stack.push(val);
self.advance_ip(SIZE_U16); self.advance_ip(SIZE_U16 + 1); // 4 bytes: opcode + u16 + u8
} }
// --- Properties --- // --- Properties ---
@@ -402,31 +455,34 @@ impl Vm {
let result = self.stack.pop().unwrap(); let result = self.stack.pop().unwrap();
let frame = self.frames.pop().unwrap(); let frame = self.frames.pop().unwrap();
self.close_upvalues(frame.stack_base); self.close_upvalues(frame.stack_base);
self.stack.truncate(frame.stack_base); // Remove callee + args + function locals, push return value
self.stack.truncate(frame.stack_base.saturating_sub(1));
self.stack.push(result); self.stack.push(result);
if self.frames.is_empty() { if self.frames.is_empty() {
return Ok(()); return Ok(());
} }
// IP of caller is unchanged (already at next instruction)
} }
OpCode::Closure => { OpCode::Closure => {
let proto_idx = read_u16(code, ip) as usize; let proto_idx = read_u16(code, ip) as usize;
let upvalue_count = code[ip + 3] as usize; let upvalue_count = code[ip + 3] as usize;
let proto = Rc::clone(&self.protos[proto_idx]); let proto = proto.protos.get(proto_idx).ok_or_else(|| RuntimeError::RuntimeError {
message: format!("Closure proto index {} out of bounds", proto_idx),
token: None,
})?.clone();
// Collect upvalue capture info from bytecode first // Collect upvalue capture info
struct UpCapture { is_local: bool, index: usize } struct UpCapture { is_local: bool, index: usize }
let mut captures: Vec<UpCapture> = Vec::new(); let mut captures: Vec<UpCapture> = Vec::new();
let mut offset = ip + 4; let mut off = ip + 4;
for _ in 0..upvalue_count { for _ in 0..upvalue_count {
let is_local = code[offset] != 0; let is_local = code[off] != 0;
offset += 1; off += 1;
let index = code[offset] as usize; let index = code[off] as usize;
offset += 1; off += 1;
captures.push(UpCapture { is_local, index }); captures.push(UpCapture { is_local, index });
} }
// Now do the actual captures (no conflicting borrows) // Do the actual captures
let base = self.frames.last().unwrap().stack_base; let base = self.frames.last().unwrap().stack_base;
let parent_upvalues = self.frames.last().unwrap().closure.upvalues.clone(); let parent_upvalues = self.frames.last().unwrap().closure.upvalues.clone();
let mut upvalues = Vec::new(); let mut upvalues = Vec::new();
@@ -439,17 +495,17 @@ impl Vm {
} }
} }
// Store closure (stub for now) let closure = Rc::new(Closure { proto, upvalues });
let _closure = Rc::new(Closure { proto, upvalues }); let func = Rc::new(crate::interpreter::Function {
self.stack.push(Value::Function(Rc::new(
crate::interpreter::Function {
params: Vec::new(), params: Vec::new(),
body: Vec::new(), body: Vec::new(),
env: Rc::new(RefCell::new(crate::interpreter::Env::new(None))), env: Rc::new(RefCell::new(crate::interpreter::Env::new(None))),
name: None, name: None,
} });
))); let key = Rc::as_ptr(&func) as *const crate::interpreter::Function;
self.advance_ip_to(offset); self.closures.borrow_mut().insert(key, closure);
self.stack.push(Value::Function(func));
self.advance_ip_to(off);
} }
// --- Object/Array --- // --- Object/Array ---
@@ -471,32 +527,41 @@ impl Vm {
// --- For-in --- // --- For-in ---
OpCode::ForInInit => { OpCode::ForInInit => {
let iterable = self.stack.pop().unwrap(); let iterable = self.stack.pop().unwrap();
// Push iterator state: (collection, index) let items = self.for_in_items(&iterable);
let iter = Value::Number(0.0); self.stack.push(Value::Array(Rc::new(RefCell::new(items))));
self.stack.push(iterable);
self.stack.push(iter);
self.advance_ip(SIZE_OP); self.advance_ip(SIZE_OP);
} }
OpCode::ForInNext => { OpCode::ForInNext => {
let exit_offset = read_i16(code, ip) as isize; // Encoding: opcode(1) + items_slot(1) + idx_slot(1) + exit_offset(2) = 5
let exit_ip = ((ip as isize) + exit_offset) as usize; let items_slot = code[ip + 1] as usize;
let iter_idx = self.stack.pop().unwrap(); // current index let idx_slot = code[ip + 2] as usize;
let collection = self.stack.pop().unwrap(); // the iterable let exit_offset = ((code[ip + 3] as u16) | ((code[ip + 4] as u16) << 8)) as i16;
let exit_ip = ((ip as isize) + exit_offset as isize) as usize;
let base = self.frames.last().unwrap().stack_base;
// Read index from local slot
let idx_val = self.stack[base + idx_slot].clone();
let idx = self.as_usize(&idx_val, "for-in index")?;
let items_val = self.stack[base + items_slot].clone();
let items = match &items_val {
Value::Array(arr) => arr.borrow(),
_ => return Err(RuntimeError::RuntimeError {
message: "ForInNext: items is not an array".into(),
token: None,
}),
};
let idx = self.as_number(&iter_idx)? as usize;
let items = self.for_in_items(&collection);
if idx >= items.len() { if idx >= items.len() {
// Done iterating — push back state and jump to exit // Done: push Nil (keeps stack balanced for Pop at exit)
self.stack.push(collection); self.stack.push(Value::Nil);
self.stack.push(iter_idx);
self.advance_ip_to(exit_ip); self.advance_ip_to(exit_ip);
} else { } else {
// Push back incremented state // Push current element for loop body
self.stack.push(collection);
self.stack.push(Value::Number((idx + 1) as f64));
// Push the current value for the loop body
self.stack.push(items[idx].clone()); self.stack.push(items[idx].clone());
self.advance_ip(SIZE_U16); // Increment index in local slot
self.stack[base + idx_slot] = Value::Number((idx + 1) as f64);
self.advance_ip(5); // SIZE_FORIN_NEXT = 5
} }
} }
@@ -507,9 +572,16 @@ impl Vm {
let compound_op = CompoundOp::from_u8(op_tag).unwrap(); let compound_op = CompoundOp::from_u8(op_tag).unwrap();
let rhs = self.stack.pop().unwrap(); let rhs = self.stack.pop().unwrap();
let base = self.frames.last().unwrap().stack_base; let base = self.frames.last().unwrap().stack_base;
let lhs = self.stack[base + slot].clone(); let idx = base + slot;
if idx >= self.stack.len() {
return Err(RuntimeError::RuntimeError {
message: format!("CompoundAssignLocal to uninitialized local {}", slot),
token: None,
});
}
let lhs = self.stack[idx].clone();
let result = self.apply_compound_op(lhs, rhs, compound_op)?; let result = self.apply_compound_op(lhs, rhs, compound_op)?;
self.stack[base + slot] = result.clone(); self.stack[idx] = result.clone();
self.stack.push(result); self.stack.push(result);
self.advance_ip(SIZE_U8 + 1); self.advance_ip(SIZE_U8 + 1);
} }
@@ -538,6 +610,28 @@ impl Vm {
self.stack.push(result); self.stack.push(result);
self.advance_ip(SIZE_OP + 1); self.advance_ip(SIZE_OP + 1);
} }
OpCode::CompoundAssignUpvalue => {
let uv_idx = read_u8(code, ip) as usize;
let op_tag = code[ip + 2];
let compound_op = CompoundOp::from_u8(op_tag).unwrap();
let rhs = self.stack.pop().unwrap();
let uv = Rc::clone(&self.frames.last().unwrap().closure.upvalues[uv_idx]);
let mut uv_ref = uv.borrow_mut();
let lhs = if let Some(ref closed) = uv_ref.closed {
closed.clone()
} else {
self.stack[uv_ref.location].clone()
};
let result = self.apply_compound_op(lhs, rhs, compound_op)?;
if let Some(ref mut closed) = uv_ref.closed {
*closed = result.clone();
} else {
self.stack[uv_ref.location] = result.clone();
}
drop(uv_ref);
self.stack.push(result);
self.advance_ip(SIZE_U8 + 1);
}
} }
} }
} }
@@ -546,10 +640,6 @@ impl Vm {
// IP management // IP management
// ======================================================================== // ========================================================================
fn frame(&self) -> &CallFrame {
self.frames.last().unwrap()
}
fn frame_mut(&mut self) -> &mut CallFrame { fn frame_mut(&mut self) -> &mut CallFrame {
self.frames.last_mut().unwrap() self.frames.last_mut().unwrap()
} }
@@ -570,8 +660,13 @@ impl Vm {
let callee_idx = self.stack.len() - 1 - arg_count; let callee_idx = self.stack.len() - 1 - arg_count;
let callee = self.stack[callee_idx].clone(); let callee = self.stack[callee_idx].clone();
match callee { match &callee {
Value::NativeFunction(native_fn) => { Value::NativeFunction(_) => {
// Get the native function
let native_fn = match self.stack[callee_idx].clone() {
Value::NativeFunction(f) => f,
_ => unreachable!(),
};
// Pop arguments from stack // Pop arguments from stack
let mut args = Vec::new(); let mut args = Vec::new();
for _ in 0..arg_count { for _ in 0..arg_count {
@@ -583,27 +678,27 @@ impl Vm {
self.stack.push(result); self.stack.push(result);
self.advance_ip(SIZE_U8); self.advance_ip(SIZE_U8);
} }
Value::Function(f) => { Value::Function(_) => {
// User-defined function: create new call frame // Look up the VM-compiled closure
let base = callee_idx; // where the new frame's locals start let func_ptr = match &self.stack[callee_idx] {
let new_closure = Rc::new(Closure { Value::Function(f) => Rc::as_ptr(f) as *const crate::interpreter::Function,
proto: Rc::new(FunctionProto::new(f.name.clone())), _ => unreachable!(),
upvalues: Vec::new(), };
}); let closure = self.closures.borrow().get(&func_ptr).cloned().ok_or_else(|| RuntimeError::RuntimeError {
message: "VM: call to non-VM function (tree-walker function not supported in VM)".into(),
token: None,
})?;
// Advance caller's IP past the Call instruction before pushing new frame
self.frame_mut().ip += SIZE_U8;
// Callee is at callee_idx, args start at callee_idx+1
let base = callee_idx + 1; // first param is here
self.frames.push(CallFrame { self.frames.push(CallFrame {
closure: new_closure, closure,
ip: 0, ip: 0,
stack_base: base, stack_base: base,
}); });
// Arguments are already on the stack at the right position
// (they become locals 0..arg_count in the new frame)
// The callee itself becomes slot 0 (TODO: handle properly)
// For now, this is a stub — full function support needs
// proper closure/proto integration
return Err(RuntimeError::RuntimeError {
message: "VM user-defined function calls not yet fully implemented".into(),
token: None,
});
} }
_ => { _ => {
return Err(RuntimeError::RuntimeError { return Err(RuntimeError::RuntimeError {
@@ -856,13 +951,6 @@ impl Vm {
} }
} }
fn get_constant(&self, idx: usize) -> Result<Value, RuntimeError> {
let frame = self.frames.last().unwrap();
frame.closure.proto.constants.get(idx).cloned().ok_or_else(|| RuntimeError::RuntimeError {
message: format!("Constant index {} out of bounds", idx),
token: None,
})
}
// ======================================================================== // ========================================================================
// Upvalues // Upvalues
@@ -912,7 +1000,7 @@ impl Vm {
impl Runtime for Vm { impl Runtime for Vm {
fn require(&mut self, path: &str) -> Result<Value, RuntimeError> { fn require(&mut self, path: &str) -> Result<Value, RuntimeError> {
// Simplified require for VM: lex → parse → compile → execute // 1. Path resolution
let resolved = { let resolved = {
let path = std::path::Path::new(path); let path = std::path::Path::new(path);
let resolved = if path.is_absolute() { let resolved = if path.is_absolute() {
@@ -928,16 +1016,19 @@ impl Runtime for Vm {
})? })?
}; };
// 2. Cache check
if let Some(cached) = self.module_cache.borrow().get(&resolved) { if let Some(cached) = self.module_cache.borrow().get(&resolved) {
return Ok(cached.clone()); return Ok(cached.clone());
} }
// 3. Read file
let src = std::fs::read_to_string(&resolved) let src = std::fs::read_to_string(&resolved)
.map_err(|e| RuntimeError::RuntimeError { .map_err(|e| RuntimeError::RuntimeError {
message: format!("Cannot read module '{}': {}", path, e), message: format!("Cannot read module '{}': {}", path, e),
token: None, token: None,
})?; })?;
// 4. Lex
let (tokens, lex_errors) = Lexer::new(&src).tokenize(); let (tokens, lex_errors) = Lexer::new(&src).tokenize();
if !lex_errors.is_empty() { if !lex_errors.is_empty() {
return Err(RuntimeError::RuntimeError { return Err(RuntimeError::RuntimeError {
@@ -946,6 +1037,7 @@ impl Runtime for Vm {
}); });
} }
// 5. Parse
let mut parser = Parser::new(tokens); let mut parser = Parser::new(tokens);
let (stmts, parse_errors) = parser.parse(); let (stmts, parse_errors) = parser.parse();
if !parse_errors.is_empty() { if !parse_errors.is_empty() {
@@ -955,7 +1047,13 @@ impl Runtime for Vm {
}); });
} }
// Create isolated VM for module execution // 6. Compile
let proto = Compiler::compile(&stmts).map_err(|e| RuntimeError::RuntimeError {
message: format!("Compile error in module '{}': {}", path, e),
token: None,
})?;
// 7. Create isolated module VM with shared module cache (for cyclic requires)
let module_dir = std::path::Path::new(&resolved) let module_dir = std::path::Path::new(&resolved)
.parent() .parent()
.map(|p| p.to_string_lossy().to_string()) .map(|p| p.to_string_lossy().to_string())
@@ -963,31 +1061,33 @@ impl Runtime for Vm {
let mut module_vm = Vm::new(); let mut module_vm = Vm::new();
module_vm.current_dir = module_dir; module_vm.current_dir = module_dir;
// Share module cache and builtins
module_vm.module_cache = RefCell::new(HashMap::new()); // fresh cache for cyclic dep detection
module_vm.builtins = Rc::clone(&self.builtins); module_vm.builtins = Rc::clone(&self.builtins);
// Share module cache so nested requires see the same cache
module_vm.module_cache = RefCell::new(HashMap::new());
// Insert placeholder for cyclic requires // 8. Insert placeholder in module_vm cache (for cyclic requires within module)
let exports_obj = Value::Object(Rc::new(RefCell::new(HashMap::new()))); let exports_obj = Value::Object(Rc::new(RefCell::new(HashMap::new())));
self.module_cache.borrow_mut().insert(resolved.clone(), exports_obj.clone()); module_vm.module_cache.borrow_mut().insert(resolved.clone(), exports_obj.clone());
// Compile and run // 9. Execute module
let proto = Compiler::compile(&stmts).map_err(|e| RuntimeError::RuntimeError {
message: format!("Compile error in module '{}': {}", path, e),
token: None,
})?;
module_vm.run(Rc::new(proto))?; module_vm.run(Rc::new(proto))?;
// Collect exports from module's globals // 10. Collect exports from module globals
if let Value::Object(exports_map) = &exports_obj { if let Value::Object(exports_map) = &exports_obj {
let mut map = exports_map.borrow_mut(); let mut map = exports_map.borrow_mut();
for (name, val) in module_vm.globals.borrow().iter() { for (name, (val, _)) in module_vm.globals.borrow().iter() {
map.insert(name.clone(), val.clone()); map.insert(name.clone(), val.clone());
} }
// Update shared module cache
self.module_cache.borrow_mut().insert(resolved, exports_obj.clone());
} }
// 11. Transfer closures from module VM to parent (so exported fns are callable)
for (key, closure) in module_vm.closures.borrow().iter() {
self.closures.borrow_mut().insert(*key, Rc::clone(closure));
}
// 12. Cache in parent
self.module_cache.borrow_mut().insert(resolved, exports_obj.clone());
Ok(exports_obj) Ok(exports_obj)
} }
} }