assembler: wrap Module's with Arc and update Tokeniser (still WIP)
Implements complete tokenizer with Arc-wrapped modules Enhances module handling by wrapping Module instances in Arc for thread-safe sharing across the assembler pipeline. Implements full tokenization logic with pattern matching for all token types including labels, registers, immediates, directives, instructions, symbols, and strings. Adds comma token support and proper EOF handling to complete the lexical analysis phase. Generated AI slop commit message, may not be super accurate or it may be a bit too serious lol.
This commit is contained in:
@@ -23,8 +23,10 @@ pub enum TokenType {
|
||||
Instruction(InstructionToken),
|
||||
/// Label definition (e.g., `loop_start:`).
|
||||
Label(LabelToken),
|
||||
/// Assembler directive (e.g., `.global`, `.section`, `.dw`).
|
||||
/// Assembler directive (e.g., `.global`, `.section`, `.dw`, `.resb`).
|
||||
Directive(DirectiveToken),
|
||||
/// Comma separator.
|
||||
Comma,
|
||||
/// End of line.
|
||||
Newline,
|
||||
/// End of file.
|
||||
|
||||
Reference in New Issue
Block a user