Implements complete tokenizer with Arc-wrapped modules
Enhances module handling by wrapping Module instances in Arc for thread-safe sharing across the assembler pipeline.
Implements full tokenization logic with pattern matching for all token types including labels, registers, immediates, directives, instructions, symbols, and strings.
Adds comma token support and proper EOF handling to complete the lexical analysis phase.
Generated AI slop commit message, may not be super accurate or it may be a bit too serious lol.
nullndvoid
9b9e153500