This required a massive cross-cutting rework to the scope system,
multiple subtle bugfixes and relearning of the parser internals, but it
does appear that functions are working correctly.
A few caveats: for now, parameters are always constant, regardless of
type, return values can't be specified, and some script tests have been
written.
Most importantly, a key feature is working: closures.
Also fixed a minor bug with printing, and removed the ability to
configure the parser.
Added and updated QUICKSTART.md as a quick way to get people started.
There's some broken scripts under 'scripts/' that require functions to
work properly.
The 'source' directory compiles, but the repl and tests are almost
untouched so far. There's no guarantee that the code in 'source' is
correct, so I'm branching this for a short time, until I'm confident the
whole project passes the CI again.
I'm adjusting the concepts of routines and bytecode to make them more
consistent, and tweaking the VM so it loads from an instance of
'Toy_Module'.
* 'Toy_ModuleBuilder' (formally 'Toy_Routine')
This is where the AST is compiled, producing a chunk of memory that can
be read by the VM. This will eventually operate on individual
user-defined functions as well.
* 'Toy_ModuleBundle' (formally 'Toy_Bytecode')
This collects one or more otherwise unrelated modules into one chunk of
memory, stored in sequence. It is also preprended with the version data for
Toy's reference implementation:
For each byte in the bytecode:
0th: TOY_VERSION_MAJOR
1st: TOY_VERSION_MINOR
2nd: TOY_VERSION_PATCH
3rd: (the number of modules in the bundle)
4th and onwards: TOY_VERSION_BUILD
TOY_VERSION_BUILD has always been a null terminated C-string, but from
here on, it begins at the word-alignment, and continues until the first
word-alignment after the null terminator.
As for the 3rd byte listed, since having more than 256 modules in one
bundle seems unlikely, I'm storing the count here, as it was otherwise
unused. This is a bit janky, but it works for now.
* 'Toy_Module'
This new structure represents a single complete unit of operation, such
as a single source file, or a user-defined function. It is divided into
three main sections, with various sub-sections.
HEADER (all members are unsigned ints):
total module size in bytes
jumps count
param count
data count
subs count
code addr
jumps addr (if jumps count > 0)
param addr (if param count > 0)
data addr (if data count > 0)
subs addr (if subs count > 0)
BODY:
<raw opcodes, etc.>
DATA:
jumps table
uint array, pointing to addresses in 'data' or 'subs'
param table
uint array, pointing to addresses in 'data'
data
heterogeneous data, including strings
subs
an array of modules, using recursive logic
The reference implementation as a whole uses a lot of recursion, so this
makes sense.
The goal of this rework is so 'Toy_Module' can be added as a member of
'Toy_Value', as a simple and logical way to handle functions. I'll
probably use the union pattern, similarly to Toy_String, so functions
can be written in C and Toy, and used without needing to worry which is
which.
Toy now fits into the C spec.
Fixed#158
Addendum: MacOS test caught an error:
error: a function declaration without a prototype is deprecated in all versions of C
That took 3 attempts to fix correctly.
Addendum: 'No new line at the end of file' are you shitting me?
I attempted to add '-Wpedantic' to CFLAGS, but it seems that my usage of
the variable length arrays within unions is causing an error that can't
be selectively disabled:
error: invalid use of structure with flexible array member [-Werror=pedantic]
This is the offending code: /source/toy_string.h#L9-L37
It seems that tagged unions, with VLAs within, is simply not allowed.
Unfortunately, my whole string system depends on it. I'll have to find some way
around it.
I've also updated the debugging output in repl/main.c.
Getting the array's length is still not available yet, so I'm not
marking arrays as done - but everything that is there is tested.
I've also tweaked the assert output callbacks to also print 'assert failure'.
I had intended to solve the Advent of Code puzzles in Toy, but they
don't work without arrays. I wasn't able to enable arrays in time, so
I need to focus on doing things correctly.
The most immediate tasks are marked with 'URGENT', and a lot of tests
need writing.
I've also added some support for compiler errors in general, but these
will get expanded on later.
I've also quickly added a valgrind option to the tests and found a few
leaks. I'll deal with these later.
Summary of changes:
* Clarified the lifetime of the bytecode in memory
* Erroneous routines exit without compiling
* Empty VMs don't run
* Added a check for malformed assignments
* Renamed "routine" to "module" within the VM
* VM no longer tries to free the bytecode - must be done manually
* Started experimenting with valgrind, not yet ready
I was coding earlier this week, but my brain was so foggy I ended up not
knowing what I was doing. After a few days break, I've cleaned up the
mess, which took hours.
Changes:
* Variables can be assigned
* Added new value types as placeholders
* Added 'compare' and 'assign' to the AST
* Added duplicate opcode
* Added functions to copy and free values
* Max name length is 255 chars
* Compound assigns are squeezed into one word
To be completed:
* Tests for this commit's changes
* Compound assignments
* Variable access
Kayne Ruse