The 'source' directory compiles, but the repl and tests are almost
untouched so far. There's no guarantee that the code in 'source' is
correct, so I'm branching this for a short time, until I'm confident the
whole project passes the CI again.
I'm adjusting the concepts of routines and bytecode to make them more
consistent, and tweaking the VM so it loads from an instance of
'Toy_Module'.
* 'Toy_ModuleBuilder' (formally 'Toy_Routine')
This is where the AST is compiled, producing a chunk of memory that can
be read by the VM. This will eventually operate on individual
user-defined functions as well.
* 'Toy_ModuleBundle' (formally 'Toy_Bytecode')
This collects one or more otherwise unrelated modules into one chunk of
memory, stored in sequence. It is also preprended with the version data for
Toy's reference implementation:
For each byte in the bytecode:
0th: TOY_VERSION_MAJOR
1st: TOY_VERSION_MINOR
2nd: TOY_VERSION_PATCH
3rd: (the number of modules in the bundle)
4th and onwards: TOY_VERSION_BUILD
TOY_VERSION_BUILD has always been a null terminated C-string, but from
here on, it begins at the word-alignment, and continues until the first
word-alignment after the null terminator.
As for the 3rd byte listed, since having more than 256 modules in one
bundle seems unlikely, I'm storing the count here, as it was otherwise
unused. This is a bit janky, but it works for now.
* 'Toy_Module'
This new structure represents a single complete unit of operation, such
as a single source file, or a user-defined function. It is divided into
three main sections, with various sub-sections.
HEADER (all members are unsigned ints):
total module size in bytes
jumps count
param count
data count
subs count
code addr
jumps addr (if jumps count > 0)
param addr (if param count > 0)
data addr (if data count > 0)
subs addr (if subs count > 0)
BODY:
<raw opcodes, etc.>
DATA:
jumps table
uint array, pointing to addresses in 'data' or 'subs'
param table
uint array, pointing to addresses in 'data'
data
heterogeneous data, including strings
subs
an array of modules, using recursive logic
The reference implementation as a whole uses a lot of recursion, so this
makes sense.
The goal of this rework is so 'Toy_Module' can be added as a member of
'Toy_Value', as a simple and logical way to handle functions. I'll
probably use the union pattern, similarly to Toy_String, so functions
can be written in C and Toy, and used without needing to worry which is
which.
Toy now fits into the C spec.
Fixed#158
Addendum: MacOS test caught an error:
error: a function declaration without a prototype is deprecated in all versions of C
That took 3 attempts to fix correctly.
Addendum: 'No new line at the end of file' are you shitting me?
I attempted to add '-Wpedantic' to CFLAGS, but it seems that my usage of
the variable length arrays within unions is causing an error that can't
be selectively disabled:
error: invalid use of structure with flexible array member [-Werror=pedantic]
This is the offending code: /source/toy_string.h#L9-L37
It seems that tagged unions, with VLAs within, is simply not allowed.
Unfortunately, my whole string system depends on it. I'll have to find some way
around it.
I've also updated the debugging output in repl/main.c.
Getting the array's length is still not available yet, so I'm not
marking arrays as done - but everything that is there is tested.
I've also tweaked the assert output callbacks to also print 'assert failure'.
I had intended to solve the Advent of Code puzzles in Toy, but they
don't work without arrays. I wasn't able to enable arrays in time, so
I need to focus on doing things correctly.
The most immediate tasks are marked with 'URGENT', and a lot of tests
need writing.
I've also added some support for compiler errors in general, but these
will get expanded on later.
I've also quickly added a valgrind option to the tests and found a few
leaks. I'll deal with these later.
Summary of changes:
* Clarified the lifetime of the bytecode in memory
* Erroneous routines exit without compiling
* Empty VMs don't run
* Added a check for malformed assignments
* Renamed "routine" to "module" within the VM
* VM no longer tries to free the bytecode - must be done manually
* Started experimenting with valgrind, not yet ready
I was coding earlier this week, but my brain was so foggy I ended up not
knowing what I was doing. After a few days break, I've cleaned up the
mess, which took hours.
Changes:
* Variables can be assigned
* Added new value types as placeholders
* Added 'compare' and 'assign' to the AST
* Added duplicate opcode
* Added functions to copy and free values
* Max name length is 255 chars
* Compound assigns are squeezed into one word
To be completed:
* Tests for this commit's changes
* Compound assignments
* Variable access
This wasn't an easy fix, as it was primarily the test pipelines that
were failing. I resorted to using forced pushes to run the CI, to try
and track down the problems.
The primary cause seems to be the differences in how each supported
platform handles file paths, specifically, slash vs. backslash.
I've also added gdb scripts to set up automated breakpoints, and to run
operations on them to check for issues - the 'gdb_init' files are mostly
empty for the time being.
commit a34b0ff5d407bbe7d70ff9504aa035ec6fbecb7c
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Fri Oct 18 11:45:40 2024 +1100
Restored the workflows
commit eb3d94f30d4dc4150139517f44cc874f2901124f
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Fri Oct 18 11:35:39 2024 +1100
I think the library path on macos is fixed
commit 964572b5e93c7cb464686f19ddbe3e9d315f391b
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Fri Oct 18 11:22:56 2024 +1100
I think the file paths are fixed
commit 1721f3da7252b4063f4347926e800ef4f7c9bf4c
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Thu Oct 17 15:57:28 2024 +1100
Added standalone tests
commit 90c783f4059d88f4a7bbaf18215a9b414f3ab66f
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Thu Oct 17 15:01:18 2024 +1100
Trying to fix the integration test pipeline
commit fccced1396568a55c1385e2f1b04fedf7c2585a5
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Wed Oct 16 00:31:39 2024 +1100
Workflow integration tests are not passing
commit 6b1e0d1e0f89291e89768bf6102f4f7ed4581496
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Tue Oct 15 20:07:20 2024 +1100
Fixed file paths in workflow
commit c0f1ec78fe79a5abb34c3e05308236cb18c23b97
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Tue Oct 15 19:46:14 2024 +1100
Moved example scripts into proper integration tests
Also adjusted makefiles to allow easy invoking of the tests.
Adjusted and updated CI to invoke tests correctly.
Fixed#141
Strings, due to their potentially large size, are stored outside of a
routine's code section, in the data section. To access the correct
string, you must read the jump index, then the real address from the
jump table - and extra layer of indirection will result in more flexible
data down the road, I hope.
Other changes include:
* Added string concat operator ..
* Added TOY_STRING_MAX_LENGTH
* Strings can't be created or concatenated longer than the max length
* The parser will display a warning if the bucket is too small for a
string at max length, but it will continue
* Added TOY_BUCKET_IDEAL to correspend with max string length
* The bucket now allocates an address that is 4-byte aligned
* Fixed missing entries in the parser rule table
* Corrected some failing TOY_BITNESS tests
The following structures are now more independant:
- Toy_Array
- Toy_Stack
- Toy_Bucket
- Toy_String
I reworked a lot of the memory allocation, so now there are more direct
calls to malloc() or realloc(), rather than relying on the macros from
toy_memory.h.
I've also split toy_memory into proper array and bucket files, because
it makes more sense this way, rather than having them both jammed into
one file. This means the eventual hashtable structure can also stand on
its own.
Toy_Array is a new wrapper around raw array pointers, and all of the
structures have their metadata embedded into their allocated memory now,
using variable length array members.
A lot of 'capacity' and 'count' variables were changed to 'size_t'
types, but this doesn't seem to be a problem anywhere.
If the workflow fails, then I'll leave it for tonight - I'm too tired,
and I don't want to overdo myself.
It's not ready yet, but considering how much crap this took to get
working... I'm done for the night.
commit 0f3ee91a0628654a61d47a0c41cd3e39c801b8f9
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 20:27:44 2024 +1000
I have a titanic patience... but even the titanic sank
commit d606e0948b78bc0614c81bbea48abf5ffd9f2194
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 20:03:57 2024 +1000
Terminated
commit 7b3b59321d349a8e361857ee3dbe955ec27cb38c
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 20:00:11 2024 +1000
What the hell?
commit 36104b7b5a5d5487c84ab98fec1ae69487fd1a90
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 19:55:28 2024 +1000
PLEASE
commit 0de373d10ad56cba228e9473509a527bf7a3208d
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 19:31:28 2024 +1000
Remember to commit the file!
commit 1cb5780e2dc281ecc723b4042cd9ee1f9a3115ab
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 19:24:32 2024 +1000
Please be correct
commit 2235d716c1d872db3744905fbd2305e65f96361a
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:58:00 2024 +1000
Nearly there
commit eabe49130cfdbe93cc3a2056819e267f97304cf1
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:51:04 2024 +1000
Oh green world, don't desert me now
commit f0127e2d4947ce47284f2ec72f41a5b6a62121b8
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:42:50 2024 +1000
I'm on a highway to hell...
commit f96c0f51a48b080d11d15947940d678ae4f88024
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:41:01 2024 +1000
OK, so there's a lot of security there.
commit af3645893a744b7cfa2638bf607373565b8e6ef9
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:33:51 2024 +1000
I'm going to steal the declaration of independence
commit e71e1c1b4243e89a580d8b04b4d236ea66ded78c
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:27:11 2024 +1000
Show me the money!
commit f29ba6eb96c31e9bbcf23b6ece137381f5be4050
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:12:23 2024 +1000
Say hello to my little friend!
commit 3a5ee93884b1169abae48923161d7b47b8a45cdd
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:06:59 2024 +1000
You had me at hello
commit 2901d92f24082173e7119a37baa12a6e968796c8
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 18:02:49 2024 +1000
You and your little dog too!
commit 967194e0593233e17ea7737647dc8042ee3bf0aa
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:52:13 2024 +1000
I'll get you my pretty!
commit ad48a7aec8ba7ab9e11d23104cd171169cc5ff45
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:50:14 2024 +1000
Do you feel lucky, punk?
commit 13fa5c1e96fc7c64f71009f705b8bd69b8dfabf2
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:48:04 2024 +1000
Frankly my dear, I don't give a damn
commit 2f15a5a7a71c22bd3f34f619887137c7cd09001b
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:20:06 2024 +1000
One of us, one of us, one of us!
commit e696d15ea62cb5d965638e04e6f0458b19cf9e28
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:16:04 2024 +1000
Why did it have to be snakes?
commit 90631eda6263c30b0643007fc2c7a84e5e08826d
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:07:11 2024 +1000
No, I am your father!
commit 22e1d61caac3d37ce7fe9fc41bc3b60949fe5a81
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 17:00:18 2024 +1000
I'm sorry Dave, I can't let you do that
commit 9aa17b8b04eb65c6c9bf8b015458f749db97494d
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:57:24 2024 +1000
LIVE, DARN YOU, LIVE!
commit 1b7f0704d9a2744f10377384ff5f36ea7f61c2da
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:52:58 2024 +1000
That's a whole lotta nothing
commit fd4001a9eb557afb3d6cbe409c72f7416b08db60
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:44:34 2024 +1000
Here goes nothing
commit 7ba09ef0acda2fede952a912a8a1849e9b6c59b7
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:40:28 2024 +1000
Oh, this is gonna be fun, trying to build on a machine I don't even have
commit 1155054552ef46ceb4c21117e35f8e5e46b26dbe
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:34:06 2024 +1000
There is no repl-gdb in Ba Sing Se
commit 5361a31da1d9278972dcbe3c8a0ad59ea6e2ab41
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:31:05 2024 +1000
Changed -R to -rpath
commit de982ce2846dd6ffe316648e32a8a04b9d7f242d
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:27:14 2024 +1000
Fixed printf() string literal
commit 5ddec42af5b46a0c25f64b088a4992433e5a4116
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:24:05 2024 +1000
Workflow experiment
commit 28570940b8758fc7f05c957055580e4286a887af
Author: Kayne Ruse <kayneruse@gmail.com>
Date: Sat Sep 28 16:10:19 2024 +1000
Fixed workflow for macos
After a few hours struggling with the linker, I've got the main.c file
running correctly, with caveats:
- must be executed from out/
- only building on linux for the moment
- no tests written yet
I will write some CI jobs to see if the repl works eventually.
