Updated repl_tools.h and toy.h

c-api/repl_tools_h.md

@@ -1,44 +1,51 @@
-# repl_tools.h
+
-
+# repl_tools.h
-This header provides a number of tools for compiling and running Toy, and is used primarily by the repl. However, it can also be modified and used by any host program with little effort.
+
-
+This header provides a number of tools for compiling and running Toy, and is used primarily by the repl. However, it can also be modified and used by any host program with a little effort.
-This is not a core part of the Toy library, and as such `repl_tools.h` and `repl_tools.c` can both be found in the `repl/` folder.
+
-
+This is not a core part of Toy or a library, and as such `repl_tools.h` and `repl_tools.c` can both be found in the `repl/` folder.
-### const char* Toy_readFile(const char* path, size_t* fileSize)
+
-
+### const char* Toy_readFile(const char* path, size_t* fileSize)
-This function reads in a file, and returns it as a constant buffer. It also sets the variable pointed to by `fileSize` to the size of the given buffer.
+
-
+This function reads in a file, and returns it as a constant buffer. It also sets the variable pointed to by `fileSize` to the size of the given buffer.
-On error, this function returns `NULL`.
+
-
+On error, this function returns `NULL`.
-### int Toy_writeFile(const char* path, const unsigned char* bytes, size_t size)
+
-
+### int Toy_writeFile(const char* path, const unsigned char* bytes, size_t size)
-This function writes the buffer pointed to by `bytes` to a file specified by `path`. The buffer's size should be specified by `size`.
+
-
+This function writes the buffer pointed to by `bytes` to a file specified by `path`. The buffer's size should be specified by `size`.
-On error, this function returns a non-zero value.
+
-
+On error, this function returns a non-zero value.
-### const unsigned char* Toy_compileString(const char* source, size_t* size)
+
-
+### const unsigned char* Toy_compileString(const char* source, size_t* size)
-This function takes a cstring of Toy source code, and returns a compiled buffer based on that source code. The variable pointed to by `size` is set to the size of the bytecode.
+
-
+This function takes a cstring of Toy source code, and returns a compiled buffer based on that source code. The variable pointed to by `size` is set to the size of the bytecode.
-On error, this function returns `NULL`.
+
-
+On error, this function returns `NULL`.
-### void Toy_runBinary(const unsigned char* tb, size_t size)
+
-
+### void Toy_runBinary(const unsigned char* tb, size_t size)
-This function takes a bytecode array of `size` size, and executes it. The libraries available to the code are:
+
-
+This function takes a bytecode array of `size` size, and executes it. The libraries available to the code are currently:
-* lib_about
+
-* lib_standard
+* lib_about
-* lib_runner
+* lib_standard
-
+* lib_random
-### void Toy_runBinaryFile(const char* fname)
+* lib_runner
-
+
-This function loads in the binary file specified by `fname`, and passes it to `Toy_runBinary`.
+### void Toy_runBinaryFile(const char* fname)
-
+
-### void Toy_runSource(const char* source)
+This function loads in the binary file specified by `fname`, and passes it to `Toy_runBinary()`.
-
+
-This function compiles the source with `Toy_compileString`, and passes it to `Toy_runBinary`.
+### void Toy_runSource(const char* source)
-
+
-### void Toy_runSourceFile(const char* fname)
+This function compiles the source with `Toy_compileString()`, and passes it to `Toy_runBinary()`.
-
+
-This function loads in the file specified by `fname`, compiles it, and passes it to `Toy_runBinary`.
+### void Toy_runSourceFile(const char* fname)
-
+
+This function loads in the file specified by `fname`, compiles it, and passes it to `Toy_runBinary()`.
+
+### void Toy_parseBinaryFileHeader(const char* fname)
+
+This function parses the header information stored within the bytecode file `fname`.
+
+This is only used for debugging and validation purposes.

c-api/toy_h.md

@@ -1,4 +1,60 @@
-# toy.h
+
-
+# toy.h - A Toy Programming Language
-This header includes a number of other header files, which are essential for the normal operation of Toy. It also includes some broad outlines, written in the comments, of what each header does. It's intended as a good starting point for people looking to dive deeper into the langauge's implementation.
+
-
+If you're looking how to use Toy directly, try https://toylang.com/
+Otherwise, this header may help learn how Toy works internally.
+
+## Utilities
+
+These headers define a bunch of useful macros, based on what platform you build for.
+
+The most important macro is `TOY_API`, which specifies functions intended for the end user.
+
+* [toy_common.h](toy_common_h.md)
+* [toy_console_colors.h](toy_console_colors_h.md)
+* [toy_memory.h](toy_memory_h.md)
+* [toy_drive_system.h](toy_drive_system_h.md)
+
+## Core Pipeline
+
+From source to execution, each step is as follows:
+
+```
+source   -> lexer       -> token
+token    -> parser      -> AST
+AST      -> compiler    -> bytecode
+bytecode -> interpreter -> result
+```
+
+I should note that the parser -> compiler phase is actually made up of two steps - the write step and the collate step. See `Toy_compileString()` in `repl/repl_tools.c` for an example of how to compile properly.
+
+* [toy_lexer.h](toy_lexer_h.md)
+* [toy_parser.h](toy_parser_h.md)
+* [toy_compiler.h](toy_compiler_h.md)
+* [toy_interpreter.h](toy_interpreter_h.md)
+
+## Building Block Structures
+
+Literals represent any value within the language, including some internal ones that you never see.
+
+Literal arrays are contiguous arrays within memory, and are the most heavily used structure in Toy.
+
+Literal dictionaries are unordered key-value hashmaps, that use a running strategy for collisions.
+
+* [toy_literal.h](toy_literal_h.md)
+* [toy_literal_array.h](toy_literal_array_h.md)
+* [toy_literal_dictionary.h](toy_literal_dictionary_h.md)
+
+## Other Components
+
+You probably won't use these directly, but they're a good learning opportunity.
+
+`Toy_Scope` holds the variables of a specific scope within Toy - be it a script, a function, a block, etc. Scopes are also where the type system lives at runtime. They use identifier literals as keys, exclusively.
+
+`Toy_RefString` is a utility class that wraps traditional C strings, making them less memory intensive and faster to copy and move. In reality, since strings are considered immutable, multiple variables can point to the same string to save memory, and you can just create a new one of these vars pointing to the original rather than copying entirely for a speed boost. This module has it's own memory allocator system that is plugged into the main memory allocator.
+
+`Toy_RefFunction` acts similarly to `Toy_RefString`, but instead operates on function bytecode.
+
+* [toy_scope.h](toy_scope_h.md)
+* [toy_refstring.h](toy_refstring_h.md)
+* [toy_reffunction.h](toy_reffunction_h.md)