Ratstail91/sineQL
1
0
Fork 0
mirror of https://github.com/Ratstail91/sineQL.git synced 2025-11-29 02:34:28 +11:00
Code Issues Packages Projects Releases Wiki Activity

Stripped this project to it's bones

Browse Source
This commit is contained in:
Kayne Ruse
2021-03-30 06:17:55 +11:00
parent 128a42aaa6
commit 9d17d350fa
16 changed files with 130 additions and 4493 deletions
Download Patch File Download Diff File Expand all files Collapse all files

175
README.md
View File

@@ -1,26 +1,73 @@
# Things That Need To Be Done
* ~~Debugging options~~
* N + 1 problem solved
* Change "match" to "filter"
* Full documentation
* Graphical tool
* GitHub CI testing
* Implement the create command
* Implement the update command
* Implement the delete command
# sineQL
sineQL is a web API query language that mimics graphQL, designed solely for fun.
sineQL consists of two languages - the schema language, and the query language.
sineQL consists of two languages - the schema language, and the query language. sineQL assumes that the records are related in a non-looping tree-structure, defined by the schema language.
You can try the API right now!
## Example Server
A simple express server using sineQL.
```js
//create the wave function, wrapping a fetch to a server
const wave = body => fetch('https://krgamestudios.com/pokemon', {
//express for testing
const express = require('express');
const app = express();
app.use(express.text());
//test the library
const sineQL = require('sineql');
const schema = require('./schema.js');
const handler = require('./handler.js');
const sine = sineQL(schema, handler, { debug: true });
//open the endpoint
app.post('/sineql', async (req, res) => {
const [code, result] = await sine(req.body);
res.status(code).send(result);
});
//startup
const port = process.env.WEB_PORT || 4000;
app.listen(port, err => {
console.log(`listening to *:${port}`);
});
```
```js
const schema = `
scalar Date
type Book {
String title
Date published
}
type Author {
String name
Book books
}
`;
module.exports = schema;
```
```js
//TODO: define the handler object's API properly
const handler = {
Book: () => null,
Author: () => null
};
module.exports = handler;
```
Create a matching client-side function pointing to the server.
```js
//create the wave function, wrapping a fetch to the server
const wave = body => fetch('http://example.com/sineql', {
method: 'POST',
headers: {
'Content-Type': 'text/plain'
@@ -28,11 +75,11 @@ const wave = body => fetch('https://krgamestudios.com/pokemon', {
body: body
});
//get a list of pokemon names
wave('Pokemon { name }')
//get a list of content
wave('Author { name books { title } }')
.then(blob => blob.text())
.then(text => console.log(text))
.catch(err => console.log(err))
.catch(e => console.error(e))
;
```
@@ -59,7 +106,6 @@ scalar Date
type Book {
String title
Author author
Date published
}
@@ -71,16 +117,14 @@ type Author {
## The Query Language
The query langauge can be used to request data from a server, either in whole or in part by listing it's type and it's fields, and subfields.
The query langauge can be used to request data from a server, either in whole or in part by listing its type and its needed fields:
```
Book {
title
author {
name
books {
title
}
Author {
name
books {
title
published
}
}
```
@@ -93,45 +137,21 @@ The fields can be altered as well, using the query language's built-in keywords:
* match
* set
`create`, `update` and `delete` do as you would expect them to.
`create`, `update` and `delete` work as expected.
When using `create`, `match` will find an existing record for a compound type and use that as it's value (multiple matches is an error):
### Create
When using `create`, `match` will find an existing record and associate that with the created values (multiple matches is an error):
```
create Book {
set title "The Wind in the Willows"
match author {
name "Kenneth Grahame"
}
Author {
match name "Kenneth Grahame"
create books {
create title "The Wind in the Willows"
}
}
```
When using `update`, `match` will find all existing records and update those using the `set` keyword:
```
update Book {
match title "The Wind in the Willows"
set published "15 June 1908"
}
```
```
update Book {
match title "The Wind in the Willows"
set title "The Fart in the Fronds"
}
```
When using `delete`, only `match` is valid, and will delete all matching records:
```
delete Book {
match title "The Fart in the Fronds"
}
```
You can use as many instances of `match` and `set` as you like, as long as the result is valid.
You can create multiple records at once by surrounding them with `[]`:
```
@@ -158,4 +178,35 @@ create Book [
set title "Yeah, I Got Nothing"
}
]
```
```
### Update
When using `update`, `match` will find all existing records and update those using the `set` keyword:
```
update Book {
match title "The Wind in the Willows"
set published "15 June 1908"
}
```
```
update Book {
match title "The Wind in the Willows"
set title "The Fart in the Fronds"
}
```
### Delete
When using `delete`, only `match` is valid, and will delete all matching records:
```
delete Book {
match title "The Fart in the Fronds"
}
```
You can use as many instances of `match` and `set` as you like, as long as the result is valid.

16
source/build-type-graph.js
View File

@@ -21,7 +21,8 @@ const buildTypeGraph = (schema, options) => {
//check for keywords
switch(tokens[pos - 1]) {
case 'type':
graph[tokens[pos++]] = parseCompoundType(tokens, pos, options);
//delegate
graph[tokens[pos++]] = parseCompoundType(tokens, pos, Object.keys(graph), options);
//advance to the end of the compound type
pos = eatBlock(tokens, pos);
@@ -29,7 +30,9 @@ const buildTypeGraph = (schema, options) => {
break;
case 'scalar':
//check against keyword list
if (keywords.includes(graph[tokens[pos - 1]])) {
//TODO: test this error
throw 'Unexpected keyword ' + graph[tokens[pos - 1]];
}
@@ -49,7 +52,7 @@ const buildTypeGraph = (schema, options) => {
};
//moved this routine to a separate function for clarity
const parseCompoundType = (tokens, pos, options) => {
const parseCompoundType = (tokens, pos, scalars, options) => {
//format check (not strictly necessary, but it looks nice)
if (tokens[pos] !== '{') {
throw 'Expected \'{\' in compound type definition';
@@ -69,12 +72,17 @@ const parseCompoundType = (tokens, pos, options) => {
//can't use keywords
if (keywords.includes(type) || keywords.includes(name)) {
throw 'Unexpected keyword found as type field or type name (' + type + ' ' + name + ')';
throw `Unexpected keyword found as type field or type name (${type} ${name})`;
}
//can only use existing types (prevents looping tree structure)
if (!scalars.includes(type)) { //TODO: test this error
throw `Unexpected value found as type field ('${type}' is undefined)`;
}
//check for duplicate fields
if (Object.keys(compound).includes(name)) {
throw 'Unexpected duplicate field name';
throw `Unexpected duplicate field name (${name})`;
}
//finally, push to the compound definition

16
source/index.js
View File

@@ -10,12 +10,12 @@ const sineQL = (schema, handler, options = {}) => {
typeGraph = buildTypeGraph(schema, options);
}
catch(e) {
console.log('Type Graph Error:', e);
console.error('Type Graph Error:', e);
return null;
}
//the receiving function (sine()) - this will be called multiple times
return async (reqBody) => {
return async reqBody => {
try {
//parse the query
const tokens = parseInput(reqBody, true, options);
@@ -32,18 +32,12 @@ const sineQL = (schema, handler, options = {}) => {
//no leading keyword - regular query
default:
const [result, endPos] = await parseQuery(handler, tokens, pos, typeGraph);
//reject the request, despite finishing processing it
if (tokens[endPos]) {
throw 'Unexpected data found at the end of the token list (found ' + tokens[endPos] + ')';
}
return [200, result];
//TODO: implement queries
return [501, 'Queries not implemented'];
}
}
catch(e) {
console.log('Error:', e);
console.error('Error:', e);
return [400, e.stack || e];
}
};

117
source/parse-query.js
View File

@@ -1,117 +0,0 @@
const keywords = require('./keywords.json');
const { eatBlock } = require('./utils');
//returns an object result from handler for all custom types
const parseQuery = async (handler, tokens, pos, typeGraph, parent = null, superMatching = false) => {
//only read past tokens
pos++;
//determine this query's supertype
let superType;
if (!parent) { //top-level
superType = tokens[pos - 1];
}
else if (typeGraph[parent.typeName][ tokens[pos-1] ]) {
superType = typeGraph[parent.typeName][ tokens[pos-1] ].typeName;
}
else {
throw `Missing supertype in type graph (pos = ${pos})`;
}
//error handling
if (!handler[superType]) {
throw 'Unrecognized type ' + superType;
}
if (tokens[pos++] != '{') {
throw 'Expected \'{\' after supertype';
}
//the scalars to pass to the handler - components of the compound types
const scalarFields = [];
const deferredCalls = []; //functions (promises) that will be called at the end of this function
while(tokens[pos++] && tokens[pos - 1] !== '}') { //while not at the end of this block
//check for matching flag
let matching = false;
if (tokens[pos - 1] === 'match') {
matching = true;
pos++;
}
//prevent using keywords
if (keywords.includes(tokens[pos - 1])) {
throw 'Unexpected keyword ' + tokens[pos - 1];
}
//type is a scalar
if (typeGraph[superType] && typeGraph[superType][tokens[pos - 1]] && typeGraph[typeGraph[superType][tokens[pos - 1]].typeName].scalar) {
//push the scalar object to the queryFields
scalarFields.push({ typeName: typeGraph[superType][tokens[pos - 1]].typeName, name: tokens[pos - 1], filter: matching ? tokens[pos++] : null });
//if I am a scalar child of a match and I do not match
if (parent && superMatching && !matching) {
throw 'Broken match chain in scalar type ' + tokens[pos - 1];
}
}
//type is a compound, and must be recursed
else if (typeGraph[superType] && typeGraph[superType][tokens[pos - 1]]) {
const pos2 = pos; //cache the value to keep it from changing
//recurse
deferredCalls.push(async (result) => {
//if I am a compound child of a match amd I do not match
if (parent && superMatching && !matching) {
throw 'Broken match chain in compound type ' + tokens[pos2 - 1];
}
const [queryResult, dummyPos] = await parseQuery(
handler,
tokens,
pos2 - 1,
typeGraph,
{ typeName: superType, scalars: scalarFields, context: result }, //parent object (this one)
matching
);
return [tokens[pos2 - 1], queryResult, matching]; //HACK: match piggybacking on the tuple
});
pos = eatBlock(tokens, pos + 2);
} else {
//token is something else?
throw 'Found something not in the type graph: ' + tokens[pos - 1] + " " + (pos - 1);
}
}
//eat the end bracket
if (tokens[pos - 1] !== '}') {
throw 'Expected \'}\' at the end of query (found ' + tokens[pos - 1] + ')';
}
let results = handler[superType](parent, scalarFields, superMatching);
//WTF: related to the recusion above (turning the results inside out?)
results = await Promise.all(results.map(async res => {
const tuples = await Promise.all(deferredCalls.map(async call => await call(res)));
if (!tuples.every(tuple => !tuple[2] || tuple[1].length > 0)) {
return [];
}
tuples.forEach(tuple => res[tuple[0]] = tuple[1]);
return res;
}));
results = results.filter(r => !Array.isArray(r) || r.length > 0);
return [results, pos];
};
module.exports = parseQuery;

53
test-server-books/database.js
View File

@@ -1,53 +0,0 @@
//the authors
let authors = [
{
name: 'J.K. Roaring',
books: [
{ title: 'The Philosepher\'s Kidney Stone' },
{ title: 'The Chamber Pot of Secrets' },
{ title: 'The Prisoner of Aunt Kazban' },
{ title: 'The Goblet of the Fire Cocktail' },
{ title: 'The Order for Kleenex' },
{ title: 'The Half-Priced Pharmacy' },
{ title: 'Yeah, I Got Nothing' },
]
},
{
name: 'Kenneth Grahame',
books: [
{ title: 'The Wind in the Willows', published: '1 April 1908' }
]
},
{
name: 'Kayne Ruse',
books: [
{ title: 'alpha', published: "1" },
{ title: 'beta', published: "2" },
{ title: 'gamma', published: "3" },
{ title: 'delta', published: "4" },
{ title: 'epsilon', published: "5" },
]
},
];
//insert the authors into the books (relationship)
authors = authors.map(a => {
a.books = a.books.map(b => {
b.authors = [a];
return b;
});
return a;
});
//get the books array
let books = [];
authors.forEach(a => books = books.concat(a.books));
//the fake database
module.exports = {
authors,
books,
};

178
test-server-books/handler.js
View File

@@ -1,178 +0,0 @@
/* DOCS: handler parameter types
parent: Type | null
scalars: [{ typeName: String, name: String, filter: any | null }, ...]
matching: Boolean
*/
//BUG: Book { authors { name } } - this gives a weird result
const database = require('./database.js');
//the handler routines
const handler = {
//type queries
Author: (parent, scalars, matching) => {
let authors;
//check parentage
if (parent) {
//find the author(s) of the parent Book object
authors = database.authors.filter(author => author.books.some(b => b.title == parent.context.title));
} else {
authors = database.authors;
}
//I am being matched (if true, ALL present scalars will have a filter field)
if (matching) {
//check every scalar to every author - a single false match is a miss on that author
authors = authors.filter(author => {
return scalars.every(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return author[scalar.name] == scalar.filter; //dumb comparison for now
//custom handling
//NOTE: Only books used the `Date` scalar
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no authors left, then the book's filters missed matches
if (authors.length == 0) {
return [];
}
}
//scalars are being matched on their own
if (scalars.some(s => s.filter)) {
//check every scalar to every author - a single match is a hit
authors = authors.filter(author => {
return scalars.some(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return author[scalar.name] == scalar.filter; //dumb comparison for now
//custom handling
//NOTE: Only books used the `Date` scalar
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no authors left, then the book's filters missed matches
if (authors.length == 0) {
return [];
}
}
//process (filter out unwanted fields) and return the array of authors
return authors.map(author => {
let ret = {};
//that's a big O(damn)
scalars.forEach(scalar => {
ret[scalar.name] = author[scalar.name];
});
return ret;
});
},
Book: (parent, scalars, matching) => {
let books;
//check parentage
if (parent) {
//find the books of the parent author object
books = database.books.filter(book => book.authors.some(a => a.name == parent.context.name));
} else {
books = database.books;
}
//I am being matched (if true, ALL present scalars will have a filter field)
if (matching) {
//check every scalar to every book - a single false match is a miss on that book
books = books.filter(book => {
return scalars.every(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return book[scalar.name] == scalar.filter; //dumb comparison for now
//custom handling
case 'Date':
return book[scalar.name] == scalar.filter; //could have a more complex comparator function, like date-ranges
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no books left, then the authos's filters missed matches
if (books.length == 0) {
return [];
}
}
//scalars are being matched on their own
if (scalars.some(s => s.filter)) {
//check every scalar to every author - a single match is a hit
books = books.filter(author => {
return scalars.some(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return author[scalar.name] == scalar.filter; //dumb comparison for now
//custom handling
//NOTE: Only books used the `Date` scalar
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no authors left, then the book's filters missed matches
if (books.length == 0) {
return [];
}
}
//process (filter out unwanted fields) and return the array of books
return books.map(book => {
let ret = {};
//that's a big O(damn)
scalars.forEach(scalar => {
ret[scalar.name] = book[scalar.name];
});
return ret;
});
},
};
module.exports = handler;

1879
test-server-books/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

20
test-server-books/package.json
View File

@@ -1,20 +0,0 @@
{
"name": "test-server",
"version": "0.2.1",
"description": "",
"main": "server.js",
"scripts": {
"start": "pm2 start server.js",
"restart": "pm2 restart server.js",
"stop": "pm2 stop server.js",
"list": "pm2 list",
"node": "node server.js"
},
"author": "Kayne Ruse",
"license": "ISC",
"dependencies": {
"cors": "latest",
"express": "latest",
"pm2": "latest"
}
}

14
test-server-books/schema.js
View File

@@ -1,14 +0,0 @@
module.exports = `
scalar Date
type Author {
String name
Book books
}
type Book {
String title
Date published
Author authors
}
`;

27
test-server-books/server.js
View File

@@ -1,27 +0,0 @@
//express for testing
const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser');
const app = express();
app.use(cors());
app.use(bodyParser.text());
//test the library
const schema = require('./schema.js');
const handler = require('./handler.js');
const sineQL = require('../source/index.js');
const sine = sineQL(schema, handler, { debug: false });
//open the end
app.post('/sineql', async (req, res) => {
const [code, result] = await sine(req.body);
res.status(code).send(result);
});
//startup
app.listen(process.env.WEB_PORT || 3100, err => {
console.log(`listening to *:${process.env.WEB_PORT || 3100}`);
});

183
test-server-pokemon/handler.js
View File

@@ -1,183 +0,0 @@
/* DOCS: handler parameter types
parent: Type | null
scalars: [{ typeName: String, name: String, filter: any | null }, ...]
matching: Boolean
*/
const database = require('./pokemon.json');
//the handler routines
const handler = {
Pokemon: (parent, scalars, matching) => {
let pokemon = database;
//if this is a sub-query of Pokemon (a form), use the parent to narrow the search
if (parent && parent.typeName == 'Pokemon') {
//filter based on parent object + scalars
pokemon = pokemon.filter(poke => {
return scalars.every(scalar => poke[scalar.name] == parent.context[scalar.name]);
});
pokemon = pokemon.map(poke => poke.forms)[0];
}
//I am being matched (if true, ALL present scalars will have a filter field)
if (matching) {
console.log('matching');
//check every scalar to every poke - a single false match is a miss on that poke
pokemon = pokemon.filter(poke => {
return scalars.every(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return poke[scalar.name] == scalar.filter; //dumb comparison for now
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no pokemon left, then the filters missed matches
if (pokemon.length == 0) {
return [];
}
}
//scalars are being matched on their own
if (scalars.some(s => s.filter)) {
//check every scalar to every poke - a single match is a hit
pokemon = pokemon.filter(poke => {
return scalars.some(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'String':
case 'Integer':
case 'Float':
case 'Boolean':
return poke[scalar.name] == scalar.filter; //dumb comparison for now
default:
throw `Unknown scalar typeName in handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no pokemon left, then the filters missed matches
if (pokemon.length == 0) {
return [];
}
}
//process (filter out unwanted fields) and return the array of pokemon
return pokemon.map(poke => {
let ret = {};
//that's a big O(damn)
scalars.forEach(scalar => {
ret[scalar.name] = poke[scalar.name];
});
return ret;
});
},
Stats: (parent, scalars, matching) => {
if (!parent || parent.typeName != 'Pokemon') {
throw 'Stats must be inside a Pokemon query';
}
console.log('mark 1');
//skip unknown/empty pokemon stats
let pokemon = database.filter(poke => poke.base_stats != null);
//if this is a sub-query of a Pokemon (already checked), use the parent to narrow the search
pokemon = pokemon.filter(poke => {
return scalars.every(scalar => poke[scalar.name] == parent.context[scalar.name]);
});
console.log('mark 2', pokemon);
//handle forms instead of normal queries
if (pokemon.length == 0) {
pokemon = database.filter(poke => poke.base_stats != null);//skip unknown/empty pokemon stats
pokemon = pokemon.map(p => p.forms);
pokemon = [].concat(...pokemon);
pokemon = pokemon.filter(poke => {
return poke.forms.some(form => {
return scalars.every(scalar => form[scalar.name] == parent.context[scalar.name]);
});
});
}
console.log('mark 3');
//I am being matched (if true, ALL present scalars will have a filter field)
if (matching) {
//check every scalar to every poke - a single false match is a miss on that poke
pokemon = pokemon.filter(poke => {
return scalars.every(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'Integer':
return poke.base_stats[scalar.name] === parseInt(scalar.filter); //dumb comparison for now
default:
throw `Unhandled scalar typeName in Stats handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no pokemon left, then the filters missed matches
if (pokemon.length == 0) {
return [];
}
}
console.log('mark 4');
//scalars are being matched on their own
if (scalars.some(s => s.filter)) {
//check every scalar to every poke - a single match is a hit
pokemon = pokemon.filter(poke => {
return scalars.some(scalar => {
//handle each type of scalar
switch (scalar.typeName) {
case 'Integer':
return poke.base_stats[scalar.name] === parseInt(scalar.filter); //dumb comparison for now
default:
throw `Unhandled scalar typeName in Stats handler: ${scalar.typeName} (${scalar.name})`;
}
});
});
//if there are no pokemon left, then the filters missed matches
if (pokemon.length == 0) {
return [];
}
}
console.log('mark 5');
//process (filter out unwanted fields) and return the array of pokemon
return pokemon.map(poke => {
let ret = {};
//that's a big O(damn)
scalars.forEach(scalar => {
ret[scalar.name] = poke.base_stats[scalar.name];
});
return ret;
});
},
};
module.exports = handler;

1879
test-server-pokemon/package-lock.json generated
View File

File diff suppressed because it is too large Load Diff

20
test-server-pokemon/package.json
View File

@@ -1,20 +0,0 @@
{
"name": "test-server",
"version": "0.2.1",
"description": "",
"main": "server.js",
"scripts": {
"start": "pm2 start server.js",
"restart": "pm2 restart server.js",
"stop": "pm2 stop server.js",
"list": "pm2 list",
"node": "node server.js"
},
"author": "Kayne Ruse",
"license": "ISC",
"dependencies": {
"cors": "latest",
"express": "latest",
"pm2": "latest"
}
}

1
test-server-pokemon/pokemon.json
View File

File diff suppressed because one or more lines are too long

18
test-server-pokemon/schema.js
View File

@@ -1,18 +0,0 @@
module.exports = `
type Pokemon {
String name
Integer height
Integer weight
Stats stats
Pokemon forms
}
type Stats {
Integer hp
Integer attack
Integer defense
Integer specialAttack
Integer specialDefense
Integer speed
}
`;

27
test-server-pokemon/server.js
View File

@@ -1,27 +0,0 @@
//express for testing
const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser');
const app = express();
app.use(cors());
app.use(bodyParser.text());
//test the library
const schema = require('./schema.js');
const handler = require('./handler.js');
const sineQL = require('../source/index.js');
const sine = sineQL(schema, handler, { debug: true });
//open the end
app.post('/sineql', async (req, res) => {
const [code, result] = await sine(req.body);
res.status(code).send(result);
});
//startup
app.listen(process.env.WEB_PORT || 3100, err => {
console.log(`listening to *:${process.env.WEB_PORT || 3100}`);
});