Note
This is one of 199 standalone projects, maintained as part of the @thi.ng/umbrella monorepo and anti-framework.
🚀 Please help me to work full-time on these projects by sponsoring me on GitHub. Thank you! ❤️
@thi.ng/rstream based triple store & reactive query engine with declarative query specs related to Datalog / SPARQL. Inserted triples / facts are broadcast to multiple indexing streams and any query subscriptions attached to them. This enables push-based, auto-updating query results, which are changing each time upstream transformations & filters have been triggered.
Triples are 3-tuples of
[subject, predicate, object]. Unlike with traditional
RDF
triple stores, any JS data types can be used as subject, predicate or
object (though support for such must be explicitly enabled & this
feature is currently WIP).
- Dynamic & declarative dataflow graph construction via high-level data specs and/or functions
- Entirely based on stream abstractions provided by @thi.ng/rstream
- All data transformations done using dynamically composed tranducers
- Query optimizations
- Extensive re-use of existing sub-query results (via subscriptions)
- Interim result de-duplication / dataflow gates
- Push-based, auto-updating query results
STABLE - used in production
Search or submit any issues for this package
This project is currently still in early development and intended as a continuation of the Clojure based thi.ng/fabric, this time built on the streaming primitives provided by @thi.ng/rstream.
yarn add @thi.ng/rstream-queryESM import:
import * as rsq from "@thi.ng/rstream-query";Browser ESM import:
<script type="module" src="https://esm.run/@thi.ng/rstream-query"></script>For Node.js REPL:
const rsq = await import("@thi.ng/rstream-query");Package sizes (brotli'd, pre-treeshake): ESM: 2.58 KB
- @thi.ng/api
- @thi.ng/associative
- @thi.ng/checks
- @thi.ng/equiv
- @thi.ng/errors
- @thi.ng/logger
- @thi.ng/math
- @thi.ng/rstream
- @thi.ng/rstream-dot
- @thi.ng/transducers
Note: @thi.ng/api is in most cases a type-only import (not used at runtime)
One project in this repo's /examples directory is using this package:
| Screenshot | Description | Live demo | Source |
|---|---|---|---|
 |
Triple store query results & sortable table | Demo | Source |
import { TripleStore, asTriples } from "@thi.ng/rstream-query";
import { trace } from "@thi.ng/rstream";
// create store with initial set of triples / facts
const store = new TripleStore([
["london", "type", "city"],
["london", "part-of", "uk"],
["portland", "type", "city"],
["portland", "partOf", "oregon"],
["portland", "partOf", "usa"],
["oregon", "type", "state"],
["usa", "type", "country"],
["uk", "type", "country"],
]);
// alternatively, convert an object into a sequence of triples
const store = new TripleStore(asTriples({
london: {
type: "city",
partOf: "uk"
},
portland: {
type: "city",
partOf: ["oregon", "usa"]
},
oregon: { type: "state" },
uk: { type: "country" },
usa: { type: "country" },
});
// compile the below query spec into a dataflow graph
// pattern items prefixed w/ "?" are query variables
// this query matches the following relationships
// using all currently known triples in the store
// when matching triples are added or removed, the query
// result updates automatically...
// currently only "where" and bounded "path" sub-queries are possible
// in the near future, more query types will be supported
// (e.g. optional relationships, pre/post filters etc.)
store.addQueryFromSpec({
q: [
{
// all "where" subqueries are joined (logical AND)
where: [
// match any subject of type "city"
["?city", "type", "city"],
// match each ?city var's "part-of" relationships (if any)
["?city", "partOf", "?country"],
// matched ?country var must have type = "country"
["?country", "type", "country"]
]
}
],
// `bind` is an (optional) query post-processor and
// allows injection of new variables into the result set
// here we create a new var "answer" whose values are derived from
// the other two query vars
bind: {
answer: (res) => `${res.city} is located in ${res.country}`
},
// another post-processing step, only keeps "answer" var in results
select: ["answer"]
})
.subscribe(trace("results"))
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' } }
// helper fn to insert new city relationship to the store
const addCity = (name, country) =>
store.into([
[name, "type", "city"],
[name, "partOf", country],
[country, "type", "country"],
]);
addCity("berlin", "germany");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' } }
addCity("paris", "france");
// results Set {
// { answer: 'london is located in uk' },
// { answer: 'portland is located in usa' },
// { answer: 'berlin is located in germany' },
// { answer: 'paris is located in france' } }After setting up the above query and its internal transformations, the generated dataflow topology then looks as follows:
- The blue nodes are
TripleStore-internal index stream sources, emitting changes when new triples are added - The left set of red nodes are the sub-queries of the above
whereclause, responsible for joining the individual (S)ubject, (P)redicate and (O)bject sub-queries. - The results of these are then further joined (right red node) & transformed to produce the final solution set and post-process it
Btw. The diagram has been generated using
@thi.ng/rstream-dot
and can be recreated by calling store.toDot() (for the above example)
The source code for the above example is here
(Many) more features forthcoming...
If this project contributes to an academic publication, please cite it as:
@misc{thing-rstream-query,
title = "@thi.ng/rstream-query",
author = "Karsten Schmidt",
note = "https://thi.ng/rstream-query",
year = 2018
}© 2018 - 2024 Karsten Schmidt // Apache License 2.0