This was a little more tricky than might be expected because the
initialization process uses the transactor to bootstrap the database.
Since Clojure doesn't accept mutually recursive modules, this
necessitated a third module, namely "db-factory", which uses both "db"
and "transact". While I was here, I started an "api" module, to paper
over the potentially complicated internal module structure for external
consumers. In time, this "api" module may also grow CLJS-specific JS
transformations.
This agrees with Datomic. DataScript allows tx values, possibly to
allow reconstructing DBs from Datom streams, but appears to handle
user-provided tx values in the transactor inconsistently.
The implementation of :db/tx is special and may need to change over
time. We add it as a special ident, with value the current transaction
entity ID, specified per-transaction. This works well right now but
introduces some (internal) ordering requirements that may need to be
loosened.
Internally, we use SQLite's FTS4 to maintain a fulltext_values table of
unique "text" values. Fulltext indexed datoms have value v that is the
rowid into fulltext_values. We manually maintain the map between rowid
and value in the transactor.
For convenience, we expose two views interpolating the real text values
into the datoms structure.
This version includes SQLite-level unique indexes; these should never be
needed. I've included them as a fail-safe while testing; they'll help
us catch errors in the transaction layer above.
In the future, we might add a layer of indirection, hashing values to
avoid duplicating storage, or sorting URLs, or handling fulltext indexed
values differently, or ...
Some of these were just typos, but `with-open` was fatally flawed on
CLJS (we couldn't call `.close` at all), and `deftest-async` was hiding
all failures (due to a typo).
We would prefer to talk about a knowledge base on top of a database, but
all the Datomic and DataScript code (and symbols, like :db/add, etc)
refer to the "database of datoms", so let's roll with that nomenclature
and try to be specific that the persistent storage-layer is SQLite.
This will become more clear when we actually use SQLite's unique
capabilities for text indexing.
This is a well-worn idea: use a `promise-channel` of `[result nil]` or
`[nil error]` pairs. The `go-pair` and `<?` macros handle catching
exceptions (important, given that synchronous CLJ code expects to throw
rather than return an error promise or similar), allowing code like:
```
(go-pair
(let [result (<? (pair-chan-fn))]
(when (not result)
(throw (Exception. "No result!")))
(transform result)))
```
to be expressed naturally. These are the equivalents of `async` and
`await` in JS.
The implementation is complicated by significant incompatibilities
between CLJ and CLJS. The solution presented here takes care to
separate the macro definitions into CLJ. Sadly, this requires
namespacing the per-environment symbols explicitly; but we hope to
minimize such code in files like this.
The most significant restriction to this approach is that consumers must
require the transitive dependencies of the macro-defining modules. See
the included tests (both CLJ and CLJS) for the appropriate
incantations (for pair-chan, core.async, and test).
