Commit graph

9 commits

Author SHA1 Message Date
Grisha Kruglov
c8e6a511f4 Pre: Move tolstoy/errors into tolstoy-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
d0214fad7d Pre: Move core/types.rs into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
a57ba5d79f Pre: Move Entid and KnownEntid into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
bff24c60b7
Add a top-level "syncable" feature. (#782) r=ncalexan
* Add a top-level "syncable" feature.

Tested with:

cargo test --all
cargo test --all --no-default-features
cargo build --manifest-path tools/cli/Cargo.toml --no-default-features
cargo run --manifest-path tools/cli/Cargo.toml --no-default-features debugcli

Co-authored-by: Nick Alexander <nalexander@mozilla.com>

* Add 'syncable' feature to 'db' crate to conditionally derive serialization for Partition*

This is leading up to syncing with partition support.
2018-07-11 16:26:06 -07:00
Nick Alexander
46c2a0801f Add type checking and constraint checking to the transactor. (#663, #532, #679)
This should address #663, by re-inserting type checking in the
transactor stack after the entry point used by the term builder.

Before this commit, we were using an SQLite UNIQUE index to assert
that no `[e a]` pair, with `a` a cardinality one attribute, was
asserted more than once.  However, that's not in line with Datomic,
which treats transaction inputs as a set and allows a single datom
like `[e a v]` to appear multiple times.  It's both awkward and not
particularly efficient to look for _distinct_ repetitions in SQL, so
we accept some runtime cost in order to check for repetitions in the
transactor.  This will allow us to address #532, which is really about
whether we treat inputs as sets.  A side benefit is that we can
provide more helpful error messages when the transactor does detect
that the input truly violates the cardinality constraints of the
schema.

This commit builds a trie while error checking and collecting final
terms, which should be fairly efficient.  It also allows a simpler
expression of input-provided :db/txInstant datoms, which in turn
uncovered a small issue with the transaction watcher, where-by the
watcher would not see non-input-provided :db/txInstant datoms.

This transition to Datomic-like input-as-set semantics allows us to
address #532.  Previously, two tempids that upserted to the same entid
would produce duplicate datoms, and that would have been rejected by
the transactor -- correctly, since we did not allow duplicate datoms
under the input-as-list semantics.  With input-as-set semantics,
duplicate datoms are allowed; and that means that we must allow
tempids to be equivalent, i.e., to resolve to the same tempid.

To achieve this, we:
- index the set of tempids
- identify tempid indices that share an upsert
- map tempids to a dense set of contiguous integer labels

We use the well-known union-find algorithm, as implemented by
petgraph, to efficiently manage the set of equivalent tempids.

Along the way, I've fixed and added tests for two small errors in the
transactor.  First, don't drop datoms resolved by upsert (#679).
Second, ensure that complex upserts are allocated.

I don't know quite what happened here.  The Clojure implementation
correctly kept complex upserts that hadn't resolved as complex
upserts (see
9a9dfb502a/src/common/datomish/transact.cljc (L436))
and then allocated complex upserts if they didn't resolve (see
9a9dfb502a/src/common/datomish/transact.cljc (L509)).

Based on the code comments, I think the Rust implementation must have
incorrectly tried to optimize by handling all complex upserts in at
most a single generation of evolution, and that's just not correct.
We're effectively implementing a topological sort, using very specific
domain knowledge, and its not true that a node in a topological sort
can be considered only once!
2018-05-14 15:22:45 -07:00
Richard Newman
ca451a7c9c Silence a warning in Tolstoy. 2018-04-03 14:01:29 -07:00
Grisha Kruglov
93e5dff9c8
Revised uploader flow (battle-tested); CLI support for sync (#557) r=rnewman 2018-02-16 01:44:28 -08:00
Grisha Kruglov
84f29676e8
"Unchanged server" uploader flow (#543) r=rnewman
* Remove unused struct from tx_processor

* Derive serialize & deserialize for TypedValue

* First pass of uploader flow + feedback
2018-02-09 09:55:19 -08:00
Grisha Kruglov
d848d954cf Issue 508 - Iterating transcation processor r=rnewman
Review comments
2018-02-06 12:24:12 -08:00