Commit graph

331 commits

Author SHA1 Message Date
324929a02a Update all uses of rusqlite to 0.24 2020-08-24 16:41:17 -04:00
4b1583473e Fix more issues identified by clippy (aka: lint). 2020-08-07 09:15:36 -04:00
125306e108 Update dependencies. Lint. 2020-08-05 23:03:58 -04:00
b428579865 Update dependencies, Rust version 1.44.0-nightly and fix warnings. 2020-04-23 12:23:12 -04:00
Greg Burd
4f81c4e15b Attempting to cleanup with clippy, rustfmt, etc.
Integrate https://github.com/mozilla/mentat/pull/806
2020-01-31 10:55:45 -05:00
Greg Burd
fcb3a9182f Fix module issue found when testing all-features. 2020-01-23 11:16:14 -05:00
Greg Burd
b2f92b8461 Update to 2018 edition of Rust (1.42). Fix and format code. Update dependencies. Fix tests. 2020-01-16 10:58:21 -05:00
Grisha Kruglov
b22b29679b
Basic sync support (#563) r=nalexander
* Pre: remove remnants of 'open_empty'

* Pre: Cleanup 'datoms' table after a timeline move

Since timeline move operations use a transactor, they generate a
"phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense
that it was never present in the 'transactions' table, and is entirely
synthetic as far as our database is concerned.
It's an implementational artifact, and we were not cleaning it up.

It becomes a problem when we start inserting transactions after a move.
Once the transactor clashes with the phantom 'tx', it will retract the
phantom 'txInstant' value, leaving the transactions log in an incorrect state.

This patch adds a test for this scenario and elects the easy way out: simply
remove the offending 'txInstant' datom.

* Part 1: Sync without support for side-effects

A "side-effect" is defined here as a mutation of a remote state as part
of the sync.

If, during a sync we determine that a remote state needs to be changed, bail out.

This generally supports different variations of "baton-passing" syncing, where clients
will succeed syncing if each change is non-conflicting.

* Part 2: Support basic "side-effects" syncing

This patch introduces a concept of a follow-up sync. If a sync generated
a "merge transaction" (a regular transaction that contains assertions
necessary for local and remote transaction logs to converge), then
this transaction needs to be uploaded in a follow-up sync.

Generated SyncReport indicates if a follow-up sync is required.

Follow-up sync itself is just a regular sync. If remote state did not change,
it will result in a simple RemoteFastForward. Otherwise, we'll continue
merging and requesting a follow-up.

Schema alterations are explicitly not supported.

As local transactions are rebased on top of remote, following changes happen:
- entids are changed into tempids, letting transactor upsert :db/unique values
- entids for retractions are changed into lookup-refs if we're confident they'll succeed
-- otherwise, retractions are dropped on the floor

* Post: use a macro for more readable tests

* Tolstoy README
2018-09-07 19:18:20 -07:00
Grisha Kruglov
22b17a6779 Split "mentat transaction" logic away from the main crate
Sync needs to operate over a "mentat transaction", not just a "db transaction".
This shuffle allows internal mentat crates to consume InProgress, which models
the concept of a "mentat transaction".
2018-08-20 18:23:46 -07:00
Grisha Kruglov
b8b2aef181 Pre: Split a Db error for clarity
error_chain stack limitations no longer apply, so let's have better errors!
2018-08-20 18:23:46 -07:00
Nick Alexander
0b84a0802d Pre: Remove open_empty.
This was a work-around for Tolstoy, which couldn't gracefully handle
syncing a store with a bootstrap transaction.  Tolstoy now handles
that single transaction, so this is no longer necessary.
2018-08-20 18:23:46 -07:00
Grisha Kruglov
1e488d720b Post: Use a single implementation of bail macro 2018-08-09 13:16:05 -07:00
Grisha Kruglov
e9398dd50d Part 1: Move public errors into public-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
c00e14f5ff Pre: Remove :: dependency from src/errors.rs 2018-08-09 13:16:05 -07:00
Grisha Kruglov
c8e6a511f4 Pre: Move tolstoy/errors into tolstoy-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
9381af4289 Pre: Move core/Attribute* to core-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
68d0e17824 Pre: Move sql/errors into sql_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
05ef149545 Pre: Fold query-translator into query-projector 2018-08-09 13:16:05 -07:00
Grisha Kruglov
6312e89aba Pre: Move query-projectors/errors and aggregates into query-projector-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
ccdd17551a Pre: Move query-algebrizer/error.rs into query-algebrizer-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
07beb68c7a Pre: Remove query/ crate 2018-08-09 13:16:05 -07:00
Grisha Kruglov
11aaa193f5 Pre: Move query-pull/errors into query-pull-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
cebb85a7fe Pre: Move db/errors.rs into db_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
d0214fad7d Pre: Move core/types.rs into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
a57ba5d79f Pre: Move Entid and KnownEntid into core_traits 2018-08-09 13:16:05 -07:00
Nick Alexander
9e6505a930 [sdks/android] Pre: Fix unused warnings. 2018-07-25 20:38:48 -07:00
Grisha Kruglov
6290cc9db2 Enforce partition integrity when setting its index r=nalexander
Timelines work starts to perform modifications on the partitions
that go beyond simple allocations. This change pre-emptively protects
partition integrity by asserting that index modifications are legal.
2018-07-17 06:20:37 -07:00
Grisha Kruglov
bff24c60b7
Add a top-level "syncable" feature. (#782) r=ncalexan
* Add a top-level "syncable" feature.

Tested with:

cargo test --all
cargo test --all --no-default-features
cargo build --manifest-path tools/cli/Cargo.toml --no-default-features
cargo run --manifest-path tools/cli/Cargo.toml --no-default-features debugcli

Co-authored-by: Nick Alexander <nalexander@mozilla.com>

* Add 'syncable' feature to 'db' crate to conditionally derive serialization for Partition*

This is leading up to syncing with partition support.
2018-07-11 16:26:06 -07:00
Nick Alexander
f65512158b Make kw!(foo.bar/bar.baz) work. 2018-07-06 14:19:50 -07:00
Nick Alexander
2cb7d441dc Part 2: Make it easier to match tuple results.
Right now, we write code like
```rust
match q_once(q, inputs)?.into_tuple()? {
    Some(vs) => match (vs.len(), vs.get(0), vs.get(1)) {
        (2, &Some(Binding::Scalar(TypedValue::Long(a))), &Some(Binding::Scalar(TypedValue::Instant(ref b)))) => Some((a, b.clone())),
        _ => panic!(),
    },
    None => None,
}
```
to length-check tuples coming out of the database.  It can also lead
to a lot of cloning because references are the easiest thing to hand.

This commit allows to write code like
```rust
match q_once(q, inputs)?.into_tuple()? {
    Some((Binding::Scalar(TypedValue::Long(a)), Binding::Scalar(TypedValue::Instant(b)))) => Some((a, b)),
    Some(_) => panic!(),
    None => None,
}
```
which is generally much easier to reason about.
2018-07-05 16:45:42 -07:00
Nick Alexander
2ab481f83e Part 2: Expose time related things at top-level.
Perhaps we actually want to subdivide the top-level namespace so that
there is a `mentat::time` module, but I'd prefer to make part of the
process of fixing the public API as we get ready to christen version
1.0.
2018-07-05 16:45:42 -07:00
Nick Alexander
3744982cd9 Add last_tx_id. 2018-07-05 16:45:42 -07:00
Nick Alexander
d49f702512 Part 1: Expand Binding::val() into Binding::{into_*,as_*}.
This is simply for completeness: we should provide fundamental
conversion patterns even when they are mostly unused in our code base.
2018-07-05 16:45:42 -07:00
Nick Alexander
eb1df31ac4 Part 7: Improve TermBuilder interface; expose lookup refs and tx functions.
These are functions on `TermBuilder` itself to prevent mixing mutable
and immutable references in the most natural style.  That is,
```
builder.add(e, a, builder.lookup_ref(...))
```
fails because `add` borrows `builder` mutably and `lookup_ref` borrows
`builder` immutably.  There's nothing here that requires a specific
builder (since we're not interning lookup refs on the builder, like we
are tempids) so we don't need an instance.
2018-07-05 16:33:51 -07:00
Nick Alexander
06056a8468 Part 6: Lift TxReport to core crate.
The `core` create didn't exist when the `db` was started, but this
type is clearly part of the public interface of Mentat.
2018-07-05 16:33:51 -07:00
Nick Alexander
1cb1847aa6 Part 5: Make existing TermBuilder actually build Entity instances.
There are a few tricky details to call out here.  The first is the
`TransactableValueMarker` trait.  This is strictly a marker (like
`Sized`, for example) to give some control over what types can be used
as value types in `Entity` instances.  This expression is needed due
to the network of `Into` and `From` relations between the parts of
valid `Entity` instances.  This allows to drop the `IntoThing`
work-around trait and use the established patterns.  (Observe that
`KnownEntid` makes this a little harder, due to the cross-crate
consistency restrictions.)

The second is that we can get rid `{add,retract}_kw`, since the
network of relations expresses the coercions directly.

The third is that this commit doesn't change the name `TermBuilder`,
even though it is now building `Entity` instances.  This is because
there's _already_ an `EntityBuilder` which fixes the `EntityPlace`.
It's not clear whether the existing entity building interface should
be removed or whether both should be renamed.  That can be follow-up.
2018-07-05 16:33:51 -07:00
Nick Alexander
87f850a44e Part 1: Move intern_set into edn crate.
It's not great to keep lifting functionality higher and higher up the
crate hierarchy, but we really do want to intern while we parse.
Eventually, I expect that we will split the `edn` crate into `types`
and `parsing`, and the `types` crate can depend on a more efficient
interning dependency.
2018-07-05 11:16:48 -07:00
Nick Alexander
d82c7f8ef2 Cull unused mentat_parser_utils crate.
With the transition toward parsing with `rust-peg` and away from
`combine`, we're not using some of the many helpers we built to
support our unusual `combine` usage.  They can just go!
2018-06-30 16:21:50 -07:00
Nick Alexander
8725bad18c Pre: Fix error printing rusqlite::Error. 2018-06-30 14:58:23 -07:00
Grisha Kruglov
8af5288a60 Use TolstoyError for tolstoy's Results; wrap tolstoy's dependency errors r=nalexander
This is inline with the rest of mentat, and helps with upcoming tolstoy work.
2018-06-29 00:47:19 -04:00
Nick Alexander
ae427849d5 Expose sub-crate *Error types at top-level.
We're not exposing a uniform API with `mentat::Result` yet, meaning
that early consumers (e.g., the logins work for Mozilla Lockbox) need
to wrap errors from all over the Mentat crate hierarchy.
2018-06-27 15:05:43 -07:00
Nick Alexander
d31ec28aa8 Patch it all together: use MentatError at top-level.
I elected to keep Tolstoy using `failure::Error`, because Tolstoy
looks rather more like a high-level application (and will continue to
do so for a while) than a production-ready mid- or low-level API.
2018-06-27 15:05:43 -07:00
Nick Alexander
0e4991fa26 Make db/ use DbErrorKind. 2018-06-27 15:05:43 -07:00
Thom
72a9b302f9
Rename or delete things so that there is only one type named Entid (#768)
* Delete the (apparently unused) EntId

* Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64

* Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 16:34:18 -07:00
Grisha Kruglov
4e46adeba1 Convert tolstoy/ to failure. 2018-06-20 14:42:36 -07:00
Grisha Kruglov
4e01929334 Convert src/ to failure. 2018-06-20 14:42:18 -07:00
Thom Chiovoloni
99a73ccb03 Avoid using 1.26.0-only features when using sqlcipher, and move the sqlcipher Store support to the correct file 2018-06-13 22:48:28 -07:00
Emily Toop
8e918949fb Separate Store from Conn.
This is a Pre: part extracted from #660.
2018-06-13 15:29:11 -07:00
Thom
6a1a265894
Add support for using sqlcipher (#737). Fixes #118 2018-06-13 08:49:40 -07:00
Nick Alexander
e68cc4016c Part 7: Remove tx entirely.
This was left over from #681.
2018-06-04 15:04:39 -07:00