Commit graph

86 commits

Author SHA1 Message Date
125306e108 Update dependencies. Lint. 2020-08-05 23:03:58 -04:00
5899bf8624 Minor version adjustments and fixes. 2020-05-25 10:51:22 -04:00
bf1ac14d32 Update dependency versions. Fix minor warnings. 2020-05-12 10:21:51 -04:00
b428579865 Update dependencies, Rust version 1.44.0-nightly and fix warnings. 2020-04-23 12:23:12 -04:00
Greg Burd
4f81c4e15b Attempting to cleanup with clippy, rustfmt, etc.
Integrate https://github.com/mozilla/mentat/pull/806
2020-01-31 10:55:45 -05:00
Greg Burd
b2f92b8461 Update to 2018 edition of Rust (1.42). Fix and format code. Update dependencies. Fix tests. 2020-01-16 10:58:21 -05:00
Conrad Dean
3d965fdf6e try fixing build by upgrading rusqlite to 0.19 2019-07-17 10:59:38 -04:00
Grisha Kruglov
b22b29679b
Basic sync support (#563) r=nalexander
* Pre: remove remnants of 'open_empty'

* Pre: Cleanup 'datoms' table after a timeline move

Since timeline move operations use a transactor, they generate a
"phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense
that it was never present in the 'transactions' table, and is entirely
synthetic as far as our database is concerned.
It's an implementational artifact, and we were not cleaning it up.

It becomes a problem when we start inserting transactions after a move.
Once the transactor clashes with the phantom 'tx', it will retract the
phantom 'txInstant' value, leaving the transactions log in an incorrect state.

This patch adds a test for this scenario and elects the easy way out: simply
remove the offending 'txInstant' datom.

* Part 1: Sync without support for side-effects

A "side-effect" is defined here as a mutation of a remote state as part
of the sync.

If, during a sync we determine that a remote state needs to be changed, bail out.

This generally supports different variations of "baton-passing" syncing, where clients
will succeed syncing if each change is non-conflicting.

* Part 2: Support basic "side-effects" syncing

This patch introduces a concept of a follow-up sync. If a sync generated
a "merge transaction" (a regular transaction that contains assertions
necessary for local and remote transaction logs to converge), then
this transaction needs to be uploaded in a follow-up sync.

Generated SyncReport indicates if a follow-up sync is required.

Follow-up sync itself is just a regular sync. If remote state did not change,
it will result in a simple RemoteFastForward. Otherwise, we'll continue
merging and requesting a follow-up.

Schema alterations are explicitly not supported.

As local transactions are rebased on top of remote, following changes happen:
- entids are changed into tempids, letting transactor upsert :db/unique values
- entids for retractions are changed into lookup-refs if we're confident they'll succeed
-- otherwise, retractions are dropped on the floor

* Post: use a macro for more readable tests

* Tolstoy README
2018-09-07 19:18:20 -07:00
Grisha Kruglov
22b17a6779 Split "mentat transaction" logic away from the main crate
Sync needs to operate over a "mentat transaction", not just a "db transaction".
This shuffle allows internal mentat crates to consume InProgress, which models
the concept of a "mentat transaction".
2018-08-20 18:23:46 -07:00
Grisha Kruglov
db4350aab7 Bump version to 0.11.1 2018-08-09 13:16:05 -07:00
Grisha Kruglov
e9398dd50d Part 1: Move public errors into public-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
c8e6a511f4 Pre: Move tolstoy/errors into tolstoy-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
68d0e17824 Pre: Move sql/errors into sql_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
05ef149545 Pre: Fold query-translator into query-projector 2018-08-09 13:16:05 -07:00
Grisha Kruglov
6312e89aba Pre: Move query-projectors/errors and aggregates into query-projector-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
ccdd17551a Pre: Move query-algebrizer/error.rs into query-algebrizer-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
07beb68c7a Pre: Remove query/ crate 2018-08-09 13:16:05 -07:00
Grisha Kruglov
11aaa193f5 Pre: Move query-pull/errors into query-pull-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
cebb85a7fe Pre: Move db/errors.rs into db_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
a57ba5d79f Pre: Move Entid and KnownEntid into core_traits 2018-08-09 13:16:05 -07:00
Nick Alexander
65e9822ad6 Bump to version 0.11.0. 2018-07-31 09:59:18 -07:00
Nick Alexander
faef4e9ee8 Bump to version 0.10.0. 2018-07-26 13:09:18 -07:00
Nick Alexander
67a14ca756 [sdks/android] Build Mentat Android SDK in TaskCluster; publish org.mozilla.mentat to nalexander's personal Bintray repo.
The automation parts were cribbed directly from
50add3e176.

The automation permissions were added in
https://bugzilla.mozilla.org/show_bug.cgi?id=1477311.

This uses a very rudimentary Gradle plugin, `rust-android-gradle`,
with custom fixes and extensions.  It works pretty well for what it
is!  See https://github.com/ncalexan/rust-android-gradle.
2018-07-25 20:50:44 -07:00
Grisha Kruglov
bff24c60b7
Add a top-level "syncable" feature. (#782) r=ncalexan
* Add a top-level "syncable" feature.

Tested with:

cargo test --all
cargo test --all --no-default-features
cargo build --manifest-path tools/cli/Cargo.toml --no-default-features
cargo run --manifest-path tools/cli/Cargo.toml --no-default-features debugcli

Co-authored-by: Nick Alexander <nalexander@mozilla.com>

* Add 'syncable' feature to 'db' crate to conditionally derive serialization for Partition*

This is leading up to syncing with partition support.
2018-07-11 16:26:06 -07:00
Nick Alexander
f65512158b Make kw!(foo.bar/bar.baz) work. 2018-07-06 14:19:50 -07:00
Nick Alexander
07c5d733d6 Bump to version 0.8.0.
We've made many breaking changes, especially to error handling, so
it's time to bump versions.
2018-07-05 16:48:27 -07:00
Nick Alexander
d82c7f8ef2 Cull unused mentat_parser_utils crate.
With the transition toward parsing with `rust-peg` and away from
`combine`, we're not using some of the many helpers we built to
support our unusual `combine` usage.  They can just go!
2018-06-30 16:21:50 -07:00
Grisha Kruglov
4e01929334 Convert src/ to failure. 2018-06-20 14:42:18 -07:00
Thom
6a1a265894
Add support for using sqlcipher (#737). Fixes #118 2018-06-13 08:49:40 -07:00
Nick Alexander
e68cc4016c Part 7: Remove tx entirely.
This was left over from #681.
2018-06-04 15:04:39 -07:00
Nick Alexander
d4166cc67c Part 6: Remove query-parser entirely. 2018-06-04 15:04:39 -07:00
Nick Alexander
7a8c9d90c2 Post: Remove tx-parser crate entirely. 2018-05-10 10:24:05 -07:00
Richard Newman
e21156a754
Implement simple pull expressions (#638) r=nalexander
* Refactor AttributeCache populator code for use from pull.

* Pre: add to_value_rc to Cloned.

* Pre: add From<StructuredMap> for Binding.

* Pre: clarify Store::open_empty.

* Pre: StructuredMap cleanup.

* Pre: clean up a doc test.

* Split projector crate. Pass schema to projector.

* CLI support for printing bindings.

* Add and use ConjoiningClauses::derive_types_from_find_spec.

* Define pull types.

* Implement pull on top of the attribute cache layer.

* Add pull support to the projector.

* Parse pull expressions.

* Add simple pull support to connection objects.

* Tests for pull.

* Compile with Rust 1.25.

The only choice involved in this commit is that of replacing the
anonymous lifetime '_ with a named lifetime for the cache; since we're
accepting a Known, which includes the cache in question, I think it's
clear that we expect the function to apply to any given cache
lifetime.

* Review comments.

* Bail on unnamed attribute.

* Make assert_parse_failure_contains safe to use.

* Rework query parser to report better errors for pull.

* Test for mixed wildcard and simple attribute.
2018-05-04 12:56:00 -07:00
Richard Newman
a2e13f624c
Add 'Binding', a structured value type to return from queries. (#657) r=nalexander
Bump to 0.7: breaking change.
2018-04-24 15:08:38 -07:00
Nick Alexander
36eca0bfb0 (chore) Pre: Use the same features of uuid throughout the project. 2018-04-19 09:58:41 -07:00
Richard Newman
39f1d61175 Bump version to 0.6.2. 2018-04-09 09:47:49 -07:00
Emily Toop
9f30fe6295 Create Mentat FFI and expose observers (#574)
* Tidy up and add txid at beginning of transaction

* Add ffi crate and new_store function

* Add register and unregister observer FFI, Store and Conn functions.
Also add android logging facilities

* Add function for fetching entids for attribute strings

* Add functions for iterating through TxReports

* Add sync to ffi boundary

* Move Extern types from submodule to lib in FFI.
For some reason, if these types are in a submodule, even if they are publically used, the functions inside the FFI are not found in
Android. Works for iOS though. To be investigated later....

* Return to passing TxReports to observer function.
Also, remove some debug

* Expose DateTime and Utc publically

* Use Store in observer tests
2018-03-20 19:16:32 +00:00
Emily Toop
ab957948b4 Move to using watcher.
Simplify.

This has a watcher collect txid -> AttributeSet mappings each time a
transact occurs. On commit we retrieve those mappings and hand them over
to the observer service, which filters them and packages them up for
dispatch.

Tidy up
2018-03-20 16:27:35 +00:00
Emily Toop
d4365fa4cd Execute commands in a separate thread
Command Queue Executor to watch for new commands and execute on longer running background thread
2018-03-20 16:27:35 +00:00
Richard Newman
0ca61017a1 Use LTO for release builds. 2018-03-06 08:19:17 -08:00
Richard Newman
4a0b67ab50 Require Rust 0.1.24 or higher. 2018-03-06 08:19:17 -08:00
Richard Newman
30bf827d16
CLI improvements (#577) r=grisha
* Add a prepared query command to CLI.
* Print nanoseconds in the REPL. This is a good problem to have.
* Better CLI timing.
* Use release for 'cargo cli', debug for 'cargo debugcli'.
* Don't enable debug symbols in release builds.
* Clean up CLI code. Fixed order for help.
* Column-align help output.
2018-03-05 12:52:20 -08:00
Richard Newman
5e50d2a9b4 Update to rusqlite 0.13. 2018-02-22 11:41:57 -08:00
Grisha Kruglov
93e5dff9c8
Revised uploader flow (battle-tested); CLI support for sync (#557) r=rnewman 2018-02-16 01:44:28 -08:00
Richard Newman
2ac7a1b1de Add a feature flag to control the use of rusqlite's bundled SQLite. r=emily
You can use this in conjunction with setting SQLITE3_LIB_DIR to control which SQLite is used.

See https://github.com/jgallagher/rusqlite for more.

Also add recent contributors to the authors array.
2018-02-13 08:25:58 -08:00
Richard Newman
66e6fef75e Define Store, use TabWriter in the CLI for aligning columnar output. (#540) r=emily
* Define Store, which is a simple container for a SQLite connection and a Conn.
  This is a breaking change.
* Return the FindSpec as part of QueryOutput, not just results.
* Switch to using stderr in appropriate places in CLI.
* Print columns in CLI output.
2018-02-01 09:29:07 -08:00
Richard Newman
6ed5413cd4 Implement simple vocabulary management. (#504) r=emily,nalexander
Bump version to 0.5.1 to reflect this change.
2018-01-23 08:52:15 -08:00
Richard Newman
812f10b3e4 Add an EntityBuilder abstraction. r=nalexander,emily
This includes two other changes:

* Split transact to expose an interface for TermWithTempIds.
* Return TxReport from each InProgress operation, not from commit.
2018-01-23 08:52:09 -08:00
Richard Newman
224570fb45 Switch InProgress to be mutated in place. r=nalexander,emily
This is a breaking change, and involves a very small additional cost
in managing the partition map, but it makes it much more feasible to
implement traits on InProgress: now they don't need to chain back a
new InProgress each time.

Bump version to 0.5 to reflect the change in InProgress.
2018-01-23 08:14:13 -08:00
Grisha Kruglov
c61bc79b99 Sync metadata schema and SyncMetadataClient. (#502) r=rnewman 2017-12-13 14:19:05 -06:00