Commit graph

1001 commits

Author SHA1 Message Date
Grisha Kruglov
b22b29679b
Basic sync support (#563) r=nalexander
* Pre: remove remnants of 'open_empty'

* Pre: Cleanup 'datoms' table after a timeline move

Since timeline move operations use a transactor, they generate a
"phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense
that it was never present in the 'transactions' table, and is entirely
synthetic as far as our database is concerned.
It's an implementational artifact, and we were not cleaning it up.

It becomes a problem when we start inserting transactions after a move.
Once the transactor clashes with the phantom 'tx', it will retract the
phantom 'txInstant' value, leaving the transactions log in an incorrect state.

This patch adds a test for this scenario and elects the easy way out: simply
remove the offending 'txInstant' datom.

* Part 1: Sync without support for side-effects

A "side-effect" is defined here as a mutation of a remote state as part
of the sync.

If, during a sync we determine that a remote state needs to be changed, bail out.

This generally supports different variations of "baton-passing" syncing, where clients
will succeed syncing if each change is non-conflicting.

* Part 2: Support basic "side-effects" syncing

This patch introduces a concept of a follow-up sync. If a sync generated
a "merge transaction" (a regular transaction that contains assertions
necessary for local and remote transaction logs to converge), then
this transaction needs to be uploaded in a follow-up sync.

Generated SyncReport indicates if a follow-up sync is required.

Follow-up sync itself is just a regular sync. If remote state did not change,
it will result in a simple RemoteFastForward. Otherwise, we'll continue
merging and requesting a follow-up.

Schema alterations are explicitly not supported.

As local transactions are rebased on top of remote, following changes happen:
- entids are changed into tempids, letting transactor upsert :db/unique values
- entids for retractions are changed into lookup-refs if we're confident they'll succeed
-- otherwise, retractions are dropped on the floor

* Post: use a macro for more readable tests

* Tolstoy README
2018-09-07 19:18:20 -07:00
Nick Alexander
64821079c2
Update README.md to mark Mentat as unmaintained.
See https://mail.mozilla.org/pipermail/firefox-dev/2018-September/006780.html.
2018-09-07 14:37:50 -07:00
Grisha Kruglov
bcec011ca5 Make sure double retractions are not inserted. Fixes #818. (#819) r=nalexander 2018-09-07 13:12:28 -07:00
sc13-bioinf
fba9568d44 Allow plus symbol "+" in symbol names. (#821) r=nalexander 2018-09-05 09:28:32 -07:00
Emily Toop
e3113783ae Fix merge error on iOS automation patch 2018-08-22 16:44:45 +01:00
Emily Toop
cd99774e2c Adding iOS Build and Test to CI (#804)
* Add iOS SDK build and test to rust 1.25.0 version of travis CI build

* Address review comments

* Move iOS testing and document generation into post test jobs
2018-08-22 08:43:17 -07:00
Grisha Kruglov
66cc4e14ad Post: use dirs crate, avoiding compile warning about home_dir 2018-08-20 18:23:46 -07:00
Grisha Kruglov
22b17a6779 Split "mentat transaction" logic away from the main crate
Sync needs to operate over a "mentat transaction", not just a "db transaction".
This shuffle allows internal mentat crates to consume InProgress, which models
the concept of a "mentat transaction".
2018-08-20 18:23:46 -07:00
Grisha Kruglov
6160dd59f7 Pre: use 'db/syncable' feature; derive serialization for PartitionMap 2018-08-20 18:23:46 -07:00
Grisha Kruglov
b8b2aef181 Pre: Split a Db error for clarity
error_chain stack limitations no longer apply, so let's have better errors!
2018-08-20 18:23:46 -07:00
Grisha Kruglov
5bc6d76bb3 Pre: expose read_partition_map from the db crate 2018-08-20 18:23:46 -07:00
Nick Alexander
8c2245ff0b Pre: Add top-level NotYetImplemented error. 2018-08-20 18:23:46 -07:00
Nick Alexander
0b84a0802d Pre: Remove open_empty.
This was a work-around for Tolstoy, which couldn't gracefully handle
syncing a store with a bootstrap transaction.  Tolstoy now handles
that single transaction, so this is no longer necessary.
2018-08-20 18:23:46 -07:00
Grisha Kruglov
8ddbd18f5f
Add travis-ci build status badge to README. 2018-08-20 17:56:49 -07:00
Grisha Kruglov
9e8292e68b Allow 'sqlcipher' feature for all uses of rusqlite
This also patches our CI test script to only run "--feature sqlcipher"
tests on sub-crates which expose this feature (i.e. themselves rely on rusqlite).
2018-08-20 16:55:34 -07:00
Emily Toop
fe1a034822 Fix broken iOS tests 2018-08-20 14:40:39 -07:00
Emily Toop
d61e070e08 Get iOS tests building again. 2018-08-20 14:40:39 -07:00
Grisha Kruglov
db4350aab7 Bump version to 0.11.1 2018-08-09 13:16:05 -07:00
Grisha Kruglov
5976869b0a Post: Make tests pass on Rust 1.25.0
For some reason, the converted doc test fails on Rust 1.25.0, while
working with other Rust versions. For simplicity, just convert it into
a regular test.
2018-08-09 13:16:05 -07:00
Grisha Kruglov
bf8c2c1516 Post: Remove bunch of dependencies from query-pull 2018-08-09 13:16:05 -07:00
Grisha Kruglov
dbb4aab071 Post: Remove mentat_sql dependency from query-projector 2018-08-09 13:16:05 -07:00
Grisha Kruglov
1e488d720b Post: Use a single implementation of bail macro 2018-08-09 13:16:05 -07:00
Grisha Kruglov
e9398dd50d Part 1: Move public errors into public-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
c00e14f5ff Pre: Remove :: dependency from src/errors.rs 2018-08-09 13:16:05 -07:00
Grisha Kruglov
c8e6a511f4 Pre: Move tolstoy/errors into tolstoy-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
9381af4289 Pre: Move core/Attribute* to core-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
68d0e17824 Pre: Move sql/errors into sql_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
05ef149545 Pre: Fold query-translator into query-projector 2018-08-09 13:16:05 -07:00
Grisha Kruglov
6312e89aba Pre: Move query-projectors/errors and aggregates into query-projector-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
ccdd17551a Pre: Move query-algebrizer/error.rs into query-algebrizer-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
9fd198f96a Pre: Move ValueTypeSet into core-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
2ae8594d20 Pre: Do not re-export EdnParseError from core 2018-08-09 13:16:05 -07:00
Grisha Kruglov
07beb68c7a Pre: Remove query/ crate 2018-08-09 13:16:05 -07:00
Grisha Kruglov
11aaa193f5 Pre: Move query-pull/errors into query-pull-traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
cebb85a7fe Pre: Move db/errors.rs into db_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
d0214fad7d Pre: Move core/types.rs into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
a57ba5d79f Pre: Move Entid and KnownEntid into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
f8478835a2 Use crates.io version of the enum-set
rnewman upstreamed his changes in https://github.com/contain-rs/enum-set/pull/20
2018-08-03 15:41:19 -07:00
Nick Alexander
79113498e7 [automation] Split into generic and Mentat-specific Docker images. 2018-08-03 12:53:22 -07:00
Nick Alexander
b5d0e12a24 [automation] Re-add project-specific Mentat Docker image. 2018-08-03 12:53:01 -07:00
Nick Alexander
814ab19ecb [automation] Move project-agnostic Dockerfile into subdirectory.
Docker is directory oriented so we have to play along.
2018-08-03 12:53:01 -07:00
Nick Alexander
0cb8227750 [automation] Be project agnostic; use armv7-linux-androideabi; install Android standalone toolchains.
This is ready for Android Rust-y components: it no longer references Mentat.

The standalone toolchains are installed into
$ANDROID_NDK_TOOLCHAIN_DIR/arch-$ANDROID_NDK_API_VERSION.
2018-08-03 12:53:01 -07:00
Nick Alexander
f747e2e550 [sdks/android] Pre: Disable testCaching for frequent intermittent failures. 2018-08-03 12:53:01 -07:00
Nick Alexander
5b4f50ce1b Fix vcsTag, yet again. 2018-07-31 14:42:08 -07:00
Nick Alexander
3cd61a0c93 Fix vcsTag, again. 2018-07-31 14:05:55 -07:00
Nick Alexander
65e9822ad6 Bump to version 0.11.0. 2018-07-31 09:59:18 -07:00
Nick Alexander
4325d6c0c3 [sdks/android] Move main Mentat Android SDK tests from androidTest to test.
This leverages JNA to test the Android SDK on the host machine using
Robolectric, which is significantly faster and easier to debug than
the equivalent on-device instrumentation tests.

We'll still want instrumentation smoke tests, but they won't need to
cover the entire range of the Android SDK.
2018-07-31 09:54:29 -07:00
Nick Alexander
e06bfd1b7d [sdks/android] Workaround Android Studio JUnit test runner runtime classpath issue. 2018-07-27 10:43:53 -07:00
Nick Alexander
a7d2057bc6 [sdks/android] Post: Address most Android Studio complaints.
The only ones I cared about were unchecked access, but while I'm here,
might as well do most of them.
2018-07-27 10:43:53 -07:00
Nick Alexander
2978ad91c0 [sdks/android] Part 3: Finish conversion to Robolectric. 2018-07-27 10:43:53 -07:00