Commit graph

965 commits

Author SHA1 Message Date
Grisha Kruglov
a57ba5d79f Pre: Move Entid and KnownEntid into core_traits 2018-08-09 13:16:05 -07:00
Grisha Kruglov
f8478835a2 Use crates.io version of the enum-set
rnewman upstreamed his changes in https://github.com/contain-rs/enum-set/pull/20
2018-08-03 15:41:19 -07:00
Nick Alexander
79113498e7 [automation] Split into generic and Mentat-specific Docker images. 2018-08-03 12:53:22 -07:00
Nick Alexander
b5d0e12a24 [automation] Re-add project-specific Mentat Docker image. 2018-08-03 12:53:01 -07:00
Nick Alexander
814ab19ecb [automation] Move project-agnostic Dockerfile into subdirectory.
Docker is directory oriented so we have to play along.
2018-08-03 12:53:01 -07:00
Nick Alexander
0cb8227750 [automation] Be project agnostic; use armv7-linux-androideabi; install Android standalone toolchains.
This is ready for Android Rust-y components: it no longer references Mentat.

The standalone toolchains are installed into
$ANDROID_NDK_TOOLCHAIN_DIR/arch-$ANDROID_NDK_API_VERSION.
2018-08-03 12:53:01 -07:00
Nick Alexander
f747e2e550 [sdks/android] Pre: Disable testCaching for frequent intermittent failures. 2018-08-03 12:53:01 -07:00
Nick Alexander
5b4f50ce1b Fix vcsTag, yet again. 2018-07-31 14:42:08 -07:00
Nick Alexander
3cd61a0c93 Fix vcsTag, again. 2018-07-31 14:05:55 -07:00
Nick Alexander
65e9822ad6 Bump to version 0.11.0. 2018-07-31 09:59:18 -07:00
Nick Alexander
4325d6c0c3 [sdks/android] Move main Mentat Android SDK tests from androidTest to test.
This leverages JNA to test the Android SDK on the host machine using
Robolectric, which is significantly faster and easier to debug than
the equivalent on-device instrumentation tests.

We'll still want instrumentation smoke tests, but they won't need to
cover the entire range of the Android SDK.
2018-07-31 09:54:29 -07:00
Nick Alexander
e06bfd1b7d [sdks/android] Workaround Android Studio JUnit test runner runtime classpath issue. 2018-07-27 10:43:53 -07:00
Nick Alexander
a7d2057bc6 [sdks/android] Post: Address most Android Studio complaints.
The only ones I cared about were unchecked access, but while I'm here,
might as well do most of them.
2018-07-27 10:43:53 -07:00
Nick Alexander
2978ad91c0 [sdks/android] Part 3: Finish conversion to Robolectric. 2018-07-27 10:43:53 -07:00
Nick Alexander
190e05e360 [sdks/android] Part 2: Replace Expectation/wait/notify with CountDownLatch.
Locally, I witnessed very slow tests.  Profiling with Visual VM
revealed a lot of time spent in `wait`.

Digging in, we were trying to be clever, with a `wait(1000)/notify`
mechanism.  However, there were never multiple threads in play, so the
waiter wasn't waiting when `notify` was invoked.  That means we always
timed out.  I think this never worked and using bare `wait()` would
have revealed that.

Anyway, `CountDownLatch` maintains the one bit of state (was I
notified) and generalizes smoothly to when we have threads.
2018-07-27 10:43:53 -07:00
Nick Alexander
d23f2b373a [sdks/android] Include vcsTag when uploading to bintray. 2018-07-27 10:43:52 -07:00
Nick Alexander
6856462f1b [sdks/android] Part 1: Move androidTest to test. 2018-07-27 10:43:52 -07:00
Grisha Kruglov
536d40ad84 Part 4: Add support for moving transactions off of main timeline 2018-07-26 17:14:05 -07:00
Grisha Kruglov
4ec780c87a Part 3: Use a view to derive parts table
Being able to derive partition map from partition definitions and current
state of the world (transactions), segmented by timelines, is useful
because it lets us not worry about keeping materialized partition maps
up-to-date - since there's no need for materialized partition maps at that point.

This comes in very handy when we start moving chunks of transactions off of our mainline.
Alternative to this work would look like materializing partition maps per timeline,
growing support for incremental "backwards update" of the materialized maps, etc.

Our core partitions are defined in 'known_parts' table during bootstrap,
and what used to be 'parts' table is a generated view that operates over
transactions to figure out partition index.

'parts' is defined for the main timeline. Querying parts for other timelines
or for particular timeline+tx combinations will look similar.
2018-07-26 17:14:05 -07:00
Grisha Kruglov
3ca5255cde Part 2: Add basic support for timelines to the transactor
This records transactions onto a default timeline (0).
2018-07-26 17:14:05 -07:00
Grisha Kruglov
0974108a52 Part 1: Allow specifying transactor's commit behaviour
Normally we want to both materialize our changes (into 'datoms')
as well as commit source transactions into 'transactions' table.

However, when moving transactions from timeline to timeline
we don't want to persist artifacts (rewind assertions), just their
materializations.

This patch expands the 'db' interface to allow for this split,
and changes transactor's functions to take a crate-private 'action'
which defines desired behaviour.
2018-07-26 17:14:05 -07:00
Grisha Kruglov
5a29efa336 Part 0: Allow retractions of installed attributes
This is necessary for the timelines work ahead. When schema is being
moved off of a main timeline, we need to be able to retract it cleanly.

Retractions are only processed if the whole defining attribute set
is being retracted at once (:db/ident, :db/valueType, :db/cardinality).
2018-07-26 17:14:05 -07:00
Grisha Kruglov
9a47d8905f Pre: 'Into' implementation chaining TermWithoutTempIds -> TermWithTempIds -> TermWithTempIdsAndLookupRefs 2018-07-26 17:14:05 -07:00
Nick Alexander
e6066769ca Pre: Differentiate bad attribute retractions from unrecognized retractions. 2018-07-26 17:14:05 -07:00
Nick Alexander
fba378ee39 [sdks/android] Build Mentat Android SDK in TaskCluster; publish org.mozilla.mentat to nalexander's personal Bintray repo.
I haven't had this reviewed thoroughly, but it mostly works.
2018-07-26 13:12:20 -07:00
Nick Alexander
faef4e9ee8 Bump to version 0.10.0. 2018-07-26 13:09:18 -07:00
Nick Alexander
a8cc9cb70d [sdks/android] Don't strip Mentat library.
Help folks debugging by including symbols in our native libraries.
Yes, this makes the resulting AAR very large.  The Android ecosystem
seems to be in flux around who is in charge of stripping native
binaries, but for now let's provide symbols and see how consumers
react.
2018-07-26 13:06:03 -07:00
Nick Alexander
76d7df5548 [sdks/android] Package JNA using upstream dependency. 2018-07-26 13:05:31 -07:00
Nick Alexander
7e31ca15bc [sdks] Make store_open{_encrypted} return useful errors.
Because this was formerly a constructor, the pattern needed to change
to a factory function, but that's better than what we had.
2018-07-26 13:01:53 -07:00
Nick Alexander
67a14ca756 [sdks/android] Build Mentat Android SDK in TaskCluster; publish org.mozilla.mentat to nalexander's personal Bintray repo.
The automation parts were cribbed directly from
50add3e176.

The automation permissions were added in
https://bugzilla.mozilla.org/show_bug.cgi?id=1477311.

This uses a very rudimentary Gradle plugin, `rust-android-gradle`,
with custom fixes and extensions.  It works pretty well for what it
is!  See https://github.com/ncalexan/rust-android-gradle.
2018-07-25 20:50:44 -07:00
Nick Alexander
0955c784b7 [sdks/android] Pre: Trim unused Android bits.
We don't use UI libraries, don't require UI resources, and don't
require any permissions.
2018-07-25 20:38:56 -07:00
Nick Alexander
9e6505a930 [sdks/android] Pre: Fix unused warnings. 2018-07-25 20:38:48 -07:00
Victor Porof
89d8ac50a8
Run serde_support tests for the EDN module on CI (#792)
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2018-07-19 19:03:19 +02:00
Victor Porof
2540404b00
Generate rust documentation on CI and publish to gh-pages automatically (#793)
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2018-07-19 18:32:54 +02:00
Nick Alexander
22dad5d6ca [build] Include Gradle wrapper JAR in repository.
Presumably this was an error: `.gitignore` ignores all JAR files, but
this one really needs to be in version control.
2018-07-17 15:30:10 -07:00
Grisha Kruglov
69c9f512a0 Move entid allocation logic into Partition r=nalexander 2018-07-17 06:20:37 -07:00
Grisha Kruglov
6290cc9db2 Enforce partition integrity when setting its index r=nalexander
Timelines work starts to perform modifications on the partitions
that go beyond simple allocations. This change pre-emptively protects
partition integrity by asserting that index modifications are legal.
2018-07-17 06:20:37 -07:00
Nick Alexander
38a92229d7 Pre: Replace PartitionMapping trait with newtype. r=grisha
Generally, I think that Mentat is using too many small traits rather
than wrapping types into newtypes.  Wrapping into newtypes is cheap in
Rust, and it makes it easier to reason about the code.
2018-07-17 06:20:37 -07:00
Grisha Kruglov
675a865896
Extract and improve test macros (#787) r=nalexander
* Part 1: Extract low-level test framework into mentat_db::debug for re-use.

* Part 2: Improve assert_matches!.

This corrects an incorrect pattern: a conversion method taking &self
but returning an owned value should be named like `to_FOO(&self) -> FOO`.  (A
reference-to-reference conversion should be named like `as_FOO(&self)
-> &FOO`.  A consuming conversion should be named like `into_FOO(self)
-> FOO`.)

In addition, this pushes the conversion via `to_edn` into the
`assert_matches!` macro, which lets consumers get a real data
structure (say, `Datoms`) and use it directly before or after
`assert_matches!`.  (Currently, consumers get back `edn::Value`
instances, which aren't nearly as pleasant to use as real data
structures.)

Co-authored-by: Grisha Kruglov <gkruglov@mozilla.com>

* Part 3: Use mentat_db::debug framework in Tolstoy crate.

The advantage of this approach is that compiling Tolstoy (or anything
that's not db, really) can be quite a bit faster than compiling db.
2018-07-16 13:58:34 -07:00
Nick Alexander
e9cddd63e4 [tx] Don't treat :db/doc as defining a schema attribute. (#784) r=grisha 2018-07-16 12:20:34 -07:00
Nick Alexander
9291b2a0b0 [tx] Don't treat :db/doc as defining a schema attribute. (#784) 2018-07-13 14:29:29 -07:00
Grisha Kruglov
bff24c60b7
Add a top-level "syncable" feature. (#782) r=ncalexan
* Add a top-level "syncable" feature.

Tested with:

cargo test --all
cargo test --all --no-default-features
cargo build --manifest-path tools/cli/Cargo.toml --no-default-features
cargo run --manifest-path tools/cli/Cargo.toml --no-default-features debugcli

Co-authored-by: Nick Alexander <nalexander@mozilla.com>

* Add 'syncable' feature to 'db' crate to conditionally derive serialization for Partition*

This is leading up to syncing with partition support.
2018-07-11 16:26:06 -07:00
Nick Alexander
61e6b85e6a Make Partition include end of range and allow_excision flag. r=grisha,nalexander 2018-07-06 16:12:28 -07:00
Nick Alexander
82610f17f8 Part 2: Make partition include an allow_excision flag.
This is leading up to the implementation of
https://github.com/mozilla/mentat/issues/21.
2018-07-06 16:11:42 -07:00
Grisha Kruglov
c0ddc2ca70 Part 1: Make Partition include explicit end range bound.
It's helpful to have the full range when syncing.
2018-07-06 15:23:06 -07:00
Nick Alexander
7d2fe8c625 Remove low-hanging dependency fruit. (#773) r=nalexander 2018-07-06 14:58:06 -07:00
Thom Chiovoloni
0549bbd604 Remove needless num dependency from mentat_core. 2018-07-06 14:56:42 -07:00
Thom Chiovoloni
dcc0770ca4 Remove needless num dependency from mentat_db and optimize remove_every.
This implementation of `remove_every` is O(n) and not O(n^2) like it was before.
2018-07-06 14:56:33 -07:00
Thom Chiovoloni
ad2b646700 Remove regex dependency from query_sql. Fixes #771. 2018-07-06 14:56:11 -07:00
Nick Alexander
f65512158b Make kw!(foo.bar/bar.baz) work. 2018-07-06 14:19:50 -07:00