Commit graph

965 commits

Author SHA1 Message Date
Grisha Kruglov
061967f268 Convert query-pull/ to failure. 2018-06-20 14:42:05 -07:00
Grisha Kruglov
326fe881a0 Convert query-algebrizer/ to failure. 2018-06-20 14:41:59 -07:00
Grisha Kruglov
ce3ce1ccbf Convert sql/ and query-sql/ to failure.
sql/query-sql
2018-06-20 14:41:52 -07:00
Mario Wenzel
fb7d2357de Don't try to match "key" when not using sqlcipher. (#752, #753) r=nalexander
This causes a runtime error, since `opt_str("key")` isn't recognized.
2018-06-20 13:26:36 -07:00
Emily Toop
aae50f40ac Set theme jekyll-theme-tactile 2018-06-20 15:59:50 +01:00
Emily Toop
4282b2d332 Set theme jekyll-theme-cayman 2018-06-20 15:53:34 +01:00
Nick Alexander
91fa34e462
[website] Create placeholder index.html. 2018-06-19 11:39:08 -07:00
Thom
88c6a4b05c
Fix typo nit accidentally missed in #743 (#744) 2018-06-14 13:30:03 -07:00
Thom
87fb505c56
Make travis test sqlcipher by running the tests on macos. Fixes #738 (#743) 2018-06-14 13:23:17 -07:00
Thom
54d592df29
Merge pull request #742 from thomcc/fix-bustage 2018-06-14 07:58:09 -07:00
Thom Chiovoloni
99a73ccb03 Avoid using 1.26.0-only features when using sqlcipher, and move the sqlcipher Store support to the correct file 2018-06-13 22:48:28 -07:00
Emily Toop
8e918949fb Separate Store from Conn.
This is a Pre: part extracted from #660.
2018-06-13 15:29:11 -07:00
Nick Alexander
f041dfe509 Bustage fix: Build against Rust 1.25. 2018-06-13 15:28:44 -07:00
Nick Alexander
8cc0e5a64e Make Travis test against multiple Rust versions, including Rust 1.25. 2018-06-13 15:28:44 -07:00
Thom
6a1a265894
Add support for using sqlcipher (#737). Fixes #118 2018-06-13 08:49:40 -07:00
Emily Toop
c0d4568970 Support consumption by Carthage; bump iOS minimum version to 11. r=nalexander
Principally, this adds producing libmentat_ffi.a for consumption by Carthage.

To achieve this, we disable bitcode and bump the miniumum iOS version
to 11.  In addition, we make things open and public so that consumers
can, well, consume.
2018-06-08 13:53:36 -07:00
Nick Alexander
3d5ae797b2 Parse queries with rust-peg. r=rnewman,grisha (#728) 2018-06-04 15:46:13 -07:00
Nick Alexander
cfed968514 Review comments. 2018-06-04 15:21:27 -07:00
Nick Alexander
e68cc4016c Part 7: Remove tx entirely.
This was left over from #681.
2018-06-04 15:04:39 -07:00
Nick Alexander
d4166cc67c Part 6: Remove query-parser entirely. 2018-06-04 15:04:39 -07:00
Nick Alexander
47441f56dc Part 5: Push FindQuery into query-algebrizer; structure errors.
This is a big deck-chair re-arrangement.  This puts FindQuery into
query-algebrizer and puts the validation from ParsedFindQuery ->
FindQuery their as well.

Some tests were re-homed for this.

In addition, the little-used maplit crate dependency was replaced with
inline expressions.
2018-06-04 15:04:39 -07:00
Nick Alexander
09f1d633b5 Part 4: Parse queries with rust-peg.
There's an unfortunate conflation here between implementing the query
parser in `rust-peg` and moving some validation that now happens at
parse time to happen later.  The result is that we introduce
`ParsedFindQuery` as a less-processed `FindQuery`, and that we only
use string errors (which is all `rust-peg` supports) instead of the
structured errors in query-parser's errors module.  The next commit
will address this, on the road to removing the `query-parser` module
entirely.
2018-06-04 15:04:39 -07:00
Nick Alexander
a8073056f2 Part 3: Move query into edn.
It's unfortunate to squash two crates together like this, but it's the
best option.
2018-06-04 15:04:37 -07:00
Nick Alexander
a4a8892309 Part 3a: Move file to preserve blame. 2018-06-04 14:56:56 -07:00
Nick Alexander
1d8d94f887 Part 2: Turn (type-function ?var) into (type ?var type-keyword).
This is more general (the parser doesn't encode the set of known
types), and avoids a dependency on `ValueType`.
2018-06-04 14:52:51 -07:00
Nick Alexander
ad9a1394a3 Part 1: Push ValueRc and friends into edn crate.
This is a pre-requisite for moving the existing `combine`-based parser
to use `rust-peg` -- part of the push to use `rust-peg` for all parsing
started in https://github.com/mozilla/mentat/pull/681.  We need the
types for the parsed structure "very early", and the `edn` crate is
the earliest such crate.

This is an unfortunate destruction of boundaries between parts of the
system, but it's the best way we have to achieve this right now.
2018-06-04 14:52:51 -07:00
Nick Alexander
f1fc9f1846 Part 0: Extract query-parser errors. 2018-06-04 14:52:51 -07:00
Nick Alexander
3cc8b4fd24 Pre: Prefer [(pred ...)] to [[pred ...]] syntax.
This is a style choice.  We supported both, perhaps for Datomic
compliance, but it's not the standard we use in our code base.  In
addition, it doesn't read like lisp (which is what EDN is copying),
since [] is not function application in most lisps.

It's also a convenience: I don't want to parse brackets that have to
agree with `rust-peg`.  It's not hard but it's also not worth doing.
2018-06-04 14:52:51 -07:00
Nick Alexander
47a0f40cce Pre: Fix warnings. 2018-06-04 14:52:51 -07:00
Nick Alexander
729fe59578 [edn] Pre: Rename keyword to namespaced_keyword.
The `Keyword` type evolved to become more general: we now use the one
type for both :regular and :name/spaced keywords.  This changes
reflects the new generality.
2018-06-04 14:52:51 -07:00
Nick Alexander
d8d18a1731
[query] Handle SQL NULL for aggregates over 0 rows. (#684) (#688) r=rnewman
This uses a `SELECT *` from an inner subselect to filter potentially `NULL` aggregates.

The alternative is to handle `NULL` values throughout the projector, which is simple but loses a valuable invariant: Mentat SQL queries produce values that are not `NULL`.
2018-06-01 14:17:31 -07:00
Grisha Kruglov
2a025916fe
Android SDK basic sample project and symlinked SDK Mentat binaries (#729) r=nalexander
* Add an IntelliJ section to gitignore
* Add Android SDK sample project which exercises mentat SDK
* Symlink libmentat_ffi.so in Android SDK to the generated --release files
* README files for Android SDK and mentat_ffi
2018-06-01 12:44:31 -07:00
Grisha Kruglov
93b7d25446
Android build script which supports target specification (#727) r=self 2018-05-31 12:25:24 -07:00
Grisha Kruglov
250e35b726
Gradle support for publishing to bintray (#720) r=ncalexan
* Rename SDK package name from com.mozilla.mentat to org.mozilla.mentat
* Gradle configuration for publishing to a bintray repository
2018-05-30 13:38:45 -07:00
Grisha Kruglov
b0421c61b4
Min SDK 16, bump dependency versions, update gradle & wrapper, fix linter error (#717) r= fluffyemily
Bump versions, update gradle wrapper, fix linter error (log tag too long)
2018-05-29 10:16:32 -07:00
Richard Newman
01db9232b4
Include namespace-separating solidus in NamespaceableName; improve type handling around ground (#713) r=nalexander
* Include the namespace-separating solidus in NamespaceableName.
* Use type annotations when deciding how to process ambiguous ground input.
* Include simple patterns in the type extraction phase of pattern application. (#705)
* Review comment.
* Add a test.
2018-05-29 16:45:53 +02:00
Chris Foster
e4447927c7 Update README.md
Grammar -- the subjunctive is appropriate here.
2018-05-24 10:00:01 -07:00
Emily Toop
6121da3592 Address review comments @nalexander 2018-05-15 15:39:15 +01:00
Emily Toop
2c0f755632 Address review comments @nalexander 2018-05-15 15:39:15 +01:00
Emily Toop
35467c1b24 Wrap caching FFI functions in Android Java library.
`CacheDirection` enum is used only on the Android side to provide a usable interface. FFI calls are more explicit.

Tests ensure that a cached query is faster than the uncached one.
2018-05-15 15:39:15 +01:00
Emily Toop
8add073001 Wrap caching FFI functions in Swift library.
`CacheDirection` enum is used only on the Swift side to provide a usable interface. FFI calls are more explicit.

Tests ensure that a cached query is faster than the uncached one.
2018-05-15 15:39:15 +01:00
Emily Toop
b4b558e196 Expose cache over the FFI.
This exposes an FFI function for each direction of caching, `Forward`, `Reverse` and `Both`. This is to make is as clear as possible to consumers which direction they are caching their attributes in. The original implementation exposed the `CacheDirection` enum over FFI and it made mistakes very easy to make. This is more explicit and therefore less prone to error.
2018-05-15 15:39:15 +01:00
Emily Toop
e0cd9b6b20 Implement InProgress transactions and InProgress and Entity builders on Android.
Rename some of the functions in TypedValue, TupleResult and QueryBuilder to make them more Javay and less Swifty
2018-05-15 15:39:15 +01:00
Emily Toop
38c1a93712 Implement InProgress transactions and InProgress and Entity builders on iOS 2018-05-15 15:39:15 +01:00
Emily Toop
ed5427253b Expose InProgress, InProgressBuilder and EntityBuilder over the FFI.
There are two ways to create each builder, directly from a `Store` or from an `InProgress`. Creating from `Store` will perform two actions, creating a new `InProgress` and then returning a builder from that `InProgress`. In the case of `store_entity_builder_with_entid` and `store_entity_builder_from_tempid`, the function goes a step further and calls `describe` or `describe_tempid` from the created `InProgressBuilder` and returning the `EntityBuilder` that results. These two functions are replicated on `InProgress`. This has been done to reduce the overhead of objects being passed over the FFI boundary.

The decision to do this enables us to go from something like

```
in_progress  = store_begin_transaction(store);
builder = in_progress_builder(in_progress);
entity_builder = in_progress_builder_describe(builder, entid);
```
to
```
entity_builder = store_entity_builder_from_entid(store);
```

There is an `add_*` and `retract_*` function specified for each `TypedValue` type for both `InProgressBuilder` and `EntityBuilder`.

To enable `transact` on `EntityBuilder` and `InProgressBuilder`, a new `repr(C)` struct has been created that contains a pointer to an `InProgress` and a pointer to a `Result<TxReport>` to allow passing the tuple result returned from `transact` on those types over the FFI.

Commit is possible from both builders and `InProgress`.
2018-05-15 15:39:15 +01:00
Richard Newman
b2e98f44f6
Generalize Entity by value type. (#701) (#691) r=rnewman
* Part 3: Parameterize Entity by value type.

This isn't quite right, because after parsing, we shouldn't care
about` `edn::ValueAndSpan`, we should care only about edn::Value.
However, I think we can drop `ValueAndSpan` entirely if we just use
`rust-peg` (and its simpler error messages) rather than a mix of
`rust-peg` and `combine`.

In any case, this paves the way to transacting `Entity<TypedValue>`,
which is a nice step towards building general entities.

* Part 1: Add AttributePlace.

* Part 2: Name other places EntityPlace and ValuePlace.

Now we're consistent and closer to self-documenting.  Both matter more
as we expose `Entity` as the thing to build for programmatic usage.

* Part 4: Allow Ident and TempId in ValuePlace.

The parser will never produce these, since determining whether an
integer/keyword or string is an ident or a tempid, respectively, in
the value place requires the schema.

But a builder that produces `Entity` instances directly will want to
produce these.
2018-05-15 00:43:07 -07:00
Nick Alexander
46c2a0801f Add type checking and constraint checking to the transactor. (#663, #532, #679)
This should address #663, by re-inserting type checking in the
transactor stack after the entry point used by the term builder.

Before this commit, we were using an SQLite UNIQUE index to assert
that no `[e a]` pair, with `a` a cardinality one attribute, was
asserted more than once.  However, that's not in line with Datomic,
which treats transaction inputs as a set and allows a single datom
like `[e a v]` to appear multiple times.  It's both awkward and not
particularly efficient to look for _distinct_ repetitions in SQL, so
we accept some runtime cost in order to check for repetitions in the
transactor.  This will allow us to address #532, which is really about
whether we treat inputs as sets.  A side benefit is that we can
provide more helpful error messages when the transactor does detect
that the input truly violates the cardinality constraints of the
schema.

This commit builds a trie while error checking and collecting final
terms, which should be fairly efficient.  It also allows a simpler
expression of input-provided :db/txInstant datoms, which in turn
uncovered a small issue with the transaction watcher, where-by the
watcher would not see non-input-provided :db/txInstant datoms.

This transition to Datomic-like input-as-set semantics allows us to
address #532.  Previously, two tempids that upserted to the same entid
would produce duplicate datoms, and that would have been rejected by
the transactor -- correctly, since we did not allow duplicate datoms
under the input-as-list semantics.  With input-as-set semantics,
duplicate datoms are allowed; and that means that we must allow
tempids to be equivalent, i.e., to resolve to the same tempid.

To achieve this, we:
- index the set of tempids
- identify tempid indices that share an upsert
- map tempids to a dense set of contiguous integer labels

We use the well-known union-find algorithm, as implemented by
petgraph, to efficiently manage the set of equivalent tempids.

Along the way, I've fixed and added tests for two small errors in the
transactor.  First, don't drop datoms resolved by upsert (#679).
Second, ensure that complex upserts are allocated.

I don't know quite what happened here.  The Clojure implementation
correctly kept complex upserts that hadn't resolved as complex
upserts (see
9a9dfb502a/src/common/datomish/transact.cljc (L436))
and then allocated complex upserts if they didn't resolve (see
9a9dfb502a/src/common/datomish/transact.cljc (L509)).

Based on the code comments, I think the Rust implementation must have
incorrectly tried to optimize by handling all complex upserts in at
most a single generation of evolution, and that's just not correct.
We're effectively implementing a topological sort, using very specific
domain knowledge, and its not true that a node in a topological sort
can be considered only once!
2018-05-14 15:22:45 -07:00
Nick Alexander
e5e37178af Pre: Remove ancient Clojure code comments. 2018-05-14 15:22:45 -07:00
Richard Newman
3cba87c74b Allow pull aliases to be non-namespaced. (#694) r=nalexander 2018-05-14 10:45:48 -07:00
Emily Toop
013629dec6
iOS and Android (Java) sdk framework (#643)
Documents the FFI layer for Mentat, and provides transaction functionality via an EDN string. Creates two native libraries for iOS (Swift) and Android (Java) and fully tests the FFI for both platforms.

Closes #619 #614 #611
2018-05-14 16:20:36 +01:00