Commit graph

973 commits

Author SHA1 Message Date
Richard Newman
3f264e9eb2 Implement fulltext. (#477) r=nalexander
* You can't use fulltext search on a non-fulltext attribute.
* Allow for implicit placeholder bindings in fulltext.
2017-06-15 10:28:11 -07:00
Richard Newman
565a0e9ff9 Implement MATCHES throughout SQL machinery. 2017-06-15 10:28:10 -07:00
Richard Newman
17c59bbff6 Apply newly bound values to existing columns.
This commit lifts some logic out of the scalar ground handler to apply
elsewhere.

When a new value binding is encountered for a variable to which column
bindings have already been established, we do two things:

- We apply a new constraint to the primary column. This ensures that the
  behavior for ground-first and ground-second is equivalent.
- We eliminate any existing column type extraction: it won't be
  necessary now that a constant value and constant type are known.
2017-06-15 10:28:09 -07:00
Richard Newman
f7a3fd5b17 Refactor arg conversion and ground into separate files. 2017-06-15 10:28:07 -07:00
Richard Newman
54bdd382fb Add a test that late inputs aren't allowed in ground. 2017-06-15 10:28:05 -07:00
Richard Newman
c2ec1a6bdf Pre: move Either to mentat_core::util. 2017-06-15 10:28:02 -07:00
Richard Newman
03c0930285 Pre: implement IntoIterator for ValueTypeSet. 2017-06-15 10:27:51 -07:00
Richard Newman
5d5e85bcba Pre: ensure that constant floats end up as floats in SQL, never integers. 2017-06-15 10:27:16 -07:00
Richard Newman
8ec24f01f6 Handle ground. (#469) r=nalexander 2017-06-09 20:20:16 -07:00
Richard Newman
e1e549440f Expand type code when applying ground. (#475) 2017-06-09 20:18:53 -07:00
Nick Alexander
79fa0994b3 Part 3: Handle ground. (#469) r=nalexander,rnewman
This version removes nalexander's lovely matrix code. It turned out
that scalar and tuple bindings are sufficiently different from coll
and rel -- they can directly apply as values in the query -- that
there was no point in jumping through hoops to turn those single
values into a matrix.

Furthermore, I've standardized us on a Vec<TypedValue>
representation for rectangular matrices, which should be much
more efficient, but would have required rewriting that code.

Finally, coll and rel are sufficiently different from each other
-- coll doesn't require processing nested collections -- that
my attempts to share code between them fell somewhat flat. I had
lots of nice ideas about zipping together cycles and such, but
ultimately I ended up with relatively straightforward, if a bit
repetitive, code.

The next commit will demonstrate the value of this work -- tests
that exercised scalar and tuple grounding now collapse down to
the simplest possible SQL.
2017-06-09 20:18:31 -07:00
Nick Alexander
d04d22a6a6 Part 2: refactor projector to be reusable from translator.
This allows the translator to also use bound values in nested queries.
2017-06-09 20:16:39 -07:00
Nick Alexander
b9cbf92205 Part 1: Parse functions in where clauses. 2017-06-09 20:16:39 -07:00
Richard Newman
c6e933c396 Pre: make rule_vars return unique vars. 2017-06-09 20:16:39 -07:00
Richard Newman
d30ad428e8 Pre: take a dependency on maplit to allow BTreeSet literals. 2017-06-09 20:16:39 -07:00
Richard Newman
4a886aae17 Pre: derive Debug. 2017-06-09 20:16:38 -07:00
Richard Newman
9ac2b8c680 Pre: add ConjoiningClauses::known_type_set. 2017-06-09 20:16:38 -07:00
Richard Newman
899e5d0971 Pre: add ConjoiningClauses::bind_value. 2017-06-09 20:16:38 -07:00
Nick Alexander
13e27c83e2 Pre: Modify predicate implementation in preparation for functions that bind. 2017-06-09 20:16:38 -07:00
Nick Alexander
4d2eb7222e Pre: Generalize NonNumericArgument to InvalidArgument. 2017-06-09 20:16:37 -07:00
Nick Alexander
2f38f1e73e Pre: Make it easier to debug binding errors. 2017-06-09 20:16:37 -07:00
Nick Alexander
002c918c96 Pre: Move PushComputed up module hierarchy; make it public. 2017-06-09 20:16:37 -07:00
Richard Newman
70c5bcfa99 Pre: simplify values SQL expansion.
This uses `interpose` instead of manual looping.
2017-06-09 20:16:37 -07:00
Richard Newman
63574af7ac Pre: flatten the representation of VALUES.
A single vec that's traversed in chunks is more efficient than multiple
vecs… and this ensures that each sub-vec is the same size.
2017-06-09 20:16:37 -07:00
Nick Alexander
06bb8e99a7 Pre: Add Values to query-sql. 2017-06-09 20:16:36 -07:00
Nick Alexander
9fe31d443d Pre: Accept EDN vectors in FnArg arguments.
Datomic accepts mostly-arbitrary EDN, and it is actually used: for
example, the following are all valid, and all mean different things:
* `(ground 1 ?x)`
* `(ground [1 2 3] [?x ?y ?z])`
* `(ground [[1 2 3] [4 5 6]] [[?x ?y ?z]])`

We could probably introduce new syntax that expresses these patterns
while avoiding collection arguments, but I don't see one right now.
I've elected to support only vectors for simplicity; I'm hoping to
avoid parsing edn::Value in the query-algebrizer.
2017-06-09 20:16:36 -07:00
Nick Alexander
08534a1a3a Pre: Handle SrcVar. 2017-06-09 20:16:36 -07:00
Richard Newman
a10c6fc67a Pre: make ValueTypeSet Copy, as it only newtypes EnumSet, which is Copy. 2017-06-09 20:16:36 -07:00
Richard Newman
dbbbd220f9 Pre: add helpers to ValueTypeSet. 2017-06-09 20:16:35 -07:00
Richard Newman
9a12ced317 Don't allow callers to specify arbitrary new entity IDs. (#447) r=nalexander
This commit adds a check to the partition map that a provided entity ID
has been mentioned (i.e., is present in the start:index range of one of
our partitions).

We introduce a newtype for known entity IDs, using this internally in
the tx expander to track user-provided entids that have passed the above
check (and IDs that we allocate as part of tempid processing). This
newtype is stripped prior to tx assertion.

In order that DB tests can continue to write

  [:db/add 111 :foo/bar 222]

we add an additional fake partition to our test connections, ranging
from 100 to 1000.
2017-06-09 15:45:26 -07:00
Nick Alexander
5c5818069f Handle :attribute/_reverse in transactor. Fixes #187. r=rnewman 2017-06-08 10:33:09 -07:00
Nick Alexander
c165972684 Post: Reject at parse-time reversed attributes in direct notation with bad values.
This is an optimization that trades rejecting inputs earlier at the
cost of expressive error messages.  It should be possible to recover
the error messages, however.

This will reject input like `[:db/{add,retract} v :attribute/_reversed NOT-AN-ENTITY]`.
2017-06-08 10:30:31 -07:00
Nick Alexander
59a710f80f Review comments: another test, add unreversed(). 2017-06-08 10:30:31 -07:00
Nick Alexander
eb220528bf Post: Indent. 2017-06-08 10:30:31 -07:00
Nick Alexander
d88823e7c4 Handle :attribute/_reverse in transactor. Fixes #187
There are two broad approaches:

1) Handle reverse attribute notation dynamically, in the style that
   Datomic does.  This is the most flexible, but it's not a good fit
   given that we produce strongly typed output from the parser.
   Strongly typed input to the transactor has had many benefits, so I
   don't want to roll it back for a relatively unimportant feature
   like reverse notation -- especially not since Mentat does not
   require :db.install/_attribute to modify schema attributes.

2) Handle reverse attribute in the parser itself, so that we can
   produce strongly typed parser output while restricting the input.
   I implemented this first and discovered that it's very difficult to
   give sensible error messages in common cases.

In any case, the bulk of the code is the same between the two
approaches, and I wrote the tests for the dynamic version (with error
output), so that's what I'm rolling with.

This patch preserves the existing indentation, to highlight the
differences.  The next patch will indent.
2017-06-08 10:30:31 -07:00
Nick Alexander
0be78cf956 Pre: Extract entity_*_into_term_* helpers. 2017-06-08 10:30:31 -07:00
Nick Alexander
4b0881a957 Pre: Push bookkeeping into an InProcess struct. 2017-06-08 10:30:31 -07:00
Nick Alexander
05129cefbb Pre: Use ValueType rather than Attribute to convert edn::Value to TypedValue.
This is expedient now, but might require work in the future to achieve
better error messages.
2017-06-08 10:30:31 -07:00
Nick Alexander
2650fe163d Pre: Intern lookup_ref by reference. 2017-06-08 10:30:31 -07:00
Nick Alexander
a4fc04ea86 Pre: Crib map_{left,right} for Either. 2017-06-08 10:30:31 -07:00
Richard Newman
634b7a816b Dedupe SQL arguments. (#471) r=nalexander 2017-06-07 11:55:42 -07:00
Richard Newman
2c52346999 Review comment: generalize from Uuid SQL arguments to byte arrays. 2017-06-07 11:55:05 -07:00
Richard Newman
ed04083ceb Dedupe SQL arguments.
This isn't perfect -- we still need to clone in a couple of cases -- but it avoids us
passing duplicate strings down into SQLite whenever the same value is mentioned more
than once in a query.
2017-06-07 11:46:18 -07:00
Richard Newman
7fc0848cb0 Fix typo in README. 2017-06-06 19:01:27 +00:00
Richard Newman
a88375fc15 Update README for master switchover. 2017-06-06 11:10:49 -07:00
Nick Alexander
8a8fcedd1c Parse without copying streams. Fixes #436. #444. r=rnewman
We were accidentally quadratic, copying the tails of owned Vec
instances around.  This brings us down to the expected linear runtime.
2017-05-18 10:20:06 -07:00
Nick Alexander
409a2ea78f Post: Use choice instead of or. 2017-05-18 10:17:13 -07:00
Nick Alexander
d1ac752de6 Parse without copying; parse keyword maps using macros.
This is a big commit, but it breaks into two conceptual pieces.  The
first is to "parse without copying".  We replace a stream of an owned
collection of edn::ValueAndSpan and instead have a stream of a
borrowed collection of &edn::ValueAndSpan references.  (Generally,
this is represented as an iterator over a slice, but it can be over
other things too.)  Cloning such iterators is constant time, which
improves on cloning an owned collection of edn::ValueAndSpan, which is
linear time in the length of the collection and additional time
depending on the complexity of the EDN values.

The second conceptual piece is to parse keyword maps using a special
parser and a macro to build the parser implementations.  Before, we
created a new edn::ValueAndSpan::Map to represent a keyword map in
vector form; since we're working with &edn::ValueAndSpan references
now, we can't create an &edn::ValueAndSpan reference with an
appropriate lifetime.  Therefore we generalize the concept of
iteration slightly and turn keyword maps in map form into linear
iterators by flattening the value maps.  This is a potentially
obscuring transformation, so we have to take care to protect against
some failure cases.  (See the comments and the tests in the code.)

After these changes, parsing using `combine` is linear time (and
reasonably fast).
2017-05-18 10:17:13 -07:00
Nick Alexander
4fa57942d3 Pre: Move macros out of lib.rs.
It seems very subtle to use macros in tests: I needed to separate the
modules in order to control load order to get everything to work.
2017-05-18 10:17:13 -07:00
Richard Newman
953f9f7734 Remove server instructions from README. 2017-05-15 13:05:24 +00:00