Commit graph

497 commits

Author SHA1 Message Date
Emily Toop
4b45827be7 Fix merge conflicts 2017-04-07 12:17:39 +01:00
Nick Alexander
5369f03464 Improve parsing of nested edn::ValueAndSpan streams. r=rnewman (#393)
* Pre: Expose more in edn.

* Pre: Make it easier to work with ValueAndSpan.

with_spans() is a temporary hack, needed only because I don't care to
parse the bootstrap assertions from text right now.

* Part 1a: Add `value_and_span` for parsing nested `edn::ValueAndSpan` instances.

I wasn't able to abstract over `edn::Value` and `edn::ValueAndSpan`;
there are multiple obstacles.  I chose to roll with
`edn::ValueAndSpan` since it exposes the additional span information
that we will want to form good error messages in the future.

* Part 1b: Add keyword_map() parsing an `edn::Value::Vector` into an `edn::Value::map`.

* Part 1c: Add `Log`/`.log(...)` for logging parser progress.

This is a terrible hack, but it sure helps to debug complicated nested
parsers.  I don't even know what a principled approach would look
like; since our parser combinators are so frequently expressed in
code, it's hard to imagine a data-driven interpreter that can help
debug things.

* Part 2: Use `value_and_span` apparatus in tx-parser/.

I break an abstraction boundary by returning a value column
`edn::ValueAndSpan` rather than just an `edn::Value`.  That is, the
transaction processor shouldn't care where the `edn::Value` it is
processing arose -- even we care to track that information we should
bake it into the `Entity` type.  We do this because we need to
dynamically parse the value column to support nested maps, and parsing
requires a full `edn::ValueAndSpan`.  Alternately, we could cheat and
fake the spans when parsing nested maps, but that's potentially
expensive.

* Part 3: Use `value_and_span` apparatus in query-parser/.

* Part 4: Use `value_and_span` apparatus in root crate.

* Review comment: Make Span and SpanPosition Copy.

* Review comment: nits.

* Review comment: Make `or` be `or_exactly`.

I baked the eof checking directly into the parser, rather than using
the skip and eof parsers.  I also took the time to restore some tests
that were mistakenly commented out.

* Review comment: Extract and use def_matches_* macros.

* Review comment: .map() as late as possible.
2017-04-06 10:06:28 -07:00
Nick Alexander
71d3aa29ed Part 3: Start binding fulltext values. 2017-04-05 15:31:05 -07:00
Nick Alexander
6544ca1594 Pre: Handle SrcVar. 2017-04-05 15:30:22 -07:00
Nick Alexander
62fda71fbc Part 2: Implement apply_fulltext and column constraints.
No bindings, yet.
2017-04-05 11:02:09 -07:00
Nick Alexander
08cae4bc7c Review comment: Find, not Bind. 2017-04-05 11:01:47 -07:00
Nick Alexander
c203046c16 Part 1: Parse functions in where clauses. 2017-04-03 16:46:11 -07:00
Nick Alexander
678a116130 Review comment: .map() as late as possible. 2017-04-03 13:15:21 -07:00
Nick Alexander
f7fb22ae7e Review comment: Extract and use def_matches_* macros. 2017-04-03 13:15:21 -07:00
Nick Alexander
0165a842ef Review comment: Make or be or_exactly.
I baked the eof checking directly into the parser, rather than using
the skip and eof parsers.  I also took the time to restore some tests
that were mistakenly commented out.
2017-04-03 13:15:21 -07:00
Nick Alexander
f8e75d817e Review comment: nits. 2017-04-03 13:15:21 -07:00
Nick Alexander
7135eeac49 Review comment: Make Span and SpanPosition Copy. 2017-04-03 13:15:21 -07:00
Nick Alexander
720fbf3d01 Part 4: Use value_and_span apparatus in root crate. 2017-04-03 13:15:21 -07:00
Nick Alexander
ff136b2546 Part 3: Use value_and_span apparatus in query-parser/. 2017-04-03 13:15:21 -07:00
Nick Alexander
e947a32c59 Part 2: Use value_and_span apparatus in tx-parser/.
I break an abstraction boundary by returning a value column
`edn::ValueAndSpan` rather than just an `edn::Value`.  That is, the
transaction processor shouldn't care where the `edn::Value` it is
processing arose -- even we care to track that information we should
bake it into the `Entity` type.  We do this because we need to
dynamically parse the value column to support nested maps, and parsing
requires a full `edn::ValueAndSpan`.  Alternately, we could cheat and
fake the spans when parsing nested maps, but that's potentially
expensive.
2017-04-03 13:12:41 -07:00
Nick Alexander
f4e2a0471f Part 1c: Add Log/.log(...) for logging parser progress.
This is a terrible hack, but it sure helps to debug complicated nested
parsers.  I don't even know what a principled approach would look
like; since our parser combinators are so frequently expressed in
code, it's hard to imagine a data-driven interpreter that can help
debug things.
2017-04-03 09:15:36 -07:00
Nick Alexander
dcc05c643c Part 1b: Add keyword_map() parsing an edn::Value::Vector into an edn::Value::map. 2017-04-03 09:15:36 -07:00
Nick Alexander
8754cf224b Part 1a: Add value_and_span for parsing nested edn::ValueAndSpan instances.
I wasn't able to abstract over `edn::Value` and `edn::ValueAndSpan`;
there are multiple obstacles.  I chose to roll with
`edn::ValueAndSpan` since it exposes the additional span information
that we will want to form good error messages in the future.
2017-04-03 09:15:36 -07:00
Nick Alexander
f807b16db1 Pre: Make it easier to work with ValueAndSpan.
with_spans() is a temporary hack, needed only because I don't care to
parse the bootstrap assertions from text right now.
2017-04-03 09:15:36 -07:00
Nick Alexander
c1409078fa Pre: Expose more in edn. 2017-04-03 09:15:36 -07:00
Richard Newman
a5023c70cb Use Rc for TypedValue, Variable, and query Ident keywords. (#395) r=nalexander
Part 1, core: use Rc for String and Keyword.
Part 2, query: use Rc for Variable.
Part 3, sql: use Rc for args in SQLiteQueryBuilder.
Part 4, query-algebrizer: use Rc.
Part 5, db: use Rc.
Part 6, query-parser: use Rc.
Part 7, query-projector: use Rc.
Part 8, query-translator: use Rc.
Part 9, top level: use Rc.
Part 10: intern Ident and IdentOrKeyword.
2017-04-02 21:38:36 -07:00
Richard Newman
8ae8466cf9 Make InternSet::intern accept Into<Rc<T>>. Add a test. r=nalexander 2017-03-31 09:59:58 -07:00
Richard Newman
92cdd72500 Diagnose and prepare simple and complex or joins. (#396) r=nalexander 2017-03-30 19:14:05 -07:00
Richard Newman
2b2b5cf696 Part 6: implement decision tree for processing simple alternation. 2017-03-30 19:13:40 -07:00
Richard Newman
74f188df9b Part 5b: rename also/instead to add_intersection and add_alternate. 2017-03-30 19:13:20 -07:00
Richard Newman
9e5c735460 Part 5: split cc.rs into a 'clauses' module.
mod.rs defines the module and ConjoiningClauses itself, complete with
methods to record facts and ask it questions.

pattern.rs, predicate.rs, resolve.rs, and or.rs include particular
functionality around accumulating certain kinds of patterns.

Only `or.rs` includes significant new code; the rest is just split.
2017-03-30 19:13:20 -07:00
Richard Newman
72eeedec74 Part 4: add OrJoin::is_fully_unified.
This allows us to tell if all the variables in a valid `or` join are to
be unified, which is necessary for simple joins.
2017-03-30 19:13:20 -07:00
Richard Newman
ce3c4f0dca Part 3: have table_for_places return a Result, not an Option. 2017-03-30 19:13:20 -07:00
Richard Newman
01ca0ae5c1 Part 2: add an EmptyBecause case for fulltext/non-string type mismatch. 2017-03-30 19:13:19 -07:00
Richard Newman
997df0b776 Part 1: introduce ColumnIntersection and ColumnAlternation.
This provides a limited form of OR and AND for column constraints, allowing
simple 'or-join' queries to be expressed on a single table alias.
2017-03-30 19:13:19 -07:00
Richard Newman
460fdac252 Pre: add Variable::from_valid_name, TypedValue::{typed_string,typed_ns_keyword}. 2017-03-30 19:13:19 -07:00
Richard Newman
439f3a2283 Pre: add some 'am I a pattern?' helper predicates to clause types. 2017-03-30 19:06:33 -07:00
Richard Newman
d2e6b767c6 Pre: add mentat_core::utils::{ResultEffect,OptionEffect}. 2017-03-30 19:06:28 -07:00
Richard Newman
95a5326e23 Pre: move EmptyBecause into types.rs. 2017-03-30 18:03:03 -07:00
Emily Toop
8e6f37e709 #260 Convert Schema into edn::Value (#384) r=nalexander, r=rnewman
* Part 1 - Create as_edn_value function.

* Do not include defaults inside output.
* Pretty-printed by default. Do we want to make that a flag?
* Includes simple test just to make sure it works.

* Part 2 - only include ident if available.

* Part 3 - Remove spacing and newlines as unnecessary.

* Update function to build edn::Value directly rather than parsing from string

* Update test to actually test the functionality.

* Address review comments ncalexan.

 * Rename `as_edn_value` to `to_edn_value`.
 * Move `db/src/values.rs` to `core/src/values.rs` so we can reference inside `core/src/ib.rs`.
 * Add `lazy-static` crate to core `Cargo.toml`
 * Expose `values` as a public module from `core`.
 * Update references to values in `db/src/bootstrap.rs` & `db/src/lib.rs`.
 * Add new static vars for `DB_FULLTEXT`, `DB_INDEX` & `DB_IS_COMPONENT`.
 * Use static vars exposed in `values` inside `to_edn_value`.
 * Remove `db/id` as key in attribute output and use `entid` as `db/ident` if no `ident` is found for that `entid`.
 * Update test to match new expected output.

* Add doc comment for function

* Address review comments ncalexan.

* Update function docstring to give clearer description of function.
* Do not all entid at all to output.
* Clean up code fetching ident (make it rustier).

* Address review comments rnewman.

* Extract out to new `to_edn_value` functions code for creating `edn::Value`\'s for `ValueType` and `Attribute`.
* Use `map()` to create schema edn value rather than a loop.

* Address review comments rnewman.

* pass cloned instance of ident to `Attribute::get_edn_value`.
* update `use` import for `edn`.
* remove unnecessary  call when using ident as key on `associate_ident`.

* Fixed bug whereby we didn't differentiate between `db.index/value` and `db.index/identity` when generating `edn::Value`

* Add extra assert at the end to ensure we get the same output when we convert the same schema to edn multiple times

* Move check for type of uniqueness to `match` statement.

* Also use `iter` instead of `into_iter` when iterating schema map.
2017-03-30 11:08:36 +01:00
Emily Toop
b24db01744 Add tests for validate_schema_map (#391) r=rnewman
* Add tests for `validate_schema_map`

* Update test to ensure we get the right error out
2017-03-30 11:07:49 +01:00
Richard Newman
8adb6d97fd Add validation for or-join. r=nalexander 2017-03-27 16:32:45 -07:00
Richard Newman
0d15381e11 Crudely parse or and or-join. (#388) r=nalexander 2017-03-27 16:32:01 -07:00
Nick Alexander
4b874deae1 Lookup refs, nested vector values, map notation. Fixes #180, fixes #183, fixes #284. (#382) r=rnewman
* Pre: Fix error in parser macros.

* Pre: Make test unwrapping more verbose.

* Pre: Make lookup refs be (lookup-ref a v) in the entity position.

This has the advantage of being explicit in all situations and
unambiguous at parse-time.  This choice agrees with the Clojure
implementation but not with Datomic.  Datomic treats [a v] as a lookup
ref, is ambiguous at parse-time, and is disambiguated in ways I do not
understand at transaction time.  We mooted making lookup refs [[a v]]
and outlawing nested value vectors in transactions, but after
implementing that approach I decided it was better to handle lookup
refs at parse time and therefore outlawing nested value vectors is not
necessary.

* Handle lookup refs in the entity and value columns. Fixes #183.

* Pre 0a: Use a stack instead of into_iter.

* Pre 0b: Dedent.

* Pre 0c: Handle `e` after `v`.

This allows to use the original `e` while handling `v`.

* Explode value lists for :db.cardinality/many attributes. Fixes #284.

* Parse and accept map notation. Fixes #180.

* Pre: Modernize add() and retract() into one add_or_retract().

* Pre: Add is_collection and is_atom to edn::Value.

* Pre: Differentiate atoms from lookup-refs in value position.

Initially, I expected to accept arbitrary edn::Value instances in the
value position, and to differentiate in the transactor.  However, the
implementation quickly became a two-stage parser, since we always
wanted to parse the resulting value position into some other known
thing using the tx-parser.  To save calls into the parser and to allow
the parser to move forward with a smaller API surface, I push as much
of this parsing as possible into the initial parse.

* Pre: Modernize entities().

* Pre: Quote edn::Value::Text in Display.

* Review comment: Add and use edn::Value::into_atom.

* Review comment: Use skip(eof()) throughout.

* Review comment: VecDeque instead of Vec.

* Review comment: Part 0: Rename TempId to TempIdHandle.

* Review comment: Part 1: Differentiate internal and external tempids.

This breaks an abstraction boundary by pushing the Internal/External
split up to the Entity level in tx/ and tx-parser/.  This just makes
it easier to explode Entity map notation instances into Entity
instances, taking an existing External tempid :db/id or generating a
new Internal tempid as appropriate.  To do this without breaking the
abstraction boundary would require adding flexibility to the
transaction processor: we'd need to be able to turn Entity instances
into some internal enum and handle the two cases independently.  It
wouldn't be too hard, but this reduces the combinatorial type
explosion.
2017-03-27 16:30:04 -07:00
Richard Newman
88df7b3b33 Correctly generate DISTINCT and LIMIT. (#386) r=nalexander 2017-03-22 14:02:00 -07:00
Richard Newman
5e971f3b22 Post: simplify type set narrowing. 2017-03-22 11:32:32 -07:00
Richard Newman
cb4ba9e68f Post: reorganize cc.rs. 2017-03-22 11:32:32 -07:00
Richard Newman
7024978517 Track ever-shrinking sets of types for variables, not a single type. (#381) r=nalexander 2017-03-22 11:30:16 -07:00
Richard Newman
97749833d0 Algebrize and translate numeric constraints. (#306) r=nalexander 2017-03-22 10:19:47 -07:00
Richard Newman
d83c8620cd Implement parsing of query predicates. (#380) r=nalexander 2017-03-22 10:19:44 -07:00
Richard Newman
1c4e30a906 Pre: switch to taking Patterns by move, not by reference, when algebrizing. 2017-03-22 10:14:15 -07:00
Richard Newman
f5aa6b2c2c Pre: add mentat_query_algebrizer::errors. 2017-03-22 10:14:15 -07:00
Richard Newman
d8d36140a9 Pre: add tests for CC constraint intersection.
Also add a failing test for #373.
2017-03-22 10:14:15 -07:00
Richard Newman
11a9a30d35 Pre: reformat query parser code. 2017-03-22 10:14:05 -07:00
Richard Newman
fe307f8b7a Pre: remove dead code in cc.rs. 2017-03-22 10:13:58 -07:00