Commit graph

55 commits

Author SHA1 Message Date
Richard Newman
ab3600e288 Part 5: parse edn::Value::Uuid. 2017-04-28 12:56:01 -07:00
Nick Alexander
5369f03464 Improve parsing of nested edn::ValueAndSpan streams. r=rnewman (#393)
* Pre: Expose more in edn.

* Pre: Make it easier to work with ValueAndSpan.

with_spans() is a temporary hack, needed only because I don't care to
parse the bootstrap assertions from text right now.

* Part 1a: Add `value_and_span` for parsing nested `edn::ValueAndSpan` instances.

I wasn't able to abstract over `edn::Value` and `edn::ValueAndSpan`;
there are multiple obstacles.  I chose to roll with
`edn::ValueAndSpan` since it exposes the additional span information
that we will want to form good error messages in the future.

* Part 1b: Add keyword_map() parsing an `edn::Value::Vector` into an `edn::Value::map`.

* Part 1c: Add `Log`/`.log(...)` for logging parser progress.

This is a terrible hack, but it sure helps to debug complicated nested
parsers.  I don't even know what a principled approach would look
like; since our parser combinators are so frequently expressed in
code, it's hard to imagine a data-driven interpreter that can help
debug things.

* Part 2: Use `value_and_span` apparatus in tx-parser/.

I break an abstraction boundary by returning a value column
`edn::ValueAndSpan` rather than just an `edn::Value`.  That is, the
transaction processor shouldn't care where the `edn::Value` it is
processing arose -- even we care to track that information we should
bake it into the `Entity` type.  We do this because we need to
dynamically parse the value column to support nested maps, and parsing
requires a full `edn::ValueAndSpan`.  Alternately, we could cheat and
fake the spans when parsing nested maps, but that's potentially
expensive.

* Part 3: Use `value_and_span` apparatus in query-parser/.

* Part 4: Use `value_and_span` apparatus in root crate.

* Review comment: Make Span and SpanPosition Copy.

* Review comment: nits.

* Review comment: Make `or` be `or_exactly`.

I baked the eof checking directly into the parser, rather than using
the skip and eof parsers.  I also took the time to restore some tests
that were mistakenly commented out.

* Review comment: Extract and use def_matches_* macros.

* Review comment: .map() as late as possible.
2017-04-06 10:06:28 -07:00
Nick Alexander
4b874deae1 Lookup refs, nested vector values, map notation. Fixes #180, fixes #183, fixes #284. (#382) r=rnewman
* Pre: Fix error in parser macros.

* Pre: Make test unwrapping more verbose.

* Pre: Make lookup refs be (lookup-ref a v) in the entity position.

This has the advantage of being explicit in all situations and
unambiguous at parse-time.  This choice agrees with the Clojure
implementation but not with Datomic.  Datomic treats [a v] as a lookup
ref, is ambiguous at parse-time, and is disambiguated in ways I do not
understand at transaction time.  We mooted making lookup refs [[a v]]
and outlawing nested value vectors in transactions, but after
implementing that approach I decided it was better to handle lookup
refs at parse time and therefore outlawing nested value vectors is not
necessary.

* Handle lookup refs in the entity and value columns. Fixes #183.

* Pre 0a: Use a stack instead of into_iter.

* Pre 0b: Dedent.

* Pre 0c: Handle `e` after `v`.

This allows to use the original `e` while handling `v`.

* Explode value lists for :db.cardinality/many attributes. Fixes #284.

* Parse and accept map notation. Fixes #180.

* Pre: Modernize add() and retract() into one add_or_retract().

* Pre: Add is_collection and is_atom to edn::Value.

* Pre: Differentiate atoms from lookup-refs in value position.

Initially, I expected to accept arbitrary edn::Value instances in the
value position, and to differentiate in the transactor.  However, the
implementation quickly became a two-stage parser, since we always
wanted to parse the resulting value position into some other known
thing using the tx-parser.  To save calls into the parser and to allow
the parser to move forward with a smaller API surface, I push as much
of this parsing as possible into the initial parse.

* Pre: Modernize entities().

* Pre: Quote edn::Value::Text in Display.

* Review comment: Add and use edn::Value::into_atom.

* Review comment: Use skip(eof()) throughout.

* Review comment: VecDeque instead of Vec.

* Review comment: Part 0: Rename TempId to TempIdHandle.

* Review comment: Part 1: Differentiate internal and external tempids.

This breaks an abstraction boundary by pushing the Internal/External
split up to the Entity level in tx/ and tx-parser/.  This just makes
it easier to explode Entity map notation instances into Entity
instances, taking an existing External tempid :db/id or generating a
new Internal tempid as appropriate.  To do this without breaking the
abstraction boundary would require adding flexibility to the
transaction processor: we'd need to be able to turn Entity instances
into some internal enum and handle the two cases independently.  It
wouldn't be too hard, but this reduces the combinatorial type
explosion.
2017-03-27 16:30:04 -07:00
Richard Newman
85f3b79f75 Support a limited set of '.'-prefixed non-keyword symbols. (#352) r=nalexander
This commit allows `.` and `...` to parse correctly as `PlainSymbol`.

Tests in edn, query-translator, and the top level have been added.
2017-03-06 15:01:19 -08:00
Nick Alexander
dcd9bcb1ce Extract partial storage abstraction; use error-chain throughout. Fixes #328. r=rnewman (#341)
* Pre: Drop unneeded tx0 from search results.

* Pre: Don't require a schema in some of the DB code.

The idea is to separate the transaction applying code, which is
schema-aware, from the concrete storage code, which is just concerned
with getting bits onto disk.

* Pre: Only reference Schema, not DB, in debug module.

This is part of a larger separation of the volatile PartitionMap,
which is modified every transaction, from the stable Schema, which is
infrequently modified.

* Pre: Fix indentation.

* Extract part of DB to new SchemaTypeChecking trait.

* Extract part of DB to new PartitionMapping trait.

* Pre: Don't expect :db.part/tx partition to advance when tx fails.

This fails right now, because we allocate tx IDs even when we shouldn't.

* Sketch a db interface without DB.

* Add ValueParseError; use error-chain in tx-parser.

This can be simplified when
https://github.com/Marwes/combine/issues/86 makes it to a published
release, but this unblocks us for now.  This converts the `combine`
error type `ParseError<&'a [edn::Value]>` to a type with owned
`Vec<edn::Value>` collections, re-using `edn::Value::Vector` for
making them `Display`.

* Pre: Accept Borrow<Schema> instead of just &Schema in debug module.

This makes it easy to use Rc<Schema> or Arc<Schema> without inserting
&* sigils throughout the code.

* Use error-chain in query-parser.

There are a few things to point out here:

- the fine grained error types have been flattened into one crate-wide
  error type; it's pretty easy to regain the granularity as needed.

- edn::ParseError is automatically lifted to
  mentat_query_parser::errors::Error;

- we use mentat_parser_utils::ValueParser to maintain parsing error
  information from `combine`.

* Patch up top-level.

* Review comment: Only `borrow()` once.
2017-02-24 15:33:48 -08:00
Victor Porof
7fc2a22d68 Implement a basic edn matcher, r=ncalexan (#271) (#338)
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-23 09:11:34 +01:00
Victor Porof
bf707acbc3 Lint for the clippy gods in the edn crate (#340)
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-22 18:11:05 +01:00
Victor Porof
0d3b8e4b29 Avoid code duplication for common Value trait implementations, r=ncalexan
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-22 08:36:45 +01:00
Victor Porof
1b26e23d02 Implement edn pretty printing using pretty.rs. Fixes #195. (#245)
* Implement pretty printing

Signed-off-by: Victor Porof <victor.porof@gmail.com>

* Rewrite pretty printing.

This does a few things.  First, it use pretty.rs directly, without the
layer of macro obfuscation.  The code is significantly simpler as a
result.

Second, it tightens the layout, using pretty.rs to group nested
layouts that fit on a single line.  This is Clojure's EDN style, more
or less.

Third, it drops "special format" support for queries.  This wasn't
completely implemented; if we want it, we can newtype
Query(edn::Value) and figure out how to really implement this idea.

* Rename to reflect functionality.

* Make write interface more Rust-like.

There isn't a clear standard in the stdlib, but a function that takes
ownership of a writer and then returns it back is definitely not
Rust-like.  That's what a (mutable) reference is for.

* Review comment: Use as_ref to avoid cloning strings.

* Post: Fix tests to use `without_spans()`.
2017-02-21 11:48:08 -08:00
Richard Newman
a10f68fdb7 Mark every project as being part of the workspace. r=nalexander
This allows `cargo test --all` to work.
2017-02-20 11:04:08 -08:00
Jordan Santell
a59f9583ac Store Idents as NamespacedKeywords, rather than Strings. Fixes #291. (#300)
r=ncalexander
2017-02-17 13:55:36 -08:00
Victor Porof
896d7f8f88 Add a span component to edn::Value, r=ncalexan
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-17 18:31:26 +01:00
Joe Walker
89949fb451 Update README for edn; r=me 2017-02-16 18:32:36 +00:00
Jordan Santell
4f5c94891a Add octal, hexadecimal, and arbitrary base integers to the EDN parser. Fixes #277. r=rnewman (#286) 2017-02-10 16:03:35 -08:00
Victor Porof
49f91b05b0 Expose EDN into_ methods that consume the edn value, r=jwalker. Fixes 256
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-10 14:58:38 +01:00
Richard Newman
4366f6d61f Expose EDN as_ methods that return copied values, not references. 2017-02-09 10:21:35 -08:00
Victor Porof
42580539b8 Properly handle whitespace for Infinity and NaN, r=rnewman (#246)
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-09 18:13:44 +01:00
Victor Porof
c585715224 Don't depend on num and ordered-float in the db and query crates, r=ncalexan (#223)
Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-08 12:19:16 +01:00
Victor Porof
4d83aafa2a Ensure printing/parsing edn strings is isomorphic
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-04 08:47:48 +01:00
Victor Porof
a627f532f0 Relax whitespace rules for edn vectors, lists, sets and maps
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-04 08:45:31 +01:00
Victor Porof
419db388da Relax whitespace rules for Infinity and NaN
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-04 08:45:02 +01:00
Jordan Santell
0b20d7691b Parse and display EDN values for NaN, +Infinity and -Infinity. Fixes #232 (#238) r=victorporof 2017-02-03 10:14:23 -08:00
Victor Porof
c038c11017 Consolidate edn peg rules to better parse keywords and symbols, r=ncalexan. Fixes #219 2017-02-03 09:08:24 +01:00
Victor Porof
9ee0ac8e00 Unify and generalize keywords and symbols parsing
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-03 09:06:42 +01:00
Victor Porof
72da5722ae Update rustpeg to latest version and follow new syntax and formatting rules
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-03 09:06:42 +01:00
Victor Porof
611fbe2eef Properly print null edn values as "nil", to allow for isomorphic write/parse
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-03 09:06:42 +01:00
Richard Newman
5b770a54cd Parse basic :find and :where clauses. (#211) r=nalexander
* Make Variable::from_symbol public.
* Implement basic parsing of queries.
* Use pinned dependencies the hard way to fix Travis.
* Bump ordered-float dependency to 0.4.0.
* Error coercions to use ?, and finishing the find interface.
2017-02-02 18:32:00 -08:00
Victor Porof
707ce36236 Don't use single-character string constants in the is_backward function
See https://github.com/Manishearth/rust-clippy/wiki#single_char_pattern for further info

Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 19:55:29 +01:00
Victor Porof
9a5ece8c89 Handle more edn::Value types for printing, precursor for #195
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 17:22:58 +01:00
Victor Porof
2ecda0a2bd Avoid needless reborrows and simplify Ord implementation for edn::Value 2017-02-02 17:22:58 +01:00
Victor Porof
a685d6c541 Move edn test functions into a submodule
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 17:22:58 +01:00
Victor Porof
cc56cec11a Add note about linked lists data type choice for edn::Value 2017-02-02 17:22:58 +01:00
Victor Porof
00048d1955 Remove edn::Pair struct since it's not used anywhere
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 17:22:58 +01:00
Victor Porof
4e9e8ed837 Use idiomatic enumerate method on interators instead of iterating over indices
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:59:03 +01:00
Victor Porof
4b2c7870c0 Wrap code indicated by the "_" in documentation as suggested by rustdoc best practices
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:59:03 +01:00
Victor Porof
17bc85fe27 Remove return statements from edn parser tests
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:59:03 +01:00
Victor Porof
25474980b1 Add rustdoc comments for to_symbol and to_keyword functions
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:59:03 +01:00
Victor Porof
8f68f68378 Use idiomatic map_or_else calls to Option<T> instead of double returns
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:59:03 +01:00
Victor Porof
85da91a0ab Add helper functions constructing OrderedFloat and BigInt to edn crate, r=ncalexan,rnewman. Fixes #198
Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:58:08 +01:00
Richard Newman
a9929249eb Use Into<Option<>> trick for to_keyword and to_symbol. 2017-02-01 17:46:53 -08:00
Richard Newman
0b3387f8b9 Minor EDN cleanup. (#217) r=jsantell
* to_reverse -> to_reversed.
* Add PlainSymbol::plain_name for examining $x and ?y.
* Fix comment.
2017-02-01 14:34:51 -08:00
Richard Newman
932a42866c Fix comments in EDN crate. 2017-01-30 17:45:02 -08:00
Jordan Santell
0a21b5dca4 Add NamespacedKeyword::{is_forward, is_backward, to_reverse} for
inspection in NamespacedKewyord for usage by query and transaction parsers. Fixes #202
2017-01-30 14:39:53 -08:00
Jordan Santell
359d356dd9 Merge pull request #213 from mozilla/199
Add Into<String> to symbol::* constructors. Fixes #199
2017-01-30 13:35:12 -08:00
Jordan Santell
d116fd7bff Add is_$type and as_$type methods to edn::Value types and add tests. Fixes #197 2017-01-30 09:35:33 -08:00
Jordan Santell
18279fdd3c Add Into<String> to symbol::* constructors. Fixes #199 2017-01-28 22:53:40 -08:00
Richard Newman
c6fa14c0c8 Rudimentary printing of EDN values. (#209) r=jsantell
* Add a little From helper for edn::parse::ParseError. Not used yet.

* Ignore more things.

* Partly implement Display for edn::Value.
2017-01-28 14:18:17 -08:00
Nick Alexander
81af295948 Start installing SQL schema. (#171) r=rnewman
* Start installing the SQLite store and bootstrapping the datom store.

* Review comment: Decomplect V2_IDENTS.

* Review comment: Decomplect V2_PARTS.

* Review comment: Pre: Expose Clojure's merge on Value instances.

* Review comment: Decomplect V2_SYMBOLIC_SCHEMA.

* Review comment: Decomplect V1_STATEMENTS.

* Review comment: Prefer ? to try!.

* Review comment: Fix typos; format; add TODOs.

* Review comment: Assert that Mentat `Schema` is valid upon creation.

* Review comment: Improve conversion to and from SQL values.

This patch factors the fundamental SQL conversion maps
between (rusqlite::Value, value_type_tag) and (edn::Value, ValueType)
through a new Mentat TypedValue.  (A future patch might rename this
fundamental type mentat::Value.)

To make certain conversion functions infallible, I removed
placeholders for :db.type/{instant,uuid,uri}.  (We could panic
instead, but there's no need to do that right now.)

* Review comment: Always uses bundled SQLite in rusqlite.

This avoids (runtime) failures in Travis CI due to old SQLite
versions.  See 432966ac77.

* Review comment: Move semantics in `from_sql_value_pair`.

* Review comment: DB_EXCISE_BEFORE_T instead of ...BEFORET (no underscore).

* Review comment: Move overview notes to the Wiki.
2017-01-25 16:13:56 -08:00
Richard Newman
2592506288 Implement parsing of simple :find expressions. (#196) r=nalexander
* Test the mentat_query directory on Travis.

* Export common types from edn.

This allows you to write

  use edn::{PlainSymbol,Keyword};

instead of

  use edn:🔣:{PlainSymbol,Keyword};

* Add an edn::Value::is_keyword predicate.

* Clean up query, preparing for query-parser.

* Make EDN keywords and symbols take Into<String> arguments.

* Implement parsing of simple :find lists.

* Rustfmt query-parser. Split find and query.

* Review comment: values_to_variables now returns a NotAVariableError on failure.

* Review comment: rename gimme to to_parsed_value.

* Review comment: add comments.
2017-01-25 14:06:19 -08:00
Nick Alexander
ab041291fb edn: Bound values by optional whitespace; treat comma as whitespace. 2017-01-18 08:34:27 -08:00