This is a big commit, but it breaks into two conceptual pieces. The
first is to "parse without copying". We replace a stream of an owned
collection of edn::ValueAndSpan and instead have a stream of a
borrowed collection of &edn::ValueAndSpan references. (Generally,
this is represented as an iterator over a slice, but it can be over
other things too.) Cloning such iterators is constant time, which
improves on cloning an owned collection of edn::ValueAndSpan, which is
linear time in the length of the collection and additional time
depending on the complexity of the EDN values.
The second conceptual piece is to parse keyword maps using a special
parser and a macro to build the parser implementations. Before, we
created a new edn::ValueAndSpan::Map to represent a keyword map in
vector form; since we're working with &edn::ValueAndSpan references
now, we can't create an &edn::ValueAndSpan reference with an
appropriate lifetime. Therefore we generalize the concept of
iteration slightly and turn keyword maps in map form into linear
iterators by flattening the value maps. This is a potentially
obscuring transformation, so we have to take care to protect against
some failure cases. (See the comments and the tests in the code.)
After these changes, parsing using `combine` is linear time (and
reasonably fast).
This doesn't yet introduce a working Cargo.toml for 'mentatweb', but it
does allow RLS to build correctly without errors, and it reduces the
core library's dependency space, which is more important in the short
term.
* Pre: unused import in translate.rs.
* Part 2: take a dependency on rusqlite for query arguments.
* Part 1: flatten V2 schema into V1. Add UUID and URI.
Bump expected ident and bootstrap datom count in tests.
* Part 5: parse edn::Value::Uuid.
* Part 3: extend ValueType and TypedValue to include Uuid.
* Part 4: add Uuid to query arguments.
* Part 6: extend db to support Uuid.
* Part 8: add a tx-parser test for #f NaN and #uuid.
* Part 7: parse and algebrize UUIDs in queries.
* Part 1: parse #inst in EDN and throughout query engine.
* Part 3: handle instants in db.
* Part 2: instants never matches integers in queries.
* Part 4: use DateTime for tx_instants.
* Add a test for adding and querying UUIDs and instants.
* Review comments.
* Part 1: added limits feature to rusqlite dependencies.
* Part 2: replace references to SQLITE_MAX_VARIABLE_NUMBER with sqlite3_limit.
* Move assertion check for correct number of variables in repeat_values to before call as this is where the variable is defined.
* Part 3: add tests
* Add a failing test for EDN parsing '…'.
* Expose a SQLValueType trait to get value_type_tag values out of a ValueType.
* Add accessors to FindSpec.
* Implement querying.
* Implement rudimentary projection.
* Export mentat_db::new_connection.
* Export symbols from mentat.
* Add rudimentary end-to-end query tests.
* Pre: Drop unneeded tx0 from search results.
* Pre: Don't require a schema in some of the DB code.
The idea is to separate the transaction applying code, which is
schema-aware, from the concrete storage code, which is just concerned
with getting bits onto disk.
* Pre: Only reference Schema, not DB, in debug module.
This is part of a larger separation of the volatile PartitionMap,
which is modified every transaction, from the stable Schema, which is
infrequently modified.
* Pre: Fix indentation.
* Extract part of DB to new SchemaTypeChecking trait.
* Extract part of DB to new PartitionMapping trait.
* Pre: Don't expect :db.part/tx partition to advance when tx fails.
This fails right now, because we allocate tx IDs even when we shouldn't.
* Sketch a db interface without DB.
* Add ValueParseError; use error-chain in tx-parser.
This can be simplified when
https://github.com/Marwes/combine/issues/86 makes it to a published
release, but this unblocks us for now. This converts the `combine`
error type `ParseError<&'a [edn::Value]>` to a type with owned
`Vec<edn::Value>` collections, re-using `edn::Value::Vector` for
making them `Display`.
* Pre: Accept Borrow<Schema> instead of just &Schema in debug module.
This makes it easy to use Rc<Schema> or Arc<Schema> without inserting
&* sigils throughout the code.
* Use error-chain in query-parser.
There are a few things to point out here:
- the fine grained error types have been flattened into one crate-wide
error type; it's pretty easy to regain the granularity as needed.
- edn::ParseError is automatically lifted to
mentat_query_parser::errors::Error;
- we use mentat_parser_utils::ValueParser to maintain parsing error
information from `combine`.
* Patch up top-level.
* Review comment: Only `borrow()` once.
Printing out failure to meet rustc version helps users during
setup with a helpful message if using an older rustc.
Rust version checking from http://stackoverflow.com/a/36607492.
* Pre: Add some value conversion tests.
This is follow-up to earlier work. Turn TypedValue::Keyword into
edn::Value::NamespacedKeyword. Don't take a reference to
value_type_tag.
* Pre: Add repeat_values.
Requires itertools, so this commit is not stand-alone.
* Pre: Expose the first transaction ID as bootstrap::TX0.
This is handy for testing.
* Pre: Improve debug module.
* Pre: Bump rusqlite version for https://github.com/jgallagher/rusqlite/issues/211.
* Pre: Use itertools.
* Start implementing bulk SQL insertion algorithms. (#214)
This is slightly simpler re-expression of the existing Clojure
implementation.
* Post: Start generic data-driven transaction testing. (#188)
* Review comment: `use ::{SYMBOL}` instead of `use {SYMBOL}`.
* Review comment: Prefer bindings_per_statement to values_per_statement.
* Start installing the SQLite store and bootstrapping the datom store.
* Review comment: Decomplect V2_IDENTS.
* Review comment: Decomplect V2_PARTS.
* Review comment: Pre: Expose Clojure's merge on Value instances.
* Review comment: Decomplect V2_SYMBOLIC_SCHEMA.
* Review comment: Decomplect V1_STATEMENTS.
* Review comment: Prefer ? to try!.
* Review comment: Fix typos; format; add TODOs.
* Review comment: Assert that Mentat `Schema` is valid upon creation.
* Review comment: Improve conversion to and from SQL values.
This patch factors the fundamental SQL conversion maps
between (rusqlite::Value, value_type_tag) and (edn::Value, ValueType)
through a new Mentat TypedValue. (A future patch might rename this
fundamental type mentat::Value.)
To make certain conversion functions infallible, I removed
placeholders for :db.type/{instant,uuid,uri}. (We could panic
instead, but there's no need to do that right now.)
* Review comment: Always uses bundled SQLite in rusqlite.
This avoids (runtime) failures in Travis CI due to old SQLite
versions. See 432966ac77.
* Review comment: Move semantics in `from_sql_value_pair`.
* Review comment: DB_EXCISE_BEFORE_T instead of ...BEFORET (no underscore).
* Review comment: Move overview notes to the Wiki.
Starting to work out the project layout for sub-crates. The crate inside query-parser/ is "datomish-query-parser" and the core code in src/ depends on it.
