Commit graph

661 commits

Author SHA1 Message Date
Emily Toop 9f3d2c08b2 Populate changeset of attributes inside TxReport during transact.
Batch up TxReports for entire transaction.
Notify observers about committed transaction.
Store transaction observer service inside Conn
2018-03-20 16:27:35 +00:00
Emily Toop 8d60f2b3d1 Expose tx observation publicly 2018-03-20 16:27:35 +00:00
Emily Toop dfa0d3e321 Add TxObservers, Commands and TxObservationService.
These are the base types upon which we will build our transaction observation
2018-03-20 16:27:35 +00:00
Richard Newman f71b2b207e Expand the README to give a guide to the crates in the repo. 2018-03-19 14:35:18 -07:00
Richard Newman 16a66517e4
Err, don't panic, on unbound variable (#518) (#590) r=emily
* Pre: switch a 'panic' to an 'unreachable'.
* Make candidate_type_column fallible. (#518)
2018-03-15 11:39:24 -07:00
Richard Newman 994a3e65e2
Tests and fixes for aggregates over different or unknown types. (#588) r=emily 2018-03-15 07:14:06 -07:00
Richard Newman df58de52f4
Correctly parse and unescape quotes etc. inside EDN strings. (#434) (#589) 2018-03-15 07:13:27 -07:00
Richard Newman ea52e214af
Small README tweaks. 2018-03-13 04:01:19 +00:00
Richard Newman 833ff92436
Simple aggregates. (#584) r=emily
* Pre: use debugcli in VSCode.
* Pre: wrap subqueries in parentheses in output SQL.
* Pre: add ExistingColumn.

This lets us make reference to columns by name, rather than only
pointing to qualified aliases.

* Pre: add Into for &str to TypedValue.
* Pre: add Store.transact.
* Pre: cleanup.
* Parse and algebrize simple aggregates. (#312)
* Follow-up: print aggregate columns more neatly in the CLI.
* Useful ValueTypeSet helpers.
* Allow for entity inequalities.
* Add 'differ', which is a ref-specialized not-equals.
* Add 'unpermute', a function for getting unique, distinct pairs from bindings.
* Review comments.
* Add 'the' pseudo-aggregation operator.

This allows for a corresponding value to be returned when a query
includes one 'min' or 'max' aggregate.
2018-03-12 15:18:50 -07:00
Richard Newman 46835885e4 Add help for timer command. 2018-03-06 10:49:41 -08:00
Richard Newman 3e615becd8 Fix printing of fractional millisecond timestamps. (#582) r=emily 2018-03-06 10:49:41 -08:00
Richard Newman 1817ce7c0b Performance and cleanup. r=emily
* Use fixed-size arrays for bootstrap datoms, not vecs.
* Wide-ranging cleanup.

    This commit:
    - Deletes some dead code.
    - Marks some functions only used by tests as cfg(test).
    - Adds pub(crate) to a bunch of functions.
    - Cleans up a few other nits.
2018-03-06 09:03:00 -08:00
Richard Newman a0c70a7cd9 Add caching example to Seattle fixture. 2018-03-06 09:02:11 -08:00
Richard Newman f42ae35b70 Update cache on write. (#566) r=emily
* Use the cache to make constant queries super fast.
* Fix translate tests to match: we no longer generate SQL for many of them!
* Accumulate additions and removals into the cache.
    * Make attribute cache clone-on-write; store it in Metadata.
    * Allow caching of fulltext attributes, interning strings.
2018-03-06 09:01:20 -08:00
Richard Newman bead9752bd Add InProgress.import to load transaction data from a file. r=emily
It's not streaming, but it'll do.
2018-03-06 08:59:30 -08:00
Richard Newman 0ca61017a1 Use LTO for release builds. 2018-03-06 08:19:17 -08:00
Richard Newman 4a0b67ab50 Require Rust 0.1.24 or higher. 2018-03-06 08:19:17 -08:00
Richard Newman 9b23cf3945
Speed up EDN parser (fixes #445) (#581) r=nalexander
Fixes from @kevinmehall.

* Prefer character sets over backtracking in the EDN parser.
* Avoid duplicate effort when parsing floats in the EDN parser.
* Clean up duplicate position tracking code.

This turns out to have little performance impact, but makes the grammar
much cleaner.

* Fix EDN work to pass tests with correct numeric precedence.
2018-03-05 20:33:51 -08:00
Richard Newman 30bf827d16
CLI improvements (#577) r=grisha
* Add a prepared query command to CLI.
* Print nanoseconds in the REPL. This is a good problem to have.
* Better CLI timing.
* Use release for 'cargo cli', debug for 'cargo debugcli'.
* Don't enable debug symbols in release builds.
* Clean up CLI code. Fixed order for help.
* Column-align help output.
2018-03-05 12:52:20 -08:00
Richard Newman d46535a7c2
When an attribute is known-fulltext, don't hit AllDatoms. (#576) r=nalexander 2018-03-05 10:09:53 -08:00
Grisha Kruglov 36d455150d
Disable TLS support, add links to issues for TODOs (#573) r=grisha/self
Landing this now to unblock Android builds of mentat until the cross-compilation of dependencies is figured out.
2018-02-28 15:54:46 -08:00
Richard Newman 5e50d2a9b4 Update to rusqlite 0.13. 2018-02-22 11:41:57 -08:00
Richard Newman 39cf26aa76 Add import command. (#456) r=emily 2018-02-21 11:51:45 -08:00
Richard Newman c53da08a00 Add VSCode command to attach to running CLI. 2018-02-21 11:51:45 -08:00
Richard Newman 23ebc3a5fc Add Datomic's Seattle neighborhood test data. 2018-02-21 11:51:45 -08:00
Richard Newman 20d1b45293 Follow-up: replace println_stderr with eprintln. 2018-02-21 11:51:45 -08:00
Richard Newman 54bd883c65 Follow-up: remove logging and such elsewhere in the codebase. 2018-02-21 11:51:45 -08:00
Richard Newman e33fe71c47 Rework caching and use it inside the query engine. (#553) r=emily
This puts caching in mentat_db, adds a reverse lookup capability for
unique attributes, and populates bidirectional caches with a single
SQL cursor walk.

Differentiate between begin_read and begin_uncached_read.

Note that we still allow toggling within InProgress, because there might be
transient local state that makes starting a new transaction impossible.
2018-02-21 11:51:45 -08:00
Richard Newman df3cdb5db6 Allow two datoms in the same transaction to have the same fulltext string. (#565) r=emily 2018-02-21 11:51:45 -08:00
Richard Newman ae91603bd0 Provide an API for creating truly empty stores (#561) r=grisha
* Part 1: split create_current_version.

* Part 2: add Store::create_empty and Conn::empty.

* Part 3 - Expose 'open_empty' command via CLI
2018-02-16 02:01:00 -08:00
Grisha Kruglov 93e5dff9c8
Revised uploader flow (battle-tested); CLI support for sync (#557) r=rnewman 2018-02-16 01:44:28 -08:00
Emily Toop c84db82575
Merge pull request #560 from mozilla/fluffyemily/entity-builder-update
Add retract_kw
2018-02-15 18:48:39 +00:00
Emily Toop 48ffa20d4c Add retract_kw 2018-02-15 18:47:58 +00:00
Emily Toop 9655c4b85d Add retract to entity builder. (#559) r=rnewman 2018-02-15 09:44:06 -08:00
Richard Newman 01d9b83a9b
Add EntityBuilder.add_kw. (#558) r=emily
* Add EntityBuilder.add_kw.

This allows you to skip your own attribute lookups, at the cost of
potentially doing the work more than once.

Also does value type checking.
2018-02-15 08:22:34 -08:00
Richard Newman 0dcb7df1c7
Timing and colorizing in the CLI. (#552) r=grisha
* Add basic coloring of CLI output.
* Add a timer to the CLI.

Toggle it on or off with 'timer on' and 'timer off'.
Output is colorized.

* Add VSCode configuration files.

These allow you to build and run the CLI, build Mentat, or run all tests.
2018-02-15 07:36:18 -08:00
Emily Toop dec86bb2c5
publically expose KnownEntid (#554) 2018-02-14 19:06:50 +00:00
Richard Newman 23e11fabe6
Add a var! macro. (#548)(#550) r=emily 2018-02-14 09:32:37 -08:00
Richard Newman 2ac7a1b1de Add a feature flag to control the use of rusqlite's bundled SQLite. r=emily
You can use this in conjunction with setting SQLITE3_LIB_DIR to control which SQLite is used.

See https://github.com/jgallagher/rusqlite for more.

Also add recent contributors to the authors array.
2018-02-13 08:25:58 -08:00
Grisha Kruglov 84f29676e8
"Unchanged server" uploader flow (#543) r=rnewman
* Remove unused struct from tx_processor

* Derive serialize & deserialize for TypedValue

* First pass of uploader flow + feedback
2018-02-09 09:55:19 -08:00
Richard Newman d11810dca7 Fix warning in tx.rs. 2018-02-09 08:51:13 -08:00
Kit Cambridge a6341f6fd6 Implement q_prepare with pre-bound variables. r=rnewman 2018-02-07 21:48:05 -08:00
Emily Toop 715d434945
Create generalized in-memory cache for attributes (#525)
* Nit: Alphabetical ordering of imports

* Create Cache and provide functions for calling it

* Get tests working. Move to using NamespacedKeyword over KnownEntid in function signature

* Add is_cached check to caching tests

* Move lazy and add/remove boolean flags to enums

* Move function definitions into generic trait and implement trait for AttributeCache

* Remove lazy cache and generalize cache

* Update tests

* Eager cache becomes simple key value store. AttributeMap handles attribute storing specifics

* Update tests to test presence of correct values in cache

* Move EagerCache, AttributeValueProvider and ValueProvider into mentat_db

* Add test for get_for_entid

* Add test for lookup attribute

* Make caches cloneable. Add value_for alongside values_for

* Use cache in attribute lookups

* Split test for values and value and add cardinality

* address review feedback r=rnewman
2018-02-07 10:56:12 -08:00
Grisha Kruglov d848d954cf Issue 508 - Iterating transcation processor r=rnewman
Review comments
2018-02-06 12:24:12 -08:00
Richard Newman e9ed103723 Revert "Make debug structs and functions non-public. r=grisha"
This reverts commit e817f67470.
2018-02-06 11:06:06 -08:00
Richard Newman e817f67470 Make debug structs and functions non-public. r=grisha 2018-02-06 10:25:42 -08:00
Richard Newman 66e6fef75e Define Store, use TabWriter in the CLI for aligning columnar output. (#540) r=emily
* Define Store, which is a simple container for a SQLite connection and a Conn.
  This is a breaking change.
* Return the FindSpec as part of QueryOutput, not just results.
* Switch to using stderr in appropriate places in CLI.
* Print columns in CLI output.
2018-02-01 09:29:07 -08:00
Richard Newman 37a7c9ea48 Validate attributes installed after open. (#538) r=emily
Make AttributeBuilder optionally helpful, fix tests.
2018-02-01 09:29:04 -08:00
Richard Newman 2614f498be Ergonomics improvements, including a kw macro. (#537) r=emily
* Add TypedValue::instant(micros).
* Add From<f64> for TypedValue.
* Add lookup_values_for_attribute to Conn.
* Add q_explain to Queryable.
* Expose an iterator over FindSpec's columns.
* Export edn from mentat crate. Export QueryExecutionResult.
* Implement Display for Variable and Element.
* Introduce a `kw` macro.

    This allows you to write:

    ```rust
    kw!(:foo/bar)
    ```

    instead of

    ```rust
    NamespacedKeyword::new("foo", "bar")
    ```

    … and it's more efficient, too.

Add `mentat::open`, eliminate use of `mentat_db` in some places.
2018-02-01 09:27:23 -08:00
Edouard Oger 3bf7459315 Allow customers to assert facts about the current transaction. (#225) r=rnewman
Also move `now` into core, implement microsecond truncation.

This is so we don't return a more granular -- and thus subtly different --
timestamp in a `TxReport` than we put into the store.
2018-01-29 16:46:04 -08:00