Commit graph

648 commits

Author SHA1 Message Date
Richard Newman f42ae35b70 Update cache on write. (#566) r=emily
* Use the cache to make constant queries super fast.
* Fix translate tests to match: we no longer generate SQL for many of them!
* Accumulate additions and removals into the cache.
    * Make attribute cache clone-on-write; store it in Metadata.
    * Allow caching of fulltext attributes, interning strings.
2018-03-06 09:01:20 -08:00
Richard Newman bead9752bd Add InProgress.import to load transaction data from a file. r=emily
It's not streaming, but it'll do.
2018-03-06 08:59:30 -08:00
Richard Newman 0ca61017a1 Use LTO for release builds. 2018-03-06 08:19:17 -08:00
Richard Newman 4a0b67ab50 Require Rust 0.1.24 or higher. 2018-03-06 08:19:17 -08:00
Richard Newman 9b23cf3945
Speed up EDN parser (fixes #445) (#581) r=nalexander
Fixes from @kevinmehall.

* Prefer character sets over backtracking in the EDN parser.
* Avoid duplicate effort when parsing floats in the EDN parser.
* Clean up duplicate position tracking code.

This turns out to have little performance impact, but makes the grammar
much cleaner.

* Fix EDN work to pass tests with correct numeric precedence.
2018-03-05 20:33:51 -08:00
Richard Newman 30bf827d16
CLI improvements (#577) r=grisha
* Add a prepared query command to CLI.
* Print nanoseconds in the REPL. This is a good problem to have.
* Better CLI timing.
* Use release for 'cargo cli', debug for 'cargo debugcli'.
* Don't enable debug symbols in release builds.
* Clean up CLI code. Fixed order for help.
* Column-align help output.
2018-03-05 12:52:20 -08:00
Richard Newman d46535a7c2
When an attribute is known-fulltext, don't hit AllDatoms. (#576) r=nalexander 2018-03-05 10:09:53 -08:00
Grisha Kruglov 36d455150d
Disable TLS support, add links to issues for TODOs (#573) r=grisha/self
Landing this now to unblock Android builds of mentat until the cross-compilation of dependencies is figured out.
2018-02-28 15:54:46 -08:00
Richard Newman 5e50d2a9b4 Update to rusqlite 0.13. 2018-02-22 11:41:57 -08:00
Richard Newman 39cf26aa76 Add import command. (#456) r=emily 2018-02-21 11:51:45 -08:00
Richard Newman c53da08a00 Add VSCode command to attach to running CLI. 2018-02-21 11:51:45 -08:00
Richard Newman 23ebc3a5fc Add Datomic's Seattle neighborhood test data. 2018-02-21 11:51:45 -08:00
Richard Newman 20d1b45293 Follow-up: replace println_stderr with eprintln. 2018-02-21 11:51:45 -08:00
Richard Newman 54bd883c65 Follow-up: remove logging and such elsewhere in the codebase. 2018-02-21 11:51:45 -08:00
Richard Newman e33fe71c47 Rework caching and use it inside the query engine. (#553) r=emily
This puts caching in mentat_db, adds a reverse lookup capability for
unique attributes, and populates bidirectional caches with a single
SQL cursor walk.

Differentiate between begin_read and begin_uncached_read.

Note that we still allow toggling within InProgress, because there might be
transient local state that makes starting a new transaction impossible.
2018-02-21 11:51:45 -08:00
Richard Newman df3cdb5db6 Allow two datoms in the same transaction to have the same fulltext string. (#565) r=emily 2018-02-21 11:51:45 -08:00
Richard Newman ae91603bd0 Provide an API for creating truly empty stores (#561) r=grisha
* Part 1: split create_current_version.

* Part 2: add Store::create_empty and Conn::empty.

* Part 3 - Expose 'open_empty' command via CLI
2018-02-16 02:01:00 -08:00
Grisha Kruglov 93e5dff9c8
Revised uploader flow (battle-tested); CLI support for sync (#557) r=rnewman 2018-02-16 01:44:28 -08:00
Emily Toop c84db82575
Merge pull request #560 from mozilla/fluffyemily/entity-builder-update
Add retract_kw
2018-02-15 18:48:39 +00:00
Emily Toop 48ffa20d4c Add retract_kw 2018-02-15 18:47:58 +00:00
Emily Toop 9655c4b85d Add retract to entity builder. (#559) r=rnewman 2018-02-15 09:44:06 -08:00
Richard Newman 01d9b83a9b
Add EntityBuilder.add_kw. (#558) r=emily
* Add EntityBuilder.add_kw.

This allows you to skip your own attribute lookups, at the cost of
potentially doing the work more than once.

Also does value type checking.
2018-02-15 08:22:34 -08:00
Richard Newman 0dcb7df1c7
Timing and colorizing in the CLI. (#552) r=grisha
* Add basic coloring of CLI output.
* Add a timer to the CLI.

Toggle it on or off with 'timer on' and 'timer off'.
Output is colorized.

* Add VSCode configuration files.

These allow you to build and run the CLI, build Mentat, or run all tests.
2018-02-15 07:36:18 -08:00
Emily Toop dec86bb2c5
publically expose KnownEntid (#554) 2018-02-14 19:06:50 +00:00
Richard Newman 23e11fabe6
Add a var! macro. (#548)(#550) r=emily 2018-02-14 09:32:37 -08:00
Richard Newman 2ac7a1b1de Add a feature flag to control the use of rusqlite's bundled SQLite. r=emily
You can use this in conjunction with setting SQLITE3_LIB_DIR to control which SQLite is used.

See https://github.com/jgallagher/rusqlite for more.

Also add recent contributors to the authors array.
2018-02-13 08:25:58 -08:00
Grisha Kruglov 84f29676e8
"Unchanged server" uploader flow (#543) r=rnewman
* Remove unused struct from tx_processor

* Derive serialize & deserialize for TypedValue

* First pass of uploader flow + feedback
2018-02-09 09:55:19 -08:00
Richard Newman d11810dca7 Fix warning in tx.rs. 2018-02-09 08:51:13 -08:00
Kit Cambridge a6341f6fd6 Implement q_prepare with pre-bound variables. r=rnewman 2018-02-07 21:48:05 -08:00
Emily Toop 715d434945
Create generalized in-memory cache for attributes (#525)
* Nit: Alphabetical ordering of imports

* Create Cache and provide functions for calling it

* Get tests working. Move to using NamespacedKeyword over KnownEntid in function signature

* Add is_cached check to caching tests

* Move lazy and add/remove boolean flags to enums

* Move function definitions into generic trait and implement trait for AttributeCache

* Remove lazy cache and generalize cache

* Update tests

* Eager cache becomes simple key value store. AttributeMap handles attribute storing specifics

* Update tests to test presence of correct values in cache

* Move EagerCache, AttributeValueProvider and ValueProvider into mentat_db

* Add test for get_for_entid

* Add test for lookup attribute

* Make caches cloneable. Add value_for alongside values_for

* Use cache in attribute lookups

* Split test for values and value and add cardinality

* address review feedback r=rnewman
2018-02-07 10:56:12 -08:00
Grisha Kruglov d848d954cf Issue 508 - Iterating transcation processor r=rnewman
Review comments
2018-02-06 12:24:12 -08:00
Richard Newman e9ed103723 Revert "Make debug structs and functions non-public. r=grisha"
This reverts commit e817f67470.
2018-02-06 11:06:06 -08:00
Richard Newman e817f67470 Make debug structs and functions non-public. r=grisha 2018-02-06 10:25:42 -08:00
Richard Newman 66e6fef75e Define Store, use TabWriter in the CLI for aligning columnar output. (#540) r=emily
* Define Store, which is a simple container for a SQLite connection and a Conn.
  This is a breaking change.
* Return the FindSpec as part of QueryOutput, not just results.
* Switch to using stderr in appropriate places in CLI.
* Print columns in CLI output.
2018-02-01 09:29:07 -08:00
Richard Newman 37a7c9ea48 Validate attributes installed after open. (#538) r=emily
Make AttributeBuilder optionally helpful, fix tests.
2018-02-01 09:29:04 -08:00
Richard Newman 2614f498be Ergonomics improvements, including a kw macro. (#537) r=emily
* Add TypedValue::instant(micros).
* Add From<f64> for TypedValue.
* Add lookup_values_for_attribute to Conn.
* Add q_explain to Queryable.
* Expose an iterator over FindSpec's columns.
* Export edn from mentat crate. Export QueryExecutionResult.
* Implement Display for Variable and Element.
* Introduce a `kw` macro.

    This allows you to write:

    ```rust
    kw!(:foo/bar)
    ```

    instead of

    ```rust
    NamespacedKeyword::new("foo", "bar")
    ```

    … and it's more efficient, too.

Add `mentat::open`, eliminate use of `mentat_db` in some places.
2018-02-01 09:27:23 -08:00
Edouard Oger 3bf7459315 Allow customers to assert facts about the current transaction. (#225) r=rnewman
Also move `now` into core, implement microsecond truncation.

This is so we don't return a more granular -- and thus subtly different --
timestamp in a `TxReport` than we put into the store.
2018-01-29 16:46:04 -08:00
Thom Chiovoloni 98502eb68f Implement type annotations in queries. (#526) r=rnewman 2018-01-29 14:37:53 -08:00
Richard Newman ef9f2d9c51 Don't allow violation of cardinality-one restrictions within a single tx. (#531) r=nalexander 2018-01-23 15:11:38 -08:00
Richard Newman 6ed5413cd4 Implement simple vocabulary management. (#504) r=emily,nalexander
Bump version to 0.5.1 to reflect this change.
2018-01-23 08:52:15 -08:00
Richard Newman 812f10b3e4 Add an EntityBuilder abstraction. r=nalexander,emily
This includes two other changes:

* Split transact to expose an interface for TermWithTempIds.
* Return TxReport from each InProgress operation, not from commit.
2018-01-23 08:52:09 -08:00
Richard Newman 3d28949add Describe the default core schema, v1 (:db.schema/core). r=nalexander 2018-01-23 08:51:58 -08:00
Richard Newman 4acc6d0658 InProgressRead, KnownEntid. r=nalexander,emily
Improve naming of read-only transactions.
    Implement entid_for_type.
    Simplify get_attribute.
    Name ignored var in algebrizer.
    Comment attribute_for_ident.
    Make KnownEntid a core concept.
    Expose lookup_value_for_attribute.
    Implement HasSchema and a new query encapsulation on Conn.
    Pre: export Queryable.
2018-01-23 08:40:18 -08:00
Richard Newman 6797a606b5 Preliminary work for vocabulary management. r=emily,nalexander
Pre: export AttributeBuilder from mentat_db.
Pre: fix module-level comment for tx/src/entities.rs.
Pre: rename some `to_` conversions to `into_`.
Pre: make AttributeBuilder::unique less verbose.
Pre: split out a HasSchema trait to abstract over Schema.
Pre: rename SchemaMap/schema_map to AttributeMap/attribute_map.
Pre: TypedValue/NamespacedKeyword conversions.
Pre: turn Unique and ValueType into TypedValue::Keyword.
Pre: export IntoResult.
Pre: export NamespacedKeyword from mentat_core.
Pre: use intern_set in tx.
Pre: add InternSet::len.
Pre: comment gardening.
Pre: remove inaccurate TODO from TxReport comment.
2018-01-23 08:25:32 -08:00
Richard Newman 224570fb45 Switch InProgress to be mutated in place. r=nalexander,emily
This is a breaking change, and involves a very small additional cost
in managing the partition map, but it makes it much more feasible to
implement traits on InProgress: now they don't need to chain back a
new InProgress each time.

Bump version to 0.5 to reflect the change in InProgress.
2018-01-23 08:14:13 -08:00
Fernando Jiménez Moreno 50a9e0c21f Add link to CQRS resources. (#534) r=rnewman
Not everyone knows what CQRS is (I had to Google it) :)
2018-01-23 09:49:49 -06:00
Richard Newman ab5b67ecf7
Tweak README to address some feedback from zbraniecki. 2018-01-22 17:35:36 +00:00
Emily Toop 8cc7f0b72e
Expose Uuid from the top level crate (#529) 2018-01-22 16:29:25 +00:00
Thom 1fcc3d7e1b
Add support for Ctrl-C aborting the current command on the CLI (#528) r=rnewman 2018-01-21 09:45:40 -05:00
Thom 9740cafdbd Automatically remove trailing whitespace from text files. (#527) r=rnewman
This was done using the following shell script:

```
find . -type f -not -path "*target*" \
       '(' -name '*.rs' -o -name '*.md' -o -name '*.toml' ')' -print0 | \
    xargs -0 sed -i '' -E 's/[[:space:]]*$//'
```

Which is admittedly imperfect, but manages to hit everything that was a problem in this repo.
2018-01-19 21:21:04 -06:00