Commit graph

693 commits

Author SHA1 Message Date
Richard Newman
a74a2deffc
Introduce RelResult rather than Vec<Vec<TypedValue>>. (#639) r=nalexander
* Pre: clean up core/src/lib.rs.
* Pre: use indexmap 1.0 in db and query-projector.
* Change rel results to be a RelResult instance, not a Vec<Vec<TypedValue>>.

This avoids memory fragmentation and improves locality by using a single
heap-allocated vector for all bindings, rather than a separate
heap-allocated vector for each row.

We hide this abstraction behind the `RelResult` type, which tracks the
stride length (width) of each row.

* Don't allocate temporary vectors when projecting RelResults.
2018-04-24 15:04:00 -07:00
Nick Alexander
0c31fc7875 (query) Implement tx-log API: (tx-ids ...) and (tx-data ...) functions. r=rnewman 2018-04-19 09:59:05 -07:00
Nick Alexander
c8da4be38f (query) Implement tx-log API: (tx-ids ...) and (tx-data ...) functions.
`tx-ids` allows to enumerate transaction IDs efficiently.

`tx-data` allows to extract transaction log data efficiently.

We might eventually allow to filter by impacted attribute sets as well.
2018-04-19 09:58:41 -07:00
Nick Alexander
e532614908 (query) Pre: Model columns that don't have type tags closer to Column. 2018-04-19 09:58:41 -07:00
Nick Alexander
36eca0bfb0 (chore) Pre: Use the same features of uuid throughout the project. 2018-04-19 09:58:41 -07:00
Richard Newman
1f1818448a
Begin adding worked examples. (#629) r=nalexander 2018-04-17 10:39:36 -07:00
Richard Newman
8ca657ec03 Simplify vocabulary migration test.
We don't need to explicitly retract if the transactor will do it
for us -- which it will if an attribute is cardinality-one.
2018-04-12 11:53:19 -07:00
Richard Newman
1509d16c3e
Fix (the ?foo) (#633) r=nalexander
Don't group by ?var when processing (the ?var).

This PR also finishes error generation in the projector.
2018-04-10 11:58:58 -07:00
Richard Newman
909b2a8be5 Refactoring: split up the projector crate. No other code changes. 2018-04-09 10:26:09 -07:00
Richard Newman
39f1d61175 Bump version to 0.6.2. 2018-04-09 09:47:49 -07:00
Richard Newman
3f8464e8ed Implement vocabulary-driven schema upgrades. (#595) r=emily 2018-04-09 09:47:49 -07:00
Richard Newman
a5cda7c3e9 Allow passing a TermBuilder to be transacted by InProgress; add TermBuilder::is_empty. r=emily 2018-04-09 09:47:49 -07:00
Richard Newman
27dde378e0 Allow retraction of some schema attributes. (#379) r=nalexander 2018-04-09 09:47:49 -07:00
Emily Toop
175958e754 Address review comments @rnewman 2018-04-06 10:46:15 +01:00
Emily Toop
19ddf9c384 Spacing 2018-04-06 10:46:15 +01:00
Emily Toop
fa7dd2ceab Add FFI for query building 2018-04-06 10:46:15 +01:00
Emily Toop
9741435026 Add helper functions for FFI. Many of these will go away as we expose the entity builder 2018-04-06 10:46:15 +01:00
Emily Toop
7382e3297d Add QueryBuilder to make querying over FFI easier 2018-04-06 10:46:15 +01:00
Emily Toop
b3e27d86a9 Add converter functions from TypedValue to underlying type 2018-04-06 10:46:15 +01:00
Richard Newman
a57f7aff99
Add specialized tx-before and tx-after predicates. (#599) r=emily 2018-04-05 10:49:06 -07:00
Richard Newman
8607ecb745 Fix merge conflict. 2018-04-03 14:54:46 -07:00
Richard Newman
4d8e179a59
Expose component_attributes on Schema. (#623) r=nalexander
Some parts of the query engine and transactor need to know whether an
attribute is a component attribute, and sometimes want to do so in
a generated SQL query. This is one way to do that.
2018-04-03 14:25:53 -07:00
Richard Newman
558906df4f
Fix: db/component should be db/isComponent. (#624) r=nalexander 2018-04-03 14:25:28 -07:00
Richard Newman
6c54e1d370
Support :db/noHistory for attributes. (#622) r=nalexander
At this point we never discard history, but this completes the API support for doing so.
2018-04-03 14:23:46 -07:00
Richard Newman
9cc5cbf288
Rename the helpful variant, AttributeBuilder::new, to AttributeBuilder::helpful. (#625) r=nalexander 2018-04-03 14:23:20 -07:00
Richard Newman
ca451a7c9c Silence a warning in Tolstoy. 2018-04-03 14:01:29 -07:00
Richard Newman
66b892572c
Don't create a CommandExecutor if there are no observers. (#603) (#604) r=emily
* Don't create a CommandExecutor if there are no observers. (#603)
* Don't log if our executor channel goes away. This is routine.
2018-04-03 09:18:22 -07:00
Emily Toop
9f30fe6295 Create Mentat FFI and expose observers (#574)
* Tidy up and add txid at beginning of transaction

* Add ffi crate and new_store function

* Add register and unregister observer FFI, Store and Conn functions.
Also add android logging facilities

* Add function for fetching entids for attribute strings

* Add functions for iterating through TxReports

* Add sync to ffi boundary

* Move Extern types from submodule to lib in FFI.
For some reason, if these types are in a submodule, even if they are publically used, the functions inside the FFI are not found in
Android. Works for iOS though. To be investigated later....

* Return to passing TxReports to observer function.
Also, remove some debug

* Expose DateTime and Utc publically

* Use Store in observer tests
2018-03-20 19:16:32 +00:00
Emily Toop
ab957948b4 Move to using watcher.
Simplify.

This has a watcher collect txid -> AttributeSet mappings each time a
transact occurs. On commit we retrieve those mappings and hand them over
to the observer service, which filters them and packages them up for
dispatch.

Tidy up
2018-03-20 16:27:35 +00:00
Emily Toop
d4365fa4cd Execute commands in a separate thread
Command Queue Executor to watch for new commands and execute on longer running background thread
2018-03-20 16:27:35 +00:00
Emily Toop
ecc4a7a35a Add tests 2018-03-20 16:27:35 +00:00
Emily Toop
c2e5052877 Allow registration and unregistration of transaction observers from Conn 2018-03-20 16:27:35 +00:00
Emily Toop
9f3d2c08b2 Populate changeset of attributes inside TxReport during transact.
Batch up TxReports for entire transaction.
Notify observers about committed transaction.
Store transaction observer service inside Conn
2018-03-20 16:27:35 +00:00
Emily Toop
8d60f2b3d1 Expose tx observation publicly 2018-03-20 16:27:35 +00:00
Emily Toop
dfa0d3e321 Add TxObservers, Commands and TxObservationService.
These are the base types upon which we will build our transaction observation
2018-03-20 16:27:35 +00:00
Richard Newman
f71b2b207e Expand the README to give a guide to the crates in the repo. 2018-03-19 14:35:18 -07:00
Richard Newman
16a66517e4
Err, don't panic, on unbound variable (#518) (#590) r=emily
* Pre: switch a 'panic' to an 'unreachable'.
* Make candidate_type_column fallible. (#518)
2018-03-15 11:39:24 -07:00
Richard Newman
994a3e65e2
Tests and fixes for aggregates over different or unknown types. (#588) r=emily 2018-03-15 07:14:06 -07:00
Richard Newman
df58de52f4
Correctly parse and unescape quotes etc. inside EDN strings. (#434) (#589) 2018-03-15 07:13:27 -07:00
Richard Newman
ea52e214af
Small README tweaks. 2018-03-13 04:01:19 +00:00
Richard Newman
833ff92436
Simple aggregates. (#584) r=emily
* Pre: use debugcli in VSCode.
* Pre: wrap subqueries in parentheses in output SQL.
* Pre: add ExistingColumn.

This lets us make reference to columns by name, rather than only
pointing to qualified aliases.

* Pre: add Into for &str to TypedValue.
* Pre: add Store.transact.
* Pre: cleanup.
* Parse and algebrize simple aggregates. (#312)
* Follow-up: print aggregate columns more neatly in the CLI.
* Useful ValueTypeSet helpers.
* Allow for entity inequalities.
* Add 'differ', which is a ref-specialized not-equals.
* Add 'unpermute', a function for getting unique, distinct pairs from bindings.
* Review comments.
* Add 'the' pseudo-aggregation operator.

This allows for a corresponding value to be returned when a query
includes one 'min' or 'max' aggregate.
2018-03-12 15:18:50 -07:00
Richard Newman
46835885e4 Add help for timer command. 2018-03-06 10:49:41 -08:00
Richard Newman
3e615becd8 Fix printing of fractional millisecond timestamps. (#582) r=emily 2018-03-06 10:49:41 -08:00
Richard Newman
1817ce7c0b Performance and cleanup. r=emily
* Use fixed-size arrays for bootstrap datoms, not vecs.
* Wide-ranging cleanup.

    This commit:
    - Deletes some dead code.
    - Marks some functions only used by tests as cfg(test).
    - Adds pub(crate) to a bunch of functions.
    - Cleans up a few other nits.
2018-03-06 09:03:00 -08:00
Richard Newman
a0c70a7cd9 Add caching example to Seattle fixture. 2018-03-06 09:02:11 -08:00
Richard Newman
f42ae35b70 Update cache on write. (#566) r=emily
* Use the cache to make constant queries super fast.
* Fix translate tests to match: we no longer generate SQL for many of them!
* Accumulate additions and removals into the cache.
    * Make attribute cache clone-on-write; store it in Metadata.
    * Allow caching of fulltext attributes, interning strings.
2018-03-06 09:01:20 -08:00
Richard Newman
bead9752bd Add InProgress.import to load transaction data from a file. r=emily
It's not streaming, but it'll do.
2018-03-06 08:59:30 -08:00
Richard Newman
0ca61017a1 Use LTO for release builds. 2018-03-06 08:19:17 -08:00
Richard Newman
4a0b67ab50 Require Rust 0.1.24 or higher. 2018-03-06 08:19:17 -08:00
Richard Newman
9b23cf3945
Speed up EDN parser (fixes #445) (#581) r=nalexander
Fixes from @kevinmehall.

* Prefer character sets over backtracking in the EDN parser.
* Avoid duplicate effort when parsing floats in the EDN parser.
* Clean up duplicate position tracking code.

This turns out to have little performance impact, but makes the grammar
much cleaner.

* Fix EDN work to pass tests with correct numeric precedence.
2018-03-05 20:33:51 -08:00