Commit graph

851 commits

Author SHA1 Message Date
Nick Alexander
9513012aa5 [tx] Fail transactions where complex upserts resolve to multiple entids. (#670) r=rnewman 2018-05-01 15:35:44 -07:00
Nick Alexander
2b82ffb2e5 [tx] Fail transactions where complex upserts resolve to multiple entids. (#670)
This innocuous looking change (upserts_ev -> upserts_e -> resolved in
all situations, rather than upserts_ev -> resolved in some situations)
is a significant change in semantics and assumptions in the
transactor.  Witness the large comment being removed about the same
tempid resolving in different generations!

To support this change, we provide more holistic errors for
conflicting upserts, which entails collecting some (relatively
expensive) diagnostic data.

I left in some debug logging, simply since it shouldn't hurt in
general, and will likely be useful for the next bug we see in the
transactor.
2018-05-01 15:34:44 -07:00
Nick Alexander
7960b4ccd2 Pre: Get ready to use log in mentat_db.
We don't yet have a logging system for production use, but I'd like to
start experimenting with log, which seems to be (close to) a Rust
standard.  We're already using it in mentat_cli.
2018-05-01 13:46:03 -07:00
Nick Alexander
d4a635f4e7 (tx) Replace :db/tx with (current-tx) transaction function and broaden support. (#664) r=rnewman 2018-04-26 19:33:16 -07:00
Nick Alexander
32ed56685e (tx) Replace :db/tx with (transaction-tx) transaction function and broaden support. (#664)
:db/tx (and Datomic's version, :datomic/tx) suffer from the same
ambiguities that [a v] lookup references do -- determining the type of
the result is context sensitive.  (In this case, is :db/tx a reference
to the current transaction ID, or is it a valid keyword?)  This commit
addresses the ambiguity by introducing a notion of a transaction
functions, and provides a little scaffolding for adding more (should
the need arise).  I left the scaffolding in place rather than handling
just (transaction-tx) because I started trying to
implement (transaction-instant) as well, which is more difficult --
see the comments.

It's worth noting that this approach generalizes more or less directly
to ?input variables, since those can be eagerly bound like the
implemented transaction function (transaction-tx).
2018-04-26 19:32:14 -07:00
Richard Newman
f979044ba1
Refactor value type boxing. (#659) r=nalexander
* Pre: eliminate some occurrences of Rc, largely through the magic of Into.
* Pre: introduce FromRc to convert between refcounted types.
* Introduce ValueRc as an abstraction over Rc/Arc choice.
* Move Cloned to core.
* Move CString-creation methods to TypedValue.
* Finish transition.
2018-04-25 14:23:27 -07:00
Richard Newman
a2e13f624c
Add 'Binding', a structured value type to return from queries. (#657) r=nalexander
Bump to 0.7: breaking change.
2018-04-24 15:08:38 -07:00
Richard Newman
1818e0b98e Split mentat_core TypedValue code into separate files for clarity. 2018-04-24 15:05:04 -07:00
Richard Newman
a74a2deffc
Introduce RelResult rather than Vec<Vec<TypedValue>>. (#639) r=nalexander
* Pre: clean up core/src/lib.rs.
* Pre: use indexmap 1.0 in db and query-projector.
* Change rel results to be a RelResult instance, not a Vec<Vec<TypedValue>>.

This avoids memory fragmentation and improves locality by using a single
heap-allocated vector for all bindings, rather than a separate
heap-allocated vector for each row.

We hide this abstraction behind the `RelResult` type, which tracks the
stride length (width) of each row.

* Don't allocate temporary vectors when projecting RelResults.
2018-04-24 15:04:00 -07:00
Nick Alexander
0c31fc7875 (query) Implement tx-log API: (tx-ids ...) and (tx-data ...) functions. r=rnewman 2018-04-19 09:59:05 -07:00
Nick Alexander
c8da4be38f (query) Implement tx-log API: (tx-ids ...) and (tx-data ...) functions.
`tx-ids` allows to enumerate transaction IDs efficiently.

`tx-data` allows to extract transaction log data efficiently.

We might eventually allow to filter by impacted attribute sets as well.
2018-04-19 09:58:41 -07:00
Nick Alexander
e532614908 (query) Pre: Model columns that don't have type tags closer to Column. 2018-04-19 09:58:41 -07:00
Nick Alexander
36eca0bfb0 (chore) Pre: Use the same features of uuid throughout the project. 2018-04-19 09:58:41 -07:00
Richard Newman
1f1818448a
Begin adding worked examples. (#629) r=nalexander 2018-04-17 10:39:36 -07:00
Richard Newman
8ca657ec03 Simplify vocabulary migration test.
We don't need to explicitly retract if the transactor will do it
for us -- which it will if an attribute is cardinality-one.
2018-04-12 11:53:19 -07:00
Richard Newman
1509d16c3e
Fix (the ?foo) (#633) r=nalexander
Don't group by ?var when processing (the ?var).

This PR also finishes error generation in the projector.
2018-04-10 11:58:58 -07:00
Richard Newman
909b2a8be5 Refactoring: split up the projector crate. No other code changes. 2018-04-09 10:26:09 -07:00
Richard Newman
39f1d61175 Bump version to 0.6.2. 2018-04-09 09:47:49 -07:00
Richard Newman
3f8464e8ed Implement vocabulary-driven schema upgrades. (#595) r=emily 2018-04-09 09:47:49 -07:00
Richard Newman
a5cda7c3e9 Allow passing a TermBuilder to be transacted by InProgress; add TermBuilder::is_empty. r=emily 2018-04-09 09:47:49 -07:00
Richard Newman
27dde378e0 Allow retraction of some schema attributes. (#379) r=nalexander 2018-04-09 09:47:49 -07:00
Emily Toop
175958e754 Address review comments @rnewman 2018-04-06 10:46:15 +01:00
Emily Toop
19ddf9c384 Spacing 2018-04-06 10:46:15 +01:00
Emily Toop
fa7dd2ceab Add FFI for query building 2018-04-06 10:46:15 +01:00
Emily Toop
9741435026 Add helper functions for FFI. Many of these will go away as we expose the entity builder 2018-04-06 10:46:15 +01:00
Emily Toop
7382e3297d Add QueryBuilder to make querying over FFI easier 2018-04-06 10:46:15 +01:00
Emily Toop
b3e27d86a9 Add converter functions from TypedValue to underlying type 2018-04-06 10:46:15 +01:00
Richard Newman
a57f7aff99
Add specialized tx-before and tx-after predicates. (#599) r=emily 2018-04-05 10:49:06 -07:00
Richard Newman
8607ecb745 Fix merge conflict. 2018-04-03 14:54:46 -07:00
Richard Newman
4d8e179a59
Expose component_attributes on Schema. (#623) r=nalexander
Some parts of the query engine and transactor need to know whether an
attribute is a component attribute, and sometimes want to do so in
a generated SQL query. This is one way to do that.
2018-04-03 14:25:53 -07:00
Richard Newman
558906df4f
Fix: db/component should be db/isComponent. (#624) r=nalexander 2018-04-03 14:25:28 -07:00
Richard Newman
6c54e1d370
Support :db/noHistory for attributes. (#622) r=nalexander
At this point we never discard history, but this completes the API support for doing so.
2018-04-03 14:23:46 -07:00
Richard Newman
9cc5cbf288
Rename the helpful variant, AttributeBuilder::new, to AttributeBuilder::helpful. (#625) r=nalexander 2018-04-03 14:23:20 -07:00
Richard Newman
ca451a7c9c Silence a warning in Tolstoy. 2018-04-03 14:01:29 -07:00
Richard Newman
66b892572c
Don't create a CommandExecutor if there are no observers. (#603) (#604) r=emily
* Don't create a CommandExecutor if there are no observers. (#603)
* Don't log if our executor channel goes away. This is routine.
2018-04-03 09:18:22 -07:00
Emily Toop
9f30fe6295 Create Mentat FFI and expose observers (#574)
* Tidy up and add txid at beginning of transaction

* Add ffi crate and new_store function

* Add register and unregister observer FFI, Store and Conn functions.
Also add android logging facilities

* Add function for fetching entids for attribute strings

* Add functions for iterating through TxReports

* Add sync to ffi boundary

* Move Extern types from submodule to lib in FFI.
For some reason, if these types are in a submodule, even if they are publically used, the functions inside the FFI are not found in
Android. Works for iOS though. To be investigated later....

* Return to passing TxReports to observer function.
Also, remove some debug

* Expose DateTime and Utc publically

* Use Store in observer tests
2018-03-20 19:16:32 +00:00
Emily Toop
ab957948b4 Move to using watcher.
Simplify.

This has a watcher collect txid -> AttributeSet mappings each time a
transact occurs. On commit we retrieve those mappings and hand them over
to the observer service, which filters them and packages them up for
dispatch.

Tidy up
2018-03-20 16:27:35 +00:00
Emily Toop
d4365fa4cd Execute commands in a separate thread
Command Queue Executor to watch for new commands and execute on longer running background thread
2018-03-20 16:27:35 +00:00
Emily Toop
ecc4a7a35a Add tests 2018-03-20 16:27:35 +00:00
Emily Toop
c2e5052877 Allow registration and unregistration of transaction observers from Conn 2018-03-20 16:27:35 +00:00
Emily Toop
9f3d2c08b2 Populate changeset of attributes inside TxReport during transact.
Batch up TxReports for entire transaction.
Notify observers about committed transaction.
Store transaction observer service inside Conn
2018-03-20 16:27:35 +00:00
Emily Toop
8d60f2b3d1 Expose tx observation publicly 2018-03-20 16:27:35 +00:00
Emily Toop
dfa0d3e321 Add TxObservers, Commands and TxObservationService.
These are the base types upon which we will build our transaction observation
2018-03-20 16:27:35 +00:00
Richard Newman
f71b2b207e Expand the README to give a guide to the crates in the repo. 2018-03-19 14:35:18 -07:00
Richard Newman
16a66517e4
Err, don't panic, on unbound variable (#518) (#590) r=emily
* Pre: switch a 'panic' to an 'unreachable'.
* Make candidate_type_column fallible. (#518)
2018-03-15 11:39:24 -07:00
Richard Newman
994a3e65e2
Tests and fixes for aggregates over different or unknown types. (#588) r=emily 2018-03-15 07:14:06 -07:00
Richard Newman
df58de52f4
Correctly parse and unescape quotes etc. inside EDN strings. (#434) (#589) 2018-03-15 07:13:27 -07:00
Richard Newman
ea52e214af
Small README tweaks. 2018-03-13 04:01:19 +00:00
Richard Newman
833ff92436
Simple aggregates. (#584) r=emily
* Pre: use debugcli in VSCode.
* Pre: wrap subqueries in parentheses in output SQL.
* Pre: add ExistingColumn.

This lets us make reference to columns by name, rather than only
pointing to qualified aliases.

* Pre: add Into for &str to TypedValue.
* Pre: add Store.transact.
* Pre: cleanup.
* Parse and algebrize simple aggregates. (#312)
* Follow-up: print aggregate columns more neatly in the CLI.
* Useful ValueTypeSet helpers.
* Allow for entity inequalities.
* Add 'differ', which is a ref-specialized not-equals.
* Add 'unpermute', a function for getting unique, distinct pairs from bindings.
* Review comments.
* Add 'the' pseudo-aggregation operator.

This allows for a corresponding value to be returned when a query
includes one 'min' or 'max' aggregate.
2018-03-12 15:18:50 -07:00
Richard Newman
46835885e4 Add help for timer command. 2018-03-06 10:49:41 -08:00