* Pre: remove remnants of 'open_empty'
* Pre: Cleanup 'datoms' table after a timeline move
Since timeline move operations use a transactor, they generate a
"phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense
that it was never present in the 'transactions' table, and is entirely
synthetic as far as our database is concerned.
It's an implementational artifact, and we were not cleaning it up.
It becomes a problem when we start inserting transactions after a move.
Once the transactor clashes with the phantom 'tx', it will retract the
phantom 'txInstant' value, leaving the transactions log in an incorrect state.
This patch adds a test for this scenario and elects the easy way out: simply
remove the offending 'txInstant' datom.
* Part 1: Sync without support for side-effects
A "side-effect" is defined here as a mutation of a remote state as part
of the sync.
If, during a sync we determine that a remote state needs to be changed, bail out.
This generally supports different variations of "baton-passing" syncing, where clients
will succeed syncing if each change is non-conflicting.
* Part 2: Support basic "side-effects" syncing
This patch introduces a concept of a follow-up sync. If a sync generated
a "merge transaction" (a regular transaction that contains assertions
necessary for local and remote transaction logs to converge), then
this transaction needs to be uploaded in a follow-up sync.
Generated SyncReport indicates if a follow-up sync is required.
Follow-up sync itself is just a regular sync. If remote state did not change,
it will result in a simple RemoteFastForward. Otherwise, we'll continue
merging and requesting a follow-up.
Schema alterations are explicitly not supported.
As local transactions are rebased on top of remote, following changes happen:
- entids are changed into tempids, letting transactor upsert :db/unique values
- entids for retractions are changed into lookup-refs if we're confident they'll succeed
-- otherwise, retractions are dropped on the floor
* Post: use a macro for more readable tests
* Tolstoy README
Sync needs to operate over a "mentat transaction", not just a "db transaction".
This shuffle allows internal mentat crates to consume InProgress, which models
the concept of a "mentat transaction".
* Add a top-level "syncable" feature.
Tested with:
cargo test --all
cargo test --all --no-default-features
cargo build --manifest-path tools/cli/Cargo.toml --no-default-features
cargo run --manifest-path tools/cli/Cargo.toml --no-default-features debugcli
Co-authored-by: Nick Alexander <nalexander@mozilla.com>
* Add 'syncable' feature to 'db' crate to conditionally derive serialization for Partition*
This is leading up to syncing with partition support.
With the transition toward parsing with `rust-peg` and away from
`combine`, we're not using some of the many helpers we built to
support our unusual `combine` usage. They can just go!
* Refactor AttributeCache populator code for use from pull.
* Pre: add to_value_rc to Cloned.
* Pre: add From<StructuredMap> for Binding.
* Pre: clarify Store::open_empty.
* Pre: StructuredMap cleanup.
* Pre: clean up a doc test.
* Split projector crate. Pass schema to projector.
* CLI support for printing bindings.
* Add and use ConjoiningClauses::derive_types_from_find_spec.
* Define pull types.
* Implement pull on top of the attribute cache layer.
* Add pull support to the projector.
* Parse pull expressions.
* Add simple pull support to connection objects.
* Tests for pull.
* Compile with Rust 1.25.
The only choice involved in this commit is that of replacing the
anonymous lifetime '_ with a named lifetime for the cache; since we're
accepting a Known, which includes the cache in question, I think it's
clear that we expect the function to apply to any given cache
lifetime.
* Review comments.
* Bail on unnamed attribute.
* Make assert_parse_failure_contains safe to use.
* Rework query parser to report better errors for pull.
* Test for mixed wildcard and simple attribute.
* Tidy up and add txid at beginning of transaction
* Add ffi crate and new_store function
* Add register and unregister observer FFI, Store and Conn functions.
Also add android logging facilities
* Add function for fetching entids for attribute strings
* Add functions for iterating through TxReports
* Add sync to ffi boundary
* Move Extern types from submodule to lib in FFI.
For some reason, if these types are in a submodule, even if they are publically used, the functions inside the FFI are not found in
Android. Works for iOS though. To be investigated later....
* Return to passing TxReports to observer function.
Also, remove some debug
* Expose DateTime and Utc publically
* Use Store in observer tests
Simplify.
This has a watcher collect txid -> AttributeSet mappings each time a
transact occurs. On commit we retrieve those mappings and hand them over
to the observer service, which filters them and packages them up for
dispatch.
Tidy up
* Add a prepared query command to CLI.
* Print nanoseconds in the REPL. This is a good problem to have.
* Better CLI timing.
* Use release for 'cargo cli', debug for 'cargo debugcli'.
* Don't enable debug symbols in release builds.
* Clean up CLI code. Fixed order for help.
* Column-align help output.
You can use this in conjunction with setting SQLITE3_LIB_DIR to control which SQLite is used.
See https://github.com/jgallagher/rusqlite for more.
Also add recent contributors to the authors array.
* Define Store, which is a simple container for a SQLite connection and a Conn.
This is a breaking change.
* Return the FindSpec as part of QueryOutput, not just results.
* Switch to using stderr in appropriate places in CLI.
* Print columns in CLI output.
This includes two other changes:
* Split transact to expose an interface for TermWithTempIds.
* Return TxReport from each InProgress operation, not from commit.
This is a breaking change, and involves a very small additional cost
in managing the partition map, but it makes it much more feasible to
implement traits on InProgress: now they don't need to chain back a
new InProgress each time.
Bump version to 0.5 to reflect the change in InProgress.
* Pre: rename begin_transaction to begin_tx_application.
* Take an EXCLUSIVE transaction when bootstrapping, and an IMMEDIATE transaction when writing.
This avoids the remote possibility of another write sneaking in the door
while we're preparing to write, avoids us needing to upgrade locks, etc.
After a BEGIN IMMEDIATE, no other database connection will be able to write
to the database or do a BEGIN IMMEDIATE or BEGIN EXCLUSIVE. Other processes
can continue to read from the database, however.
An exclusive transaction causes EXCLUSIVE locks to be acquired on all
databases. After a BEGIN EXCLUSIVE, no other database connection except for
read_uncommitted connections will be able to read the database and no other
connection without exception will be able to write the database until the
transaction is complete.
* Hacky implementation of atomic multi-tx.
* Hold the last report, returning the InProgress from each operation.
* Rewrite transact in terms of InProgress.
* Test rollback.
* Remove unused imports.
* Don't use Rc for transaction reports.
* Pre: break out USER0 as a part boundary constant.
* Export TX0 and USER0 from mentat_db. This is for testing.
* Review comments: commenting.
* Test tempid allocation and rollback.
* Create mentat command line.
* Create tools directory containing new crate for mentat_cli.
* Add simple cli with mentat prompt.
* Remove rustc-serialize dependency
* Open DB inside CLI (#452) (#463)
* Open named database OR default to in memory database if no name provided
Rearrange workspace to allow import of mentat crate in cli crate
Create store object inside repl when started for connecting to mentat
Use provided DB name to open connection in store
Accept DB name as command line arg.
Open on CLI start
Implement '.open' command to open desired DB from inside CLI
* Implement Close command to close current DB.
* Closes existing open db and opens new in memory db
* Review comment: Use `combine` to parse arguments.
Move over to using Result rather than enums with err
* Accept and parse EDN Query and Transact commands (#453) (#465)
* Parse query and transact commands
* Implement is_complete for transactions and queries
* Improve query parser. Am still not happy with it though.
There must be some way that I can retain the eof() after the `then` that means I don't have to move the skip on spaces and eof
Make in process command storing clearer.
Add comments around in process commands.
Add alternative commands for transact/t and query/q
* Address review comments r=nalexander.
* Bump rust version number.
* Use `bail` when throwing errors.
* Improve edn parser.
* Remove references to unused `more` flag.
* Improve naming of query and transact commands.
* Send queries and transactions to mentat and output the results (#466)
* Send queries and transactions to mentat and output the results
move outputting query and transaction results out of store and into repl
* Add query and transact commands to help
* Execute queries and transacts passed in at startup
* Address review comments =nalexander.
* Bump rust version number.
* Use `bail` when throwing errors.
* Improve edn parser.
* Remove references to unused `more` flag.
* Improve naming of query and transact commands.
* Execute command line args in order
* Addressing rebase issues
* Exit CLI (#457) (#484) r-rnewman
* Implement exit command for cli tool
* Address review comments r=rnewman
* Include exit commands in help
* Show schema of current DB (#487)
* Fixing rebase issues
* addressing nit
* Match updated dependencies on CLI crate and remove unused import
* Update some dependencies.
* Update rusqlite to 0.12.
* Update error-chain to a forked version that implements Sync.
* Fix some compiler warnings.
* Remove unused imports in tests.
* Parse errors no longer naturally print with the expected symbol.
