* Pre: remove remnants of 'open_empty'
* Pre: Cleanup 'datoms' table after a timeline move
Since timeline move operations use a transactor, they generate a
"phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense
that it was never present in the 'transactions' table, and is entirely
synthetic as far as our database is concerned.
It's an implementational artifact, and we were not cleaning it up.
It becomes a problem when we start inserting transactions after a move.
Once the transactor clashes with the phantom 'tx', it will retract the
phantom 'txInstant' value, leaving the transactions log in an incorrect state.
This patch adds a test for this scenario and elects the easy way out: simply
remove the offending 'txInstant' datom.
* Part 1: Sync without support for side-effects
A "side-effect" is defined here as a mutation of a remote state as part
of the sync.
If, during a sync we determine that a remote state needs to be changed, bail out.
This generally supports different variations of "baton-passing" syncing, where clients
will succeed syncing if each change is non-conflicting.
* Part 2: Support basic "side-effects" syncing
This patch introduces a concept of a follow-up sync. If a sync generated
a "merge transaction" (a regular transaction that contains assertions
necessary for local and remote transaction logs to converge), then
this transaction needs to be uploaded in a follow-up sync.
Generated SyncReport indicates if a follow-up sync is required.
Follow-up sync itself is just a regular sync. If remote state did not change,
it will result in a simple RemoteFastForward. Otherwise, we'll continue
merging and requesting a follow-up.
Schema alterations are explicitly not supported.
As local transactions are rebased on top of remote, following changes happen:
- entids are changed into tempids, letting transactor upsert :db/unique values
- entids for retractions are changed into lookup-refs if we're confident they'll succeed
-- otherwise, retractions are dropped on the floor
* Post: use a macro for more readable tests
* Tolstoy README
Sync needs to operate over a "mentat transaction", not just a "db transaction".
This shuffle allows internal mentat crates to consume InProgress, which models
the concept of a "mentat transaction".
Being able to derive partition map from partition definitions and current
state of the world (transactions), segmented by timelines, is useful
because it lets us not worry about keeping materialized partition maps
up-to-date - since there's no need for materialized partition maps at that point.
This comes in very handy when we start moving chunks of transactions off of our mainline.
Alternative to this work would look like materializing partition maps per timeline,
growing support for incremental "backwards update" of the materialized maps, etc.
Our core partitions are defined in 'known_parts' table during bootstrap,
and what used to be 'parts' table is a generated view that operates over
transactions to figure out partition index.
'parts' is defined for the main timeline. Querying parts for other timelines
or for particular timeline+tx combinations will look similar.
Normally we want to both materialize our changes (into 'datoms')
as well as commit source transactions into 'transactions' table.
However, when moving transactions from timeline to timeline
we don't want to persist artifacts (rewind assertions), just their
materializations.
This patch expands the 'db' interface to allow for this split,
and changes transactor's functions to take a crate-private 'action'
which defines desired behaviour.
This is necessary for the timelines work ahead. When schema is being
moved off of a main timeline, we need to be able to retract it cleanly.
Retractions are only processed if the whole defining attribute set
is being retracted at once (:db/ident, :db/valueType, :db/cardinality).
