mentat/tolstoy/src/types.rs

90 lines
2.7 KiB
Rust
Raw Normal View History

Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
// Copyright 2018 Mozilla
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not use
// this file except in compliance with the License. You may obtain a copy of the
// License at http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software distributed
// under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.
use std::cmp::Ordering;
use uuid::Uuid;
use core_traits::{Entid, TypedValue};
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
use mentat_db::PartitionMap;
use public_traits::errors::Result;
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
pub struct LocalGlobalTxMapping<'a> {
pub local: Entid,
pub remote: &'a Uuid,
}
impl<'a> From<(Entid, &'a Uuid)> for LocalGlobalTxMapping<'a> {
2020-08-06 03:03:58 +00:00
fn from((local, remote): (Entid, &'a Uuid)) -> LocalGlobalTxMapping<'_> {
LocalGlobalTxMapping { local, remote }
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
}
}
impl<'a> LocalGlobalTxMapping<'a> {
pub fn new(local: Entid, remote: &'a Uuid) -> LocalGlobalTxMapping<'a> {
2020-08-06 03:03:58 +00:00
LocalGlobalTxMapping { local, remote }
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
}
}
// TODO unite these around something like `enum TxIdentifier {Global(Uuid), Local(Entid)}`?
#[derive(Debug, Clone)]
pub struct LocalTx {
pub tx: Entid,
pub parts: Vec<TxPart>,
}
impl PartialOrd for LocalTx {
fn partial_cmp(&self, other: &LocalTx) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Ord for LocalTx {
fn cmp(&self, other: &LocalTx) -> Ordering {
self.tx.cmp(&other.tx)
}
}
impl PartialEq for LocalTx {
fn eq(&self, other: &LocalTx) -> bool {
self.tx == other.tx
}
}
impl Eq for LocalTx {}
// For returning out of the downloader as an ordered list.
#[derive(Debug, Clone, PartialEq)]
pub struct Tx {
pub tx: Uuid,
pub parts: Vec<TxPart>,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
pub struct TxPart {
// TODO this is a temporary for development. Only first TxPart in a chunk series should have a non-None 'parts'.
// 'parts' should actually live in a transaction, but we do this now to avoid changing the server until dust settles.
pub partitions: Option<PartitionMap>,
pub e: Entid,
pub a: Entid,
pub v: TypedValue,
pub tx: Entid,
pub added: bool,
}
pub trait GlobalTransactionLog {
fn head(&self) -> Result<Uuid>;
fn transactions_after(&self, tx: &Uuid) -> Result<Vec<Tx>>;
fn set_head(&mut self, tx: &Uuid) -> Result<()>;
2020-08-06 03:03:58 +00:00
fn put_transaction(&mut self, tx: &Uuid, parent_tx: &Uuid, chunk_txs: &[Uuid]) -> Result<()>;
Basic sync support (#563) r=nalexander * Pre: remove remnants of 'open_empty' * Pre: Cleanup 'datoms' table after a timeline move Since timeline move operations use a transactor, they generate a "phantom" 'tx' and a 'txInstant' assertion. It is "phantom" in a sense that it was never present in the 'transactions' table, and is entirely synthetic as far as our database is concerned. It's an implementational artifact, and we were not cleaning it up. It becomes a problem when we start inserting transactions after a move. Once the transactor clashes with the phantom 'tx', it will retract the phantom 'txInstant' value, leaving the transactions log in an incorrect state. This patch adds a test for this scenario and elects the easy way out: simply remove the offending 'txInstant' datom. * Part 1: Sync without support for side-effects A "side-effect" is defined here as a mutation of a remote state as part of the sync. If, during a sync we determine that a remote state needs to be changed, bail out. This generally supports different variations of "baton-passing" syncing, where clients will succeed syncing if each change is non-conflicting. * Part 2: Support basic "side-effects" syncing This patch introduces a concept of a follow-up sync. If a sync generated a "merge transaction" (a regular transaction that contains assertions necessary for local and remote transaction logs to converge), then this transaction needs to be uploaded in a follow-up sync. Generated SyncReport indicates if a follow-up sync is required. Follow-up sync itself is just a regular sync. If remote state did not change, it will result in a simple RemoteFastForward. Otherwise, we'll continue merging and requesting a follow-up. Schema alterations are explicitly not supported. As local transactions are rebased on top of remote, following changes happen: - entids are changed into tempids, letting transactor upsert :db/unique values - entids for retractions are changed into lookup-refs if we're confident they'll succeed -- otherwise, retractions are dropped on the floor * Post: use a macro for more readable tests * Tolstoy README
2018-09-08 02:18:20 +00:00
fn put_chunk(&mut self, tx: &Uuid, payload: &TxPart) -> Result<()>;
}