greg/mentat
1
0
Fork 0
Code Issues 294 Pull requests Projects Releases 9 Packages Wiki Activity

Attempting to cleanup with clippy, rustfmt, etc.

Browse source 
Integrate https://github.com/mozilla/mentat/pull/806
This commit is contained in:
Greg Burd 2020-01-23 13:16:19 -05:00 committed by Greg Burd
parent 8aec4048b9
commit 4f81c4e15b
52 changed files with 763 additions and 621 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -92,3 +92,4 @@ build.xcarchive
docs/_site
docs/.sass-cache
docs/.jekyll-metadata

19
.travis.yml
View file

@ -1,27 +1,38 @@
language: rust
cache: cargo # cache cargo-audit once installed
before_script:
- cargo install --git https://github.com/rust-lang/rust-clippy/ --force clippy
- cargo install --force cargo-audit
- cargo generate-lockfile
script:
- cargo audit
# We use OSX so that we can get a reasonably up to date version of SQLCipher.
# (The version in Travis's default Ubuntu Trusty is much too old).
os: osx
before_install:
- brew install sqlcipher --with-fts
- brew install sqlcipher
rust:
- 1.25.0
- 1.41.0
- stable
- beta
- nightly
matrix:
allow_failures:
- rust: stable
- rust: beta
- rust: nightly
fast_finish: true
jobs:
include:
- stage: "Test iOS"
rust: 1.25.0
rust: 1.41.0
script: ./scripts/test-ios.sh
- stage: "Docs"
rust: 1.25.0
rust: 1.41.0
script: ./scripts/cargo-doc.sh
script:
- cargo build --verbose --all
- cargo clippy --all-targets --all-features -- -D warnings # Check tests and non-default crate features.
- cargo test --verbose --all
- cargo test --features edn/serde_support --verbose --all
# We can't pick individual features out with `cargo test --all` (At the time of this writing, this

7
Cargo.toml
View file

@ -28,6 +28,13 @@ members = ["tools/cli", "ffi"]
[build-dependencies]
rustc_version = "0.2"
[dev-dependencies.cargo-husky]
version = "1"
default-features = false # Disable features which are enabled by default
features = ["run-for-all", "precommit-hook", "run-cargo-fmt", "run-cargo-test", "run-cargo-check", "run-cargo-clippy"]
# cargo audit
# cargo outdated
[dependencies]
chrono = "0.4"
failure = "0.1.6"

22
README.md
View file

@ -1,17 +1,13 @@
# Project Mentat
[![Build Status](https://travis-ci.org/mozilla/mentat.svg?branch=master)](https://travis-ci.org/mozilla/mentat)
**Project Mentat is [no longer being developed or actively maintained by Mozilla](https://mail.mozilla.org/pipermail/firefox-dev/2018-September/006780.html).** This repository will be marked read-only in the near future. You are, of course, welcome to fork the repository and use the existing code.
[![Build Status](https://travis-ci.org/qpdb/mentat.svg?branch=master)](https://travis-ci.org/qpdb/mentat)
Project Mentat is a persistent, embedded knowledge base. It draws heavily on [DataScript](https://github.com/tonsky/datascript) and [Datomic](http://datomic.com).
Mentat is implemented in Rust.
This project was started by Mozilla, but [is no longer being developed or actively maintained by them](https://mail.mozilla.org/pipermail/firefox-dev/2018-September/006780.html). [Their repository](https://github.com/mozilla/mentat) was marked read-only, [this fork](https://github.com/qpdb/mentat) is an attempt to revive and continue that interesting work. We owe the team at Mozilla more than words can express for inspiring us all and for this project in particular.
The first version of Project Mentat, named Datomish, [was written in ClojureScript](https://github.com/mozilla/mentat/tree/clojure), targeting both Node (on top of `promise_sqlite`) and Firefox (on top of `Sqlite.jsm`). It also worked in pure Clojure on the JVM on top of `jdbc-sqlite`. The name was changed to avoid confusion with [Datomic](http://datomic.com).
*Thank you*.
The Rust implementation gives us a smaller compiled output, better performance, more type safety, better tooling, and easier deployment into Firefox and mobile platforms.
[Documentation](https://mozilla.github.io/mentat)
[Documentation](https://docs.rs/mentat)
---
@ -77,9 +73,11 @@ We've observed that data storage is a particular area of difficulty for software
DataScript asks the question: "What if creating a database were as cheap as creating a Hashmap?"
Mentat is not interested in that. Instead, it's strongly interested in persistence and performance, with very little interest in immutable databases/databases as values or throwaway use.
Mentat is not interested in that. Instead, it's focused on persistence and performance, with very little interest in immutable databases/databases as values or throwaway use.
One might say that Mentat's question is: "What if an SQLite database could store arbitrary relations, for arbitrary consumers, without them having to coordinate an up-front storage-level schema?"
One might say that Mentat's question is: "What if a database could store arbitrary relations, for arbitrary consumers, without them having to coordinate an up-front storage-level schema?"
Consider this a practical approach to facts, to knowledge its storage and access, much like SQLite is a practical RDBMS.
(Note that [domain-level schemas are very valuable](http://martinfowler.com/articles/schemaless/).)
@ -89,7 +87,7 @@ Some thought has been given to how databases as values — long-term references
Just like DataScript, Mentat speaks Datalog for querying and takes additions and retractions as input to a transaction.
Unlike DataScript, Mentat exposes free-text indexing, thanks to SQLite.
Unlike DataScript, Mentat exposes free-text indexing, thanks to SQLite/FTS.
## Comparison to Datomic
@ -98,8 +96,6 @@ Datomic is a server-side, enterprise-grade data storage system. Datomic has a be
Many of these design decisions are inapplicable to deployed desktop software; indeed, the use of multiple JVM processes makes Datomic's use in a small desktop app, or a mobile device, prohibitive.
Mentat was designed for embedding, initially in an experimental Electron app ([Tofino](https://github.com/mozilla/tofino)). It is less concerned with exposing consistent database states outside transaction boundaries, because that's less important here, and dropping some of these requirements allows us to leverage SQLite itself.
## Comparison to SQLite

2
build/version.rs
View file

@ -14,7 +14,7 @@ use std::process::exit;
/// MIN_VERSION should be changed when there's a new minimum version of rustc required
/// to build the project.
static MIN_VERSION: &'static str = "1.41.0";
static MIN_VERSION: &str = "1.40.0";
fn main() {
let ver = version().unwrap();

76
core-traits/lib.rs
View file

@ -102,7 +102,7 @@ impl<V: TransactableValueMarker> Into<ValuePlace<V>> for KnownEntid {
/// When moving to a more concrete table, such as `datoms`, they are expanded out
/// via these flags and put into their own column rather than a bit field.
pub enum AttributeBitFlags {
IndexAVET = 1 << 0,
IndexAVET = 1,
IndexVAET = 1 << 1,
IndexFulltext = 1 << 2,
UniqueValue = 1 << 3,
@ -327,20 +327,20 @@ impl ValueType {
pub fn from_keyword(keyword: &Keyword) -> Option<Self> {
if keyword.namespace() != Some("db.type") {
return None;
None
} else {
match keyword.name() {
"ref" => Some(ValueType::Ref),
"boolean" => Some(ValueType::Boolean),
"instant" => Some(ValueType::Instant),
"long" => Some(ValueType::Long),
"double" => Some(ValueType::Double),
"string" => Some(ValueType::String),
"keyword" => Some(ValueType::Keyword),
"uuid" => Some(ValueType::Uuid),
_ => None,
}
}
return match keyword.name() {
"ref" => Some(ValueType::Ref),
"boolean" => Some(ValueType::Boolean),
"instant" => Some(ValueType::Instant),
"long" => Some(ValueType::Long),
"double" => Some(ValueType::Double),
"string" => Some(ValueType::String),
"keyword" => Some(ValueType::Keyword),
"uuid" => Some(ValueType::Uuid),
_ => None,
};
}
pub fn into_typed_value(self) -> TypedValue {
@ -372,9 +372,9 @@ impl ValueType {
}
}
pub fn is_numeric(&self) -> bool {
pub fn is_numeric(self) -> bool {
match self {
&ValueType::Long | &ValueType::Double => true,
ValueType::Long | ValueType::Double => true,
_ => false,
}
}
@ -440,14 +440,14 @@ impl TypedValue {
pub fn value_type(&self) -> ValueType {
match self {
&TypedValue::Ref(_) => ValueType::Ref,
&TypedValue::Boolean(_) => ValueType::Boolean,
&TypedValue::Long(_) => ValueType::Long,
&TypedValue::Instant(_) => ValueType::Instant,
&TypedValue::Double(_) => ValueType::Double,
&TypedValue::String(_) => ValueType::String,
&TypedValue::Keyword(_) => ValueType::Keyword,
&TypedValue::Uuid(_) => ValueType::Uuid,
TypedValue::Ref(_) => ValueType::Ref,
TypedValue::Boolean(_) => ValueType::Boolean,
TypedValue::Long(_) => ValueType::Long,
TypedValue::Instant(_) => ValueType::Instant,
TypedValue::Double(_) => ValueType::Double,
TypedValue::String(_) => ValueType::String,
TypedValue::Keyword(_) => ValueType::Keyword,
TypedValue::Uuid(_) => ValueType::Uuid,
}
}
@ -770,21 +770,21 @@ impl Binding {
pub fn as_scalar(&self) -> Option<&TypedValue> {
match self {
&Binding::Scalar(ref v) => Some(v),
Binding::Scalar(ref v) => Some(v),
_ => None,
}
}
pub fn as_vec(&self) -> Option<&Vec<Binding>> {
match self {
&Binding::Vec(ref v) => Some(v),
Binding::Vec(ref v) => Some(v),
_ => None,
}
}
pub fn as_map(&self) -> Option<&StructuredMap> {
match self {
&Binding::Map(ref v) => Some(v),
Binding::Map(ref v) => Some(v),
_ => None,
}
}
@ -856,10 +856,10 @@ impl Binding {
pub fn value_type(&self) -> Option<ValueType> {
match self {
&Binding::Scalar(ref v) => Some(v.value_type()),
Binding::Scalar(ref v) => Some(v.value_type()),
&Binding::Map(_) => None,
&Binding::Vec(_) => None,
Binding::Map(_) => None,
Binding::Vec(_) => None,
}
}
}
@ -970,56 +970,56 @@ impl Binding {
pub fn as_entid(&self) -> Option<&Entid> {
match self {
&Binding::Scalar(TypedValue::Ref(ref v)) => Some(v),
Binding::Scalar(TypedValue::Ref(ref v)) => Some(v),
_ => None,
}
}
pub fn as_kw(&self) -> Option<&ValueRc<Keyword>> {
match self {
&Binding::Scalar(TypedValue::Keyword(ref v)) => Some(v),
Binding::Scalar(TypedValue::Keyword(ref v)) => Some(v),
_ => None,
}
}
pub fn as_boolean(&self) -> Option<&bool> {
match self {
&Binding::Scalar(TypedValue::Boolean(ref v)) => Some(v),
Binding::Scalar(TypedValue::Boolean(ref v)) => Some(v),
_ => None,
}
}
pub fn as_long(&self) -> Option<&i64> {
match self {
&Binding::Scalar(TypedValue::Long(ref v)) => Some(v),
Binding::Scalar(TypedValue::Long(ref v)) => Some(v),
_ => None,
}
}
pub fn as_double(&self) -> Option<&f64> {
match self {
&Binding::Scalar(TypedValue::Double(ref v)) => Some(&v.0),
Binding::Scalar(TypedValue::Double(ref v)) => Some(&v.0),
_ => None,
}
}
pub fn as_instant(&self) -> Option<&DateTime<Utc>> {
match self {
&Binding::Scalar(TypedValue::Instant(ref v)) => Some(v),
Binding::Scalar(TypedValue::Instant(ref v)) => Some(v),
_ => None,
}
}
pub fn as_string(&self) -> Option<&ValueRc<String>> {
match self {
&Binding::Scalar(TypedValue::String(ref v)) => Some(v),
Binding::Scalar(TypedValue::String(ref v)) => Some(v),
_ => None,
}
}
pub fn as_uuid(&self) -> Option<&Uuid> {
match self {
&Binding::Scalar(TypedValue::Uuid(ref v)) => Some(v),
Binding::Scalar(TypedValue::Uuid(ref v)) => Some(v),
_ => None,
}
}

26
core-traits/value_type_set.rs
View file

@ -92,53 +92,53 @@ impl ValueTypeSet {
self.0.insert(vt)
}
pub fn len(&self) -> usize {
pub fn len(self) -> usize {
self.0.len()
}
/// Returns a set containing all the types in this set and `other`.
pub fn union(&self, other: &ValueTypeSet) -> ValueTypeSet {
pub fn union(self, other: ValueTypeSet) -> ValueTypeSet {
ValueTypeSet(self.0.union(other.0))
}
pub fn intersection(&self, other: &ValueTypeSet) -> ValueTypeSet {
pub fn intersection(self, other: ValueTypeSet) -> ValueTypeSet {
ValueTypeSet(self.0.intersection(other.0))
}
/// Returns the set difference between `self` and `other`, which is the
/// set of items in `self` that are not in `other`.
pub fn difference(&self, other: &ValueTypeSet) -> ValueTypeSet {
pub fn difference(self, other: ValueTypeSet) -> ValueTypeSet {
ValueTypeSet(self.0 - other.0)
}
/// Return an arbitrary type that's part of this set.
/// For a set containing a single type, this will be that type.
pub fn exemplar(&self) -> Option<ValueType> {
pub fn exemplar(self) -> Option<ValueType> {
self.0.iter().next()
}
pub fn is_subset(&self, other: &ValueTypeSet) -> bool {
pub fn is_subset(self, other: ValueTypeSet) -> bool {
self.0.is_subset(&other.0)
}
/// Returns true if `self` and `other` contain no items in common.
pub fn is_disjoint(&self, other: &ValueTypeSet) -> bool {
pub fn is_disjoint(self, other: ValueTypeSet) -> bool {
self.0.is_disjoint(&other.0)
}
pub fn contains(&self, vt: ValueType) -> bool {
pub fn contains(self, vt: ValueType) -> bool {
self.0.contains(&vt)
}
pub fn is_empty(&self) -> bool {
pub fn is_empty(self) -> bool {
self.0.is_empty()
}
pub fn is_unit(&self) -> bool {
pub fn is_unit(self) -> bool {
self.0.len() == 1
}
pub fn iter(&self) -> ::enum_set::Iter<ValueType> {
pub fn iter(self) -> ::enum_set::Iter<ValueType> {
self.0.iter()
}
}
@ -150,8 +150,8 @@ impl From<ValueType> for ValueTypeSet {
}
impl ValueTypeSet {
pub fn is_only_numeric(&self) -> bool {
self.is_subset(&ValueTypeSet::of_numeric_types())
pub fn is_only_numeric(self) -> bool {
self.is_subset(ValueTypeSet::of_numeric_types())
}
}

8
core-traits/values.rs
View file

@ -19,11 +19,11 @@ use edn::types::Value;
/// Declare a lazy static `ident` of type `Value::Keyword` with the given `namespace` and
/// `name`.
///
/// It may look surprising that we declare a new `lazy_static!` block rather than including
/// It may look surprising to declare a new `lazy_static!` block rather than including
/// invocations inside an existing `lazy_static!` block. The latter cannot be done, since macros
/// are expanded outside-in. Looking at the `lazy_static!` source suggests that there is no harm in
/// repeating that macro, since internally a multi-`static` block is expanded into many
/// single-`static` blocks.
/// will be expanded outside-in. Looking at the `lazy_static!` source suggests that there is no
/// harm in repeating that macro, since internally a multi-`static` block will be expanded into
/// many single-`static` blocks.
///
/// TODO: take just ":db.part/db" and define DB_PART_DB using "db.part" and "db".
macro_rules! lazy_static_namespaced_keyword_value (

2
core/src/counter.rs
View file

@ -11,7 +11,7 @@
use std::cell::Cell;
use std::rc::Rc;
#[derive(Clone)]
#[derive(Clone, Default)]
pub struct RcCounter {
c: Rc<Cell<usize>>,
}

2
core/src/lib.rs
View file

@ -135,7 +135,7 @@ impl Schema {
}
fn get_raw_entid(&self, x: &Keyword) -> Option<Entid> {
self.ident_map.get(x).map(|x| *x)
self.ident_map.get(x).copied()
}
pub fn update_component_attributes(&mut self) {

14
db-traits/errors.rs
View file

@ -69,7 +69,7 @@ impl ::std::fmt::Display for SchemaConstraintViolation {
fn fmt(&self, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result {
use self::SchemaConstraintViolation::*;
match self {
&ConflictingUpserts {
ConflictingUpserts {
ref conflicting_upserts,
} => {
writeln!(f, "conflicting upserts:")?;
@ -78,7 +78,7 @@ impl ::std::fmt::Display for SchemaConstraintViolation {
}
Ok(())
}
&TypeDisagreements {
TypeDisagreements {
ref conflicting_datoms,
} => {
writeln!(f, "type disagreements:")?;
@ -91,9 +91,9 @@ impl ::std::fmt::Display for SchemaConstraintViolation {
}
Ok(())
}
&CardinalityConflicts { ref conflicts } => {
CardinalityConflicts { ref conflicts } => {
writeln!(f, "cardinality conflicts:")?;
for ref conflict in conflicts {
for conflict in conflicts {
writeln!(f, " {:?}", conflict)?;
}
Ok(())
@ -116,10 +116,10 @@ impl ::std::fmt::Display for InputError {
fn fmt(&self, f: &mut ::std::fmt::Formatter) -> ::std::fmt::Result {
use self::InputError::*;
match self {
&BadDbId => {
BadDbId => {
writeln!(f, ":db/id in map notation must either not be present or be an entid, an ident, or a tempid")
},
&BadEntityPlace => {
BadEntityPlace => {
writeln!(f, "cannot convert value place into entity place")
},
}
@ -163,7 +163,7 @@ impl From<DbErrorKind> for DbError {
impl From<Context<DbErrorKind>> for DbError {
fn from(inner: Context<DbErrorKind>) -> Self {
DbError { inner: inner }
DbError { inner }
}
}

8
db/src/add_retract_alter_set.rs
View file

@ -48,12 +48,10 @@ where
} else {
self.asserted.insert(key, value);
}
} else if let Some(asserted_value) = self.asserted.remove(&key) {
self.altered.insert(key, (value, asserted_value));
} else {
if let Some(asserted_value) = self.asserted.remove(&key) {
self.altered.insert(key, (value, asserted_value));
} else {
self.retracted.insert(key, value);
}
self.retracted.insert(key, value);
}
}
}

20
db/src/bootstrap.rs
View file

@ -27,7 +27,7 @@ use types::{Partition, PartitionMap};
/// The first transaction ID applied to the knowledge base.
///
/// This is the start of the :db.part/tx partition.
pub const TX0: i64 = 0x10000000;
pub const TX0: i64 = 0x1000_0000;
/// This is the start of the :db.part/user partition.
pub const USER0: i64 = 0x10000;
@ -206,14 +206,14 @@ lazy_static! {
/// Convert (ident, entid) pairs into [:db/add IDENT :db/ident IDENT] `Value` instances.
fn idents_to_assertions(idents: &[(symbols::Keyword, i64)]) -> Vec<Value> {
idents
.into_iter()
.iter()
.map(|&(ref ident, _)| {
let value = Value::Keyword(ident.clone());
Value::Vector(vec![
values::DB_ADD.clone(),
value.clone(),
values::DB_IDENT.clone(),
value.clone(),
value,
])
})
.collect()
@ -225,7 +225,7 @@ fn schema_attrs_to_assertions(version: u32, idents: &[symbols::Keyword]) -> Vec<
let schema_attr = Value::Keyword(ns_keyword!("db.schema", "attribute"));
let schema_version = Value::Keyword(ns_keyword!("db.schema", "version"));
idents
.into_iter()
.iter()
.map(|ident| {
let value = Value::Keyword(ident.clone());
Value::Vector(vec![
@ -260,7 +260,7 @@ fn symbolic_schema_to_triples(
Value::Map(ref m) => {
for (ident, mp) in m {
let ident = match ident {
&Value::Keyword(ref ident) => ident,
Value::Keyword(ref ident) => ident,
_ => bail!(DbErrorKind::BadBootstrapDefinition(format!(
"Expected namespaced keyword for ident but got '{:?}'",
ident
@ -270,7 +270,7 @@ fn symbolic_schema_to_triples(
Value::Map(ref mpp) => {
for (attr, value) in mpp {
let attr = match attr {
&Value::Keyword(ref attr) => attr,
Value::Keyword(ref attr) => attr,
_ => bail!(DbErrorKind::BadBootstrapDefinition(format!(
"Expected namespaced keyword for attr but got '{:?}'",
attr
@ -289,7 +289,7 @@ fn symbolic_schema_to_triples(
Some(TypedValue::Keyword(ref k)) => ident_map
.get(k)
.map(|entid| TypedValue::Ref(*entid))
.ok_or(DbErrorKind::UnrecognizedIdent(k.to_string()))?,
.ok_or_else(|| DbErrorKind::UnrecognizedIdent(k.to_string()))?,
Some(v) => v,
_ => bail!(DbErrorKind::BadBootstrapDefinition(format!(
"Expected Mentat typed value for value but got '{:?}'",
@ -377,8 +377,6 @@ pub(crate) fn bootstrap_entities() -> Vec<Entity<edn::ValueAndSpan>> {
);
// Failure here is a coding error (since the inputs are fixed), not a runtime error.
// TODO: represent these bootstrap data errors rather than just panicing.
let bootstrap_entities: Vec<Entity<edn::ValueAndSpan>> =
edn::parse::entities(&bootstrap_assertions.to_string()).expect("bootstrap assertions");
return bootstrap_entities;
// TODO: represent these bootstrap entity data errors rather than just panicing.
edn::parse::entities(&bootstrap_assertions.to_string()).expect("bootstrap assertions")
}

58
db/src/cache.rs
View file

@ -54,8 +54,6 @@ use std::collections::btree_map::Entry::{Occupied, Vacant};
use std::iter::once;
use std::mem;
use std::sync::Arc;
use std::iter::Peekable;
@ -190,7 +188,7 @@ impl AevFactory {
return existing;
}
self.strings.insert(rc.clone());
return TypedValue::String(rc);
TypedValue::String(rc)
}
t => t,
}
@ -377,7 +375,7 @@ impl RemoveFromCache for MultiValAttributeCache {
impl CardinalityManyCache for MultiValAttributeCache {
fn acc(&mut self, e: Entid, v: TypedValue) {
self.e_vs.entry(e).or_insert(vec![]).push(v)
self.e_vs.entry(e).or_insert_with(|| vec![]).push(v)
}
fn set(&mut self, e: Entid, vs: Vec<TypedValue>) {
@ -439,7 +437,7 @@ impl UniqueReverseAttributeCache {
}
fn get_e(&self, v: &TypedValue) -> Option<Entid> {
self.v_e.get(v).and_then(|o| o.clone())
self.v_e.get(v).and_then(|o| *o)
}
fn lookup(&self, v: &TypedValue) -> Option<Option<Entid>> {
@ -494,7 +492,7 @@ impl RemoveFromCache for NonUniqueReverseAttributeCache {
impl NonUniqueReverseAttributeCache {
fn acc(&mut self, e: Entid, v: TypedValue) {
self.v_es.entry(v).or_insert(BTreeSet::new()).insert(e);
self.v_es.entry(v).or_insert_with(BTreeSet::new).insert(e);
}
fn get_es(&self, v: &TypedValue) -> Option<&BTreeSet<Entid>> {
@ -643,9 +641,9 @@ enum AccumulationBehavior {
}
impl AccumulationBehavior {
fn is_replacing(&self) -> bool {
fn is_replacing(self) -> bool {
match self {
&AccumulationBehavior::Add { replacing } => replacing,
AccumulationBehavior::Add { replacing } => replacing,
_ => false,
}
}
@ -662,7 +660,7 @@ pub struct AttributeCaches {
non_unique_reverse: BTreeMap<Entid, NonUniqueReverseAttributeCache>,
}
// TODO: if an entity or attribute is ever renumbered, the cache will need to be rebuilt.
// TODO: if an entity or attribute is ever re-numbered, the cache will need to be rebuilt.
impl AttributeCaches {
//
// These function names are brief and local.
@ -1006,7 +1004,7 @@ impl AttributeCaches {
}
}
// We need this block for fallback.
// We need this block for fall-back.
impl AttributeCaches {
fn get_entid_for_value_if_present(
&self,
@ -1076,7 +1074,7 @@ impl AttributeCaches {
) -> Result<()> {
let mut aev_factory = AevFactory::new();
let rows = statement.query_map(&args, |row| Ok(aev_factory.row_to_aev(row)))?;
let aevs = AevRows { rows: rows };
let aevs = AevRows { rows };
self.accumulate_into_cache(
None,
schema,
@ -1132,7 +1130,7 @@ impl AttributeCaches {
schema: &'s Schema,
sqlite: &'c rusqlite::Connection,
attrs: AttributeSpec,
entities: &Vec<Entid>,
entities: &[Entid],
) -> Result<()> {
// Mark the attributes as cached as we go. We do this because we're going in through the
// back door here, and the usual caching API won't have taken care of this for us.
@ -1229,17 +1227,17 @@ impl AttributeCaches {
schema: &'s Schema,
sqlite: &'c rusqlite::Connection,
mut attrs: AttributeSpec,
entities: &Vec<Entid>,
entities: &[Entid],
) -> Result<()> {
// TODO: Exclude any entities for which every attribute is known.
// TODO: initialize from an existing (complete) AttributeCache.
// Exclude any attributes for which every entity's value is already known.
match &mut attrs {
&mut AttributeSpec::All => {
AttributeSpec::All => {
// If we're caching all attributes, there's nothing we can exclude.
}
&mut AttributeSpec::Specified {
AttributeSpec::Specified {
ref mut non_fts,
ref mut fts,
} => {
@ -1285,7 +1283,7 @@ impl AttributeCaches {
schema: &'s Schema,
sqlite: &'c rusqlite::Connection,
attrs: AttributeSpec,
entities: &Vec<Entid>,
entities: &[Entid],
) -> Result<AttributeCaches> {
let mut cache = AttributeCaches::default();
cache.populate_cache_for_entities_and_attributes(schema, sqlite, attrs, entities)?;
@ -1450,7 +1448,7 @@ pub struct SQLiteAttributeCache {
}
impl SQLiteAttributeCache {
fn make_mut<'s>(&'s mut self) -> &'s mut AttributeCaches {
fn make_mut(&mut self) -> &mut AttributeCaches {
Arc::make_mut(&mut self.inner)
}
@ -1628,7 +1626,7 @@ impl InProgressSQLiteAttributeCache {
let overlay = inner.make_override();
InProgressSQLiteAttributeCache {
inner: inner.inner,
overlay: overlay,
overlay,
unregistered_forward: Default::default(),
unregistered_reverse: Default::default(),
}
@ -1818,9 +1816,7 @@ impl CachedAttributes for InProgressSQLiteAttributeCache {
.inner
.forward_cached_attributes
.iter()
.filter(|a| !self.unregistered_forward.contains(a))
.next()
.is_some()
.any(|a| !self.unregistered_forward.contains(a))
{
return true;
}
@ -1828,9 +1824,7 @@ impl CachedAttributes for InProgressSQLiteAttributeCache {
self.inner
.reverse_cached_attributes
.iter()
.filter(|a| !self.unregistered_reverse.contains(a))
.next()
.is_some()
.any(|a| !self.unregistered_reverse.contains(a))
}
fn get_entids_for_value(
@ -1944,7 +1938,7 @@ impl<'a> InProgressCacheTransactWatcher<'a> {
let mut w = InProgressCacheTransactWatcher {
collected_assertions: Default::default(),
collected_retractions: Default::default(),
cache: cache,
cache,
active: true,
};
@ -1977,10 +1971,10 @@ impl<'a> TransactWatcher for InProgressCacheTransactWatcher<'a> {
}
Entry::Occupied(mut entry) => {
match entry.get_mut() {
&mut Either::Left(_) => {
Either::Left(_) => {
// Nothing to do.
}
&mut Either::Right(ref mut vec) => {
Either::Right(ref mut vec) => {
vec.push((e, v.clone()));
}
}
@ -1989,14 +1983,12 @@ impl<'a> TransactWatcher for InProgressCacheTransactWatcher<'a> {
}
fn done(&mut self, _t: &Entid, schema: &Schema) -> Result<()> {
// Oh, I wish we had impl trait. Without it we have a six-line type signature if we
// Oh, how I wish we had `impl trait`. Without it we have a six-line type signature if we
// try to break this out as a helper function.
let collected_retractions =
mem::replace(&mut self.collected_retractions, Default::default());
let collected_assertions = mem::replace(&mut self.collected_assertions, Default::default());
let collected_retractions = std::mem::take(&mut self.collected_retractions);
let collected_assertions = std::mem::take(&mut self.collected_assertions);
let mut intermediate_expansion = once(collected_retractions)
.chain(once(collected_assertions))
.into_iter()
.map(move |tree| {
tree.into_iter()
.filter_map(move |(a, evs)| {
@ -2018,7 +2010,7 @@ impl<'a> TransactWatcher for InProgressCacheTransactWatcher<'a> {
}
impl InProgressSQLiteAttributeCache {
pub fn transact_watcher<'a>(&'a mut self) -> InProgressCacheTransactWatcher<'a> {
pub fn transact_watcher(&mut self) -> InProgressCacheTransactWatcher {
InProgressCacheTransactWatcher::new(self)
}
}

112
db/src/db.rs
View file

@ -66,10 +66,9 @@ fn make_connection(
let page_size = 32768;
let initial_pragmas = if let Some(encryption_key) = maybe_encryption_key {
assert!(
cfg!(feature = "sqlcipher"),
"This function shouldn't be called with a key unless we have sqlcipher support"
);
if !cfg!(feature = "sqlcipher") {
panic!("This function shouldn't be called with a key unless we have sqlcipher support");
}
// Important: The `cipher_page_size` cannot be changed without breaking
// the ability to open databases that were written when using a
// different `cipher_page_size`. Additionally, it (AFAICT) must be a
@ -147,10 +146,10 @@ pub const CURRENT_VERSION: i32 = 1;
/// MIN_SQLITE_VERSION should be changed when there's a new minimum version of sqlite required
/// for the project to work.
const MIN_SQLITE_VERSION: i32 = 3008000;
const MIN_SQLITE_VERSION: i32 = 3_008_000;
const TRUE: &'static bool = &true;
const FALSE: &'static bool = &false;
const TRUE: &bool = &true;
const FALSE: &bool = &false;
/// Turn an owned bool into a static reference to a bool.
///
@ -360,9 +359,10 @@ pub fn create_current_version(conn: &mut rusqlite::Connection) -> Result<DB> {
// TODO: validate metadata mutations that aren't schema related, like additional partitions.
if let Some(next_schema) = next_schema {
if next_schema != db.schema {
bail!(DbErrorKind::NotYetImplemented(format!(
bail!(DbErrorKind::NotYetImplemented(
"Initial bootstrap transaction did not produce expected bootstrap schema"
)));
.to_string()
));
}
}
@ -396,7 +396,7 @@ pub trait TypedSQLValue {
value: rusqlite::types::Value,
value_type_tag: i32,
) -> Result<TypedValue>;
fn to_sql_value_pair<'a>(&'a self) -> (ToSqlOutput<'a>, i32);
fn to_sql_value_pair(&self) -> (ToSqlOutput, i32);
fn from_edn_value(value: &Value) -> Option<TypedValue>;
fn to_edn_value_pair(&self) -> (Value, ValueType);
}
@ -446,43 +446,43 @@ impl TypedSQLValue for TypedValue {
/// This function is deterministic.
fn from_edn_value(value: &Value) -> Option<TypedValue> {
match value {
&Value::Boolean(x) => Some(TypedValue::Boolean(x)),
&Value::Instant(x) => Some(TypedValue::Instant(x)),
&Value::Integer(x) => Some(TypedValue::Long(x)),
&Value::Uuid(x) => Some(TypedValue::Uuid(x)),
&Value::Float(ref x) => Some(TypedValue::Double(x.clone())),
&Value::Text(ref x) => Some(x.clone().into()),
&Value::Keyword(ref x) => Some(x.clone().into()),
Value::Boolean(x) => Some(TypedValue::Boolean(*x)),
Value::Instant(x) => Some(TypedValue::Instant(*x)),
Value::Integer(x) => Some(TypedValue::Long(*x)),
Value::Uuid(x) => Some(TypedValue::Uuid(*x)),
Value::Float(ref x) => Some(TypedValue::Double(*x)),
Value::Text(ref x) => Some(x.clone().into()),
Value::Keyword(ref x) => Some(x.clone().into()),
_ => None,
}
}
/// Return the corresponding SQLite `value` and `value_type_tag` pair.
fn to_sql_value_pair<'a>(&'a self) -> (ToSqlOutput<'a>, i32) {
fn to_sql_value_pair(&self) -> (ToSqlOutput, i32) {
match self {
&TypedValue::Ref(x) => (x.into(), 0),
&TypedValue::Boolean(x) => ((if x { 1 } else { 0 }).into(), 1),
&TypedValue::Instant(x) => (x.to_micros().into(), 4),
TypedValue::Ref(x) => ((*x).into(), 0),
TypedValue::Boolean(x) => ((if *x { 1 } else { 0 }).into(), 1),
TypedValue::Instant(x) => (x.to_micros().into(), 4),
// SQLite distinguishes integral from decimal types, allowing long and double to share a tag.
&TypedValue::Long(x) => (x.into(), 5),
&TypedValue::Double(x) => (x.into_inner().into(), 5),
&TypedValue::String(ref x) => (x.as_str().into(), 10),
&TypedValue::Uuid(ref u) => (u.as_bytes().to_vec().into(), 11),
&TypedValue::Keyword(ref x) => (x.to_string().into(), 13),
TypedValue::Long(x) => ((*x).into(), 5),
TypedValue::Double(x) => (x.into_inner().into(), 5),
TypedValue::String(ref x) => (x.as_str().into(), 10),
TypedValue::Uuid(ref u) => (u.as_bytes().to_vec().into(), 11),
TypedValue::Keyword(ref x) => (x.to_string().into(), 13),
}
}
/// Return the corresponding EDN `value` and `value_type` pair.
fn to_edn_value_pair(&self) -> (Value, ValueType) {
match self {
&TypedValue::Ref(x) => (Value::Integer(x), ValueType::Ref),
&TypedValue::Boolean(x) => (Value::Boolean(x), ValueType::Boolean),
&TypedValue::Instant(x) => (Value::Instant(x), ValueType::Instant),
&TypedValue::Long(x) => (Value::Integer(x), ValueType::Long),
&TypedValue::Double(x) => (Value::Float(x), ValueType::Double),
&TypedValue::String(ref x) => (Value::Text(x.as_ref().clone()), ValueType::String),
&TypedValue::Uuid(ref u) => (Value::Uuid(u.clone()), ValueType::Uuid),
&TypedValue::Keyword(ref x) => (Value::Keyword(x.as_ref().clone()), ValueType::Keyword),
TypedValue::Ref(x) => (Value::Integer(*x), ValueType::Ref),
TypedValue::Boolean(x) => (Value::Boolean(*x), ValueType::Boolean),
TypedValue::Instant(x) => (Value::Instant(*x), ValueType::Instant),
TypedValue::Long(x) => (Value::Integer(*x), ValueType::Long),
TypedValue::Double(x) => (Value::Float(*x), ValueType::Double),
TypedValue::String(ref x) => (Value::Text(x.as_ref().clone()), ValueType::String),
TypedValue::Uuid(ref u) => (Value::Uuid(*u), ValueType::Uuid),
TypedValue::Keyword(ref x) => (Value::Keyword(x.as_ref().clone()), ValueType::Keyword),
}
}
}
@ -510,7 +510,7 @@ pub fn read_partition_map(conn: &rusqlite::Connection) -> Result<PartitionMap> {
// First part of the union sprinkles 'allow_excision' into the 'parts' view.
// Second part of the union takes care of partitions which are known
// but don't have any transactions.
let mut stmt: rusqlite::Statement = conn.prepare(
conn.prepare(
"
SELECT
known_parts.part,
@ -536,16 +536,14 @@ pub fn read_partition_map(conn: &rusqlite::Connection) -> Result<PartitionMap> {
known_parts
WHERE
part NOT IN (SELECT part FROM parts)",
)?;
let m = stmt
.query_and_then(rusqlite::params![], |row| -> Result<(String, Partition)> {
Ok((
row.get(0)?,
Partition::new(row.get(1)?, row.get(2)?, row.get(3)?, row.get(4)?),
))
})?
.collect();
m
)?
.query_and_then(rusqlite::params![], |row| -> Result<(String, Partition)> {
Ok((
row.get(0)?,
Partition::new(row.get(1)?, row.get(2)?, row.get(3)?, row.get(4)?),
))
})?
.collect()
}
/// Read the ident map materialized view from the given SQL store.
@ -767,7 +765,7 @@ impl MentatStoring for rusqlite::Connection {
//
// TODO: `collect` into a HashSet so that any (a, v) is resolved at most once.
let max_vars = self.limit(Limit::SQLITE_LIMIT_VARIABLE_NUMBER) as usize;
let chunks: itertools::IntoChunks<_> = avs.into_iter().enumerate().chunks(max_vars / 4);
let chunks: itertools::IntoChunks<_> = avs.iter().enumerate().chunks(max_vars / 4);
// We'd like to `flat_map` here, but it's not obvious how to `flat_map` across `Result`.
// Alternatively, this is a `fold`, and it might be wise to express it as such.
@ -900,9 +898,8 @@ impl MentatStoring for rusqlite::Connection {
let bindings_per_statement = 6;
let max_vars = self.limit(Limit::SQLITE_LIMIT_VARIABLE_NUMBER) as usize;
let chunks: itertools::IntoChunks<_> = entities
.into_iter()
.chunks(max_vars / bindings_per_statement);
let chunks: itertools::IntoChunks<_> =
entities.iter().chunks(max_vars / bindings_per_statement);
// We'd like to flat_map here, but it's not obvious how to flat_map across Result.
let results: Result<Vec<()>> = chunks.into_iter().map(|chunk| -> Result<()> {
@ -973,9 +970,8 @@ impl MentatStoring for rusqlite::Connection {
let mut outer_searchid = 2000;
let chunks: itertools::IntoChunks<_> = entities
.into_iter()
.chunks(max_vars / bindings_per_statement);
let chunks: itertools::IntoChunks<_> =
entities.iter().chunks(max_vars / bindings_per_statement);
// From string to (searchid, value_type_tag).
let mut seen: HashMap<ValueRc<String>, (i64, i32)> = HashMap::with_capacity(entities.len());
@ -996,7 +992,7 @@ impl MentatStoring for rusqlite::Connection {
u8 /* flags0 */,
i64 /* searchid */)>> = chunk.map(|&(e, a, ref attribute, ref typed_value, added)| {
match typed_value {
&TypedValue::String(ref rc) => {
TypedValue::String(ref rc) => {
datom_count += 1;
let entry = seen.entry(rc.clone());
match entry {
@ -1186,7 +1182,10 @@ pub fn update_metadata(
// TODO: use concat! to avoid creating String instances.
if !metadata_report.idents_altered.is_empty() {
// Idents is the materialized view of the [entid :db/ident ident] slice of datoms.
conn.execute(format!("DELETE FROM idents").as_str(), rusqlite::params![])?;
conn.execute(
"DELETE FROM idents".to_string().as_str(),
rusqlite::params![],
)?;
conn.execute(
format!(
"INSERT INTO idents SELECT e, a, v, value_type_tag FROM datoms WHERE a IN {}",
@ -1208,7 +1207,10 @@ pub fn update_metadata(
|| !metadata_report.attributes_altered.is_empty()
|| !metadata_report.idents_altered.is_empty()
{
conn.execute(format!("DELETE FROM schema").as_str(), rusqlite::params![])?;
conn.execute(
"DELETE FROM schema".to_string().as_str(),
rusqlite::params![],
)?;
// NB: we're using :db/valueType as a placeholder for the entire schema-defining set.
let s = format!(
r#"

23
db/src/debug.rs
View file

@ -117,7 +117,7 @@ impl Datom {
pub fn to_edn(&self) -> edn::Value {
let f = |entid: &EntidOrIdent| -> edn::Value {
match *entid {
EntidOrIdent::Entid(ref y) => edn::Value::Integer(y.clone()),
EntidOrIdent::Entid(ref y) => edn::Value::Integer(*y),
EntidOrIdent::Ident(ref y) => edn::Value::Keyword(y.clone()),
}
};
@ -134,13 +134,13 @@ impl Datom {
impl Datoms {
pub fn to_edn(&self) -> edn::Value {
edn::Value::Vector((&self.0).into_iter().map(|x| x.to_edn()).collect())
edn::Value::Vector((&self.0).iter().map(|x| x.to_edn()).collect())
}
}
impl Transactions {
pub fn to_edn(&self) -> edn::Value {
edn::Value::Vector((&self.0).into_iter().map(|x| x.to_edn()).collect())
edn::Value::Vector((&self.0).iter().map(|x| x.to_edn()).collect())
}
}
@ -148,7 +148,7 @@ impl FulltextValues {
pub fn to_edn(&self) -> edn::Value {
edn::Value::Vector(
(&self.0)
.into_iter()
.iter()
.map(|&(x, ref y)| {
edn::Value::Vector(vec![edn::Value::Integer(x), edn::Value::Text(y.clone())])
})
@ -238,7 +238,7 @@ pub fn datoms_after<S: Borrow<Schema>>(
e: EntidOrIdent::Entid(e),
a: to_entid(borrowed_schema, a),
v: value,
tx: tx,
tx,
added: None,
}))
})?
@ -286,7 +286,7 @@ pub fn transactions_after<S: Borrow<Schema>>(
e: EntidOrIdent::Entid(e),
a: to_entid(borrowed_schema, a),
v: value,
tx: tx,
tx,
added: Some(added),
})
})?
@ -332,12 +332,12 @@ pub fn dump_sql_query(
let mut stmt: rusqlite::Statement = conn.prepare(sql)?;
let mut tw = TabWriter::new(Vec::new()).padding(2);
write!(&mut tw, "{}\n", sql).unwrap();
writeln!(&mut tw, "{}", sql).unwrap();
for column_name in stmt.column_names() {
write!(&mut tw, "{}\t", column_name).unwrap();
}
write!(&mut tw, "\n").unwrap();
writeln!(&mut tw).unwrap();
let r: Result<Vec<_>> = stmt
.query_and_then(params, |row| {
@ -345,7 +345,7 @@ pub fn dump_sql_query(
let value: rusqlite::types::Value = row.get(i)?;
write!(&mut tw, "{:?}\t", value).unwrap();
}
write!(&mut tw, "\n").unwrap();
writeln!(&mut tw).unwrap();
Ok(())
})?
.collect();
@ -381,8 +381,9 @@ impl TestConn {
I: Borrow<str>,
{
// Failure to parse the transaction is a coding error, so we unwrap.
let entities = edn::parse::entities(transaction.borrow())
.expect(format!("to be able to parse {} into entities", transaction.borrow()).as_str());
let entities = edn::parse::entities(transaction.borrow()).unwrap_or_else(|_| {
panic!("to be able to parse {} into entities", transaction.borrow())
});
let details = {
// The block scopes the borrow of self.sqlite.

4
db/src/internal_types.rs
View file

@ -86,7 +86,7 @@ impl TransactableValue for ValueAndSpan {
.as_text()
.cloned()
.map(TempId::External)
.map(|v| v.into())
.map(|v| v)
}
}
@ -117,7 +117,7 @@ impl TransactableValue for TypedValue {
fn as_tempid(&self) -> Option<TempId> {
match self {
&TypedValue::String(ref s) => Some(TempId::External