From ac82bd45d4bd9d409e97baae0dc548ceb80b6a0f Mon Sep 17 00:00:00 2001 From: Richard Newman Date: Wed, 19 Oct 2016 09:24:04 -0700 Subject: [PATCH] Updated Proposal: application schema coordination and versioning (markdown) --- ...tion-schema-coordination-and-versioning.md | 90 +++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/Proposal:-application-schema-coordination-and-versioning.md b/Proposal:-application-schema-coordination-and-versioning.md index 66dfe5e..58ebb63 100644 --- a/Proposal:-application-schema-coordination-and-versioning.md +++ b/Proposal:-application-schema-coordination-and-versioning.md @@ -85,3 +85,93 @@ This is similar to the 'user version' functionality in SQLite, with important di - Many schema changes -- adding attributes, altering indexing choices, or weakening constraints -- can be performed automatically with no need to supply migration code. Under this proposal different applications can each ship shared schema fragments, coordinate upgrades, avoid conflicts in a large majority of cases, and safely detect real conflicts when they arise. + +# An example + +Let's start with a very simple core schema: pages have URLs and titles. + +``` +{:schema/name "org.mozilla.core.page" + :schema/version 1 + :schema/attributes [ + {:db/id (d/id-literal :db.part/user) + :db/ident :page/url + :db/valueType :db.type/string ; Because not all URLs are java.net.URIs. For JS we may want to use /uri. + :db/fulltext true + :db/cardinality :db.cardinality/one + :db/unique :db.unique/identity + :db/doc "A page's URL." + :db.install/_attribute :db.part/db} + {:db/id (d/id-literal :db.part/user) + :db/ident :page/title + :db/valueType :db.type/string + :db/fulltext true + :db/cardinality :db.cardinality/one ; We supersede as we see new titles. + :db/doc "A page's title." + :db.install/_attribute :db.part/db}]} +``` + +Adding an attribute is easy: the version number doesn't need to change. + +``` +{:schema/name "org.mozilla.core.page" + :schema/version 1 + :schema/attributes [ + … ;; Previous attributes elided for clarity. + {:db/id (d/id-literal :db.part/user) + :db/ident :page/visit + :db/valueType :db.type/ref + :db/cardinality :db.cardinality/one + :db/doc "A visit to the page." + :db.install/_attribute :db.part/db}]} +``` + +But we erroneously marked `:page/visit` as cardinality-one: you can only ever visit a page once! + +That's an easy automated fix: it's not possible for existing cardinality-one data to violate a cardinality-many restriction. So we fix it and bump the schema version. Datomish will weaken the constraint automatically. + +``` +{:schema/name "org.mozilla.core.page" + :schema/version 2 + :schema/attributes [ + … + {:db/id (d/id-literal :db.part/user) + :db/ident :page/visit + :db/valueType :db.type/ref + :db/cardinality :db.cardinality/many ; Weaken this. + :db/doc "A visit to the page." + :db.install/_attribute :db.part/db}]} +``` + +Then we recognize that we made another mistake. Visits should be unique: two pages can't share a single visit. We're confident in our application code, though, so we can again just change the schema without writing any code. + +``` +{:schema/name "org.mozilla.core.page" + :schema/version 3 + :schema/attributes [ + … + {:db/id (d/id-literal :db.part/user) + :db/ident :page/visit + :db/valueType :db.type/ref + :db/unique :db.unique/value ; Add this. + :db/cardinality :db.cardinality/many + :db/doc "A visit to the page." + :db.install/_attribute :db.part/db}]} +``` + +If we wanted to be thorough, we would supply a query like this to run when transitioning to schema version 3: + +``` +;; This query is incorrect, but you get the idea. +[:find [?e ?a ?v ?t] + :in $ + :where + [?ex :page/visit ?v ?tx] + [?e :page/visit ?v ?t] + [(!= ?ex ?e)] + [(ground :page/visit) ?a]] +``` + +This simply finds surplus datoms so they can be retracted prior to applying the schema change. + +In the general case, code needs to run against the open datom store in order to perform the schema change — for example, one might want to preserve old data using new vocabulary before altering the old schema — so a simple approach like the above isn't always enough. \ No newline at end of file