2 Consumer API
Richard Newman edited this page 2016-07-18 15:27:29 -07:00

Datomish consumer API

This is a draft for discussion purposes.

Goals:

  • The database itself should store everything it needs for basic operation. You should be able to open a database and run queries against its data without having full knowledge of its schemas. Tooling benefits from self-describing data. Applications should not routinely need to coordinate out-of-band.
  • Databases can be opened read-only or read-write. Read-only databases can only be queried.
  • Checking that your application schema is congruent (that is: if you assert datoms based on that schema, everything still makes sense) with the database schema set should be relatively cheap. This will be done on each startup.
  • Concurrent consumers within the same process — e.g., browser add-ons — should be aware of each other's schema changes.
  • Extending a database with new attributes should be easy, routine, and cheap.
  • Upgrading schema fragments in certain directions (e.g., enabling or disabling indexing) should be possible.
  • Upgrading schema fragments in other directions (e.g., changing cardinality) should be possible with some effort — e.g., by providing a transformer function that takes the current schema and transaction log and fills a new transaction log, allowing Datomish to rebuild the database to match. This is expected to be expensive and requires coordination with other consumers, and is thus not recommended.

Example in modern JS

class Consumer {

  // All schema fragments implicitly have:
  // * ':db/ident' implied by the key.
  // * ':db/id': #db/id[:db.part/db],
  // * ':db.install/_attribute' :db.part/db,
  // And default to cardinality: many.
  schemaFragments: {
    ':user/email': {
      ':db/valueType': ':db.type/string',
      ':db/unique': 'db.unique/identity',          // Only one user can have each email.
      ':db/index': true,
      ':db.install/_attribute' :db.part/db,
    },
    ':user/name': {
      ':db/valueType': ':db.type/string',
      ':db/cardinality': ':db.cardinality/one',   // A user can have only one name.
    },
    ':user/bio':
      ':db/valueType': ':db.type/string',
      ':db/cardinality': ':db.cardinality/one',
      ':db/fulltext': true,                       // Bios are searchable in full.
      ':db/noHistory': true,                      // Be explicit that we don't keep old bios.
    },
  },

  async open() {
    const db = await Datomish.open("/path/to/foo.kb", Datomish.ReadWrite | Datomish.Create);

    // All schema evolutions bump a counter. This allows for almost-free schema
    // checking. Pass a callback to be told if the schema changed.
    const dbSchemaVersion = await db.schemaVersion(this.schemaDidChange);

    if (dbSchemaVersion !== this.lastSchemaVersion) {
      // Do a full check.
      // On failure, throws SchemaError, which carries the check result
      // with attributes .ok, .failed, .added, .updated.
      await db.ensureSchemaFragments(this.schemaFragments);

      // Otherwise, we're good. Either we added fragments, or they were already
      // there in some congruent form.
      this.db = db;
    }
  },

  async printNameForEmail(email) {
    const results = await db.q(
      `[:find ?name :in $ :where [
        [?user :user/email '<foo@example.com>']
        [?user :user/name ?name]]]`);
    const result = results.pop();
    if (!result) {
      return;
    }
    console.log(`Name is ${result[0]}.`);
  },