Management of open stores #228

New issue

Open

opened 2020-08-06 16:57:00 +00:00 by gburd · 0 comments

gburd commented

2020-08-06 16:57:00 +00:00

(Migrated from github.com)

It's important that a particular store is only opened once at a time (see also #547).

Further, it's convenient for FFI to have a static place to root open stores.

The obvious solution is a manager, just like rkv's.

But what do we manage?

I propose:

Make Store's conn field Arc. This gives us the ability to share Conns and thus use Store as a pair of a connection and a Conn when there are multiple open SQLite connections.
Add a method to Store, perhaps named fork, that yields a new store with cloned conn using a provided rusqlite::Connection. This allows us to get a new copy of a store that's already open.
Add a static method to Store to create an in-memory store. The empty path will now be confusing. (I believe the FFI already makes a distinction…)
Implement a manager of Stores, just as in rkv. It's important to canonicalize file paths. "Stores" might be a good name.
Expose these methods:
- Is a store open? fn is_open(path: PathBuf) -> bool
- Given a path, give me a mutable reference to a suitable Store, opening it if necessary. fn open(path: PathBuf) -> Result<&mut Store>
- Given a path, give me a (mutable) reference if the store is open. fn {get,get_mut}(path: PathBuf) -> Option<{&mut,&} Store>
- Given a path, give me a Store instance with its own SQLite connection. I understand that I now have to work with SQLite's concurrency model (e.g., I can encounter SQLITE_BUSY), but Mentat will serialize writes to Metadata. fn connect(path: PathBuf) -> Result<Store>
- Close a store by path. This needs to check the reference count on Store.conn and fail if it's more than 1. It can only be more than 1 if a caller has cloned the store, either explicitly or via connect.

The usual pattern for working with a store then changes from:

        let mut store: Store = Store::open("/some/path").expect("opened");
        let mut in_progress = store.begin_transaction().expect("began");
        in_progress.import(fixture_path("cities.schema")).expect("transacted schema");

        let store: &mut Store = Stores::open("/some/path").expect("opened");
        let mut in_progress = store.begin_transaction().expect("began");
        in_progress.import(fixture_path("cities.schema")).expect("transacted schema");

— a one-character change and a change in type.

It's important that a particular store is only opened once at a time (see also #547). Further, it's convenient for FFI to have a static place to root open stores. The obvious solution is a manager, [just like rkv's](https://github.com/mozilla-prototypes/rkv/blob/master/src/manager.rs#L36). But what do we manage? I propose: - Make `Store`'s `conn` field `Arc`. This gives us the ability to share `Conn`s and thus use `Store` as a pair of a connection and a `Conn` when there are multiple open SQLite connections. - Add a method to `Store`, perhaps named `fork`, that yields a new store with cloned `conn` using a provided `rusqlite::Connection`. This allows us to get a new copy of a store that's already open. - Add a static method to `Store` to create an in-memory store. The empty path will now be confusing. (I believe the FFI already makes a distinction…) - Implement a manager of `Store`s, just as in rkv. It's important to canonicalize file paths. "Stores" might be a good name. - Expose these methods: - Is a store open? `fn is_open(path: PathBuf) -> bool` - Given a path, give me a mutable reference to a suitable `Store`, opening it if necessary. `fn open(path: PathBuf) -> Result<&mut Store>` - Given a path, give me a (mutable) reference if the store is open. `fn {get,get_mut}(path: PathBuf) -> Option<{&mut,&} Store>` - Given a path, give me a `Store` instance with its own SQLite connection. I understand that I now have to work with SQLite's concurrency model (_e.g._, I can encounter `SQLITE_BUSY`), but Mentat will serialize writes to `Metadata`. `fn connect(path: PathBuf) -> Result<Store>` - Close a store by path. This needs to check the reference count on `Store.conn` and fail if it's more than 1. It can only be more than 1 if a caller has cloned the store, either explicitly or via `connect`. The usual pattern for working with a store then changes from: ```rust let mut store: Store = Store::open("/some/path").expect("opened"); let mut in_progress = store.begin_transaction().expect("began"); in_progress.import(fixture_path("cities.schema")).expect("transacted schema"); ``` to ```rust let store: &mut Store = Stores::open("/some/path").expect("opened"); let mut in_progress = store.begin_transaction().expect("began"); in_progress.import(fixture_path("cities.schema")).expect("transacted schema"); ``` — a one-character change and a change in type.