[db] Don't collect as many intermediate data structures when looking up [a v] pairs #117

New issue

Open

opened 2020-08-06 16:55:11 +00:00 by gburd · 0 comments

gburd commented

2020-08-06 16:55:11 +00:00

(Migrated from github.com)

This is follow-up to https://github.com/mozilla/mentat/pull/214#discussion_r99386958.

What's happening is that we have a map [a v] -> e expressed as a composition of maps:

[a v] -> temporary search id
temporary search id -> e entid from database

The latter map must be chunked into a number of sub-queries because SQLite allows to bind only a limited number of variables.

The reviewer correctly points out that we don't need to collect as many intermediate data structures. We could do this as iteration and a mutable map, although I think I prefer a big fold. A "fold-in-place" is best expressed as a for loop in Rust (per the documentation), which is what the reviewer asks for.

This ticket isn't quite ready, because the [a v] lookup code is likely to change a little with #184.

This is follow-up to https://github.com/mozilla/mentat/pull/214#discussion_r99386958. What's happening is that we have a map `[a v] -> e` expressed as a composition of maps: - `[a v]` -> temporary search id - temporary search id -> `e` entid from database The latter map must be chunked into a number of sub-queries because SQLite allows to bind only a limited number of variables. The reviewer correctly points out that we don't need to collect as many intermediate data structures. We could do this as iteration and a mutable map, although I think I prefer a big `fold`. A "fold-in-place" is best expressed as a `for` loop in Rust (per the documentation), which is what the reviewer asks for. This ticket isn't quite ready, because the [a v] lookup code is likely to change a little with #184.