Optimize bulk id-literal insert: don't insert :upserts, don't check for existence of :allocations-* #85

New issue

Open

opened 2020-08-06 16:54:39 +00:00 by gburd · 0 comments

gburd commented

2020-08-06 16:54:39 +00:00

(Migrated from github.com)

This is follow-up to #88. There, I note:

As a future optimization, :upserts do not need to be inserted; they upserted, so they already exist in the DB. (We still need to verify uniqueness and ensure no overlapping can occur.) Similarly, :allocations-* do not need to be checked for existence, so they can be written to the DB faster.

This ticket tracks implementing those optimizations, which should be a nice win when importing (or re-importing) large Places databases (see https://github.com/mozilla/tofino/issues/514).

This is follow-up to #88. There, I note: > As a future optimization, :upserts do not need to be inserted; they upserted, so they already exist in the DB. (We still need to verify uniqueness and ensure no overlapping can occur.) Similarly, :allocations-\* do not need to be checked for existence, so they can be written to the DB faster. This ticket tracks implementing those optimizations, which should be a nice win when importing (or re-importing) large Places databases (see https://github.com/mozilla/tofino/issues/514).