greg/mentat
1
0
Fork 0
Code Issues 294 Pull requests Projects Releases 9 Packages Wiki Activity
mentat/project.clj

136 lines
5.4 KiB
Clojure
Raw Normal View History

Use correct project name in defproject. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-08 19:49:55 +00:00
(defproject datomish "0.1.0-SNAPSHOT"
Stub out Leiningen and npm project files. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-05 19:05:49 +00:00
:description "A persistent, embedded knowledge base inspired by Datomic and DataScript."
:url "https://github.com/mozilla/datomish"
:license {:name "Mozilla Public License Version 2.0"
Add lein-doo to run ClojureScript tests.
2016-07-06 01:55:55 +00:00
:url "https://github.com/mozilla/datomish/blob/master/LICENSE"}
Pre: bump to the latest ClojureScript release.
2016-09-08 00:03:03 +00:00
:dependencies [[org.clojure/clojurescript "1.9.229"]
Add an async and async testing framework. This is a well-worn idea: use a `promise-channel` of `[result nil]` or `[nil error]` pairs. The `go-pair` and `<?` macros handle catching exceptions (important, given that synchronous CLJ code expects to throw rather than return an error promise or similar), allowing code like: ``` (go-pair (let [result (<? (pair-chan-fn))] (when (not result) (throw (Exception. "No result!"))) (transform result))) ``` to be expressed naturally. These are the equivalents of `async` and `await` in JS. The implementation is complicated by significant incompatibilities between CLJ and CLJS. The solution presented here takes care to separate the macro definitions into CLJ. Sadly, this requires namespacing the per-environment symbols explicitly; but we hope to minimize such code in files like this. The most significant restriction to this approach is that consumers must require the transitive dependencies of the macro-defining modules. See the included tests (both CLJ and CLJS) for the appropriate incantations (for pair-chan, core.async, and test).
2016-07-11 23:48:40 +00:00
[org.clojure/clojure "1.8.0"]
[org.clojure/core.async "0.2.385"]
Take a dependency on DataScript. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-06 23:53:08 +00:00
[datascript "0.15.1"]
Bump honeysql to 0.8.0, which includes mainline ClojureScript support.
2016-08-05 20:06:42 +00:00
[honeysql "0.8.0"]
Completely rewrite main transaction logic to be faster. This is almost complete; it passes the test suite save for retracting fulltext datoms correctly. There's a lot to say about this approach, but I don't have time to give too many details. The broad outline is as follows. We collect datoms to add and retract in a tx_lookup table. Depending on flags ("search value" sv and "search value type tag" svalue_type_tag) we "complete" the tx_lookup table by joining matching datoms. This allows us to find datoms that are present (and should not be added as part of the transaction, or should be retracted as part of the transaction, or should be replaced as part of the transaction. We complete the tx_lookup (in place!) in two separate INSERTs to avoid a quadratic two-table walk (explain the queries to observe that both INSERTs walk the lookup table once and then use the datoms indexes to complete the matching values). We could simplify the code by using multiple lookup tables, both for the two cases of search parameters (eav vs. ea) and for the incomplete and completed rows. Right now we differentiate the former with NULL checks, and the latter by incrementing the added0 column. It performs well enough, so I haven't tried to understand the performance of separating these things. After the tx_lookup table is completed, we build the transaction from it; and update the datoms materialized view table as well. Observe the careful handling of the "search value" sv parameters to handle replacing :db.cardinality/one datoms. Finally, we read the processed transaction back to produce to the API. This is strictly to match the Datomic API; we might make allow to skip this, since many consumers will not want to stream this over the wire. Rough timings show the transactor processing a single >50k datom transaction in about 3.5s, of which less than 0.5s is spent in the expensive joins. Further, repeating the processing of the same transaction is only about 3.5s again! That's the worst possible for the joins, since every single inserted datom will already be present in the database, making the most expensive join match every row.
2016-08-09 06:30:45 +00:00
[com.taoensso/tufte "1.0.2"]
Add an async and async testing framework. This is a well-worn idea: use a `promise-channel` of `[result nil]` or `[nil error]` pairs. The `go-pair` and `<?` macros handle catching exceptions (important, given that synchronous CLJ code expects to throw rather than return an error promise or similar), allowing code like: ``` (go-pair (let [result (<? (pair-chan-fn))] (when (not result) (throw (Exception. "No result!"))) (transform result))) ``` to be expressed naturally. These are the equivalents of `async` and `await` in JS. The implementation is complicated by significant incompatibilities between CLJ and CLJS. The solution presented here takes care to separate the macro definitions into CLJ. Sadly, this requires namespacing the per-environment symbols explicitly; but we hope to minimize such code in files like this. The most significant restriction to this approach is that consumers must require the transitive dependencies of the macro-defining modules. See the included tests (both CLJ and CLJS) for the appropriate incantations (for pair-chan, core.async, and test).
2016-07-11 23:48:40 +00:00
[jamesmacaulay/cljs-promises "0.1.0"]]
Add lein-cljsbuild and adapt datascript's release-js vehicle.
2016-07-06 01:13:29 +00:00
Split release-js into release-node and release-browser. Also expand our clean targets, and rewrite our build targets to generate different code for Node and Firefox.
2016-09-08 00:00:47 +00:00
:cljsbuild {:builds
{
:release-node
{
:source-paths ["src-node" "src"]
:assert false
:compiler
{
:elide-asserts true
:hashbang false
:language-in :ecmascript5
:language-out :ecmascript5
:optimizations :advanced
:output-dir "release-node"
:output-to "release-node/datomish.bare.js"
:output-wrapper false
:parallel-build true
:pretty-print false
:target :nodejs
}
:notify-command ["release-node/wrap_bare.sh"]}
:release-browser
;; Release builds for use in Firefox must:
;; * Use :optimizations > :none, so that a single file is generated
;; without a need to import Closure's own libs.
;; * Be wrapped, so that a CommonJS module is produced.
;; * Have a preload script that defines what `println` does.
;;
;; There's no point in generating a source map -- it'll be wrong
;; due to wrapping.
{
:source-paths ["src-browser" "src"]
:assert false
:compiler
{
:elide-asserts true
:language-in :ecmascript5
:language-out :ecmascript5
:optimizations :advanced
:output-dir "release-browser"
:output-to "release-browser/datomish.bare.js"
:output-wrapper false
:parallel-build true
:preloads [datomish.preload]
:pretty-print true
:pseudo-names true
:static-fns true
}
:notify-command ["release-browser/wrap_bare.sh"]}
:advanced
{:source-paths ["src-node" "src"]
:compiler
{
:language-in :ecmascript5
:language-out :ecmascript5
:output-dir "target/advanced"
:output-to "target/advanced/datomish.js"
:optimizations :advanced
:parallel-build true
:pretty-print true
:source-map "target/advanced/datomish.js.map"
:target :nodejs
}}
:test
{
:source-paths ["src-node" "src" "test"]
:compiler
{
:language-in :ecmascript5
:language-out :ecmascript5
:main datomish.test
:optimizations :none
:output-dir "target/test"
:output-to "target/test/datomish.js"
:parallel-build true
:source-map true
:target :nodejs
}}
}}
Add lein-cljsbuild and adapt datascript's release-js vehicle.
2016-07-06 01:13:29 +00:00
Use cljsbuild 1.1.3. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-08 19:44:57 +00:00
:profiles {:dev {:dependencies [[cljsbuild "1.1.3"]
Add an SQLite connection abstraction.
2016-07-12 05:02:53 +00:00
[tempfile "0.2.0"]
Use cljsbuild 1.1.3. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-08 19:44:57 +00:00
[com.cemerick/piggieback "0.2.1"]
[org.clojure/tools.nrepl "0.2.10"]
Add an SQLite connection abstraction.
2016-07-12 05:02:53 +00:00
[org.clojure/java.jdbc "0.6.2-alpha1"]
[org.xerial/sqlite-jdbc "3.8.11.2"]]
Bump JVM stack size to 4MB and eliminate tiered compilation. This allows CLJS compilation to complete for complex go-pair forms. See <https://github.com/emezeske/lein-cljsbuild/issues/303> for more details.
2016-08-19 16:20:50 +00:00
:jvm-opts ["-Xss4m"]
Add lein-cljsbuild and adapt datascript's release-js vehicle.
2016-07-06 01:13:29 +00:00
:repl-options {:nrepl-middleware [cemerick.piggieback/wrap-cljs-repl]}
Use cljsbuild 1.1.3. Signed-off-by: Richard Newman <rnewman@twinql.com>
2016-07-08 19:44:57 +00:00
:plugins [[lein-cljsbuild "1.1.3"]
Add some Leiningen plugins to make testing prettier.
2016-08-26 00:23:25 +00:00
[lein-doo "0.1.6"]
[venantius/ultra "0.4.1"]
[com.jakemccrary/lein-test-refresh "0.16.0"]]
Add lein-doo to run ClojureScript tests.
2016-07-06 01:55:55 +00:00
}}
:doo {:build "test"}
Add lein-cljsbuild and adapt datascript's release-js vehicle.
2016-07-06 01:13:29 +00:00
Split release-js into release-node and release-browser. Also expand our clean targets, and rewrite our build targets to generate different code for Node and Firefox.
2016-09-08 00:00:47 +00:00
:clean-targets ^{:protect false}
[
"target"
"release-node/cljs/"
"release-node/cljs_promises/"
"release-node/clojure/"
"release-node/datascript/"
"release-node/datomish/"
"release-node/honeysql/"
"release-node/taoensso/"
"release-node/datomish.bare.js"
"release-node/datomish.js"
"release-browser/cljs/"
"release-browser/cljs_promises/"
"release-browser/clojure/"
"release-browser/datascript/"
"release-browser/datomish/"
"release-browser/honeysql/"
"release-browser/taoensso/"
"release-browser/datomish.bare.js"
"release-browser/datomish.js"
]
Add lein-cljsbuild and adapt datascript's release-js vehicle.
2016-07-06 01:13:29 +00:00
)
Reference in a new issue Copy permalink