greg/mentat
1
0
Fork 0
Code Issues 294 Pull requests Projects Releases 9 Packages Wiki Activity
mentat/edn/src/edn.rustpeg

492 lines
18 KiB
Text
Raw Normal View History

Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/* -*- comment-start: "//"; -*- */
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
/* vim: set filetype=rust.rustpeg */
// Copyright 2016 Mozilla
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not use
// this file except in compliance with the License. You may obtain a copy of the
// License at http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software distributed
// under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.
use std::collections::{BTreeSet, BTreeMap, LinkedList};
use std::iter::FromIterator;
Parse and display EDN values for NaN, +Infinity and -Infinity. Fixes #232 (#238) r=victorporof
2017-02-03 18:14:23 +00:00
use std::f64::{NAN, INFINITY, NEG_INFINITY};
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
use chrono::{
DateTime,
TimeZone,
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
};
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
use num::BigInt;
use ordered_float::OrderedFloat;
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
use uuid::Uuid;
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
use entities::*;
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
use query;
use query::FromValue;
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
use symbols::*;
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
use types::{SpannedValue, Span, ValueAndSpan};
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// Goal: Be able to parse https://github.com/edn-format/edn
// Also extensible to help parse http://docs.datomic.com/query.html
// Debugging hint: test using `cargo test --features peg/trace -- --nocapture`
// to trace where the parser is failing
// TODO: Support tagged elements
// TODO: Support discard
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub nil -> SpannedValue = "nil" { SpannedValue::Nil }
pub nan -> SpannedValue = "#f" whitespace+ "NaN" { SpannedValue::Float(OrderedFloat(NAN)) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub infinity -> SpannedValue = "#f" whitespace+ s:$(sign) "Infinity"
{ SpannedValue::Float(OrderedFloat(if s == "+" { INFINITY } else { NEG_INFINITY })) }
Parse and display EDN values for NaN, +Infinity and -Infinity. Fixes #232 (#238) r=victorporof
2017-02-03 18:14:23 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub boolean -> SpannedValue
= "true" { SpannedValue::Boolean(true) }
/ "false" { SpannedValue::Boolean(false) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
digit = [0-9]
Add octal, hexadecimal, and arbitrary base integers to the EDN parser. Fixes #277. r=rnewman (#286)
2017-02-11 00:03:35 +00:00
alphanumeric = [0-9a-zA-Z]
octaldigit = [0-7]
validbase = [3][0-6] / [12][0-9] / [2-9]
hex = [0-9a-fA-F]
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
sign = [+-]
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub raw_bigint -> BigInt = b:$( sign? digit+ ) "N"
{ b.parse::<BigInt>().unwrap() }
pub raw_octalinteger -> i64 = "0" i:$( octaldigit+ )
{ i64::from_str_radix(i, 8).unwrap() }
pub raw_hexinteger -> i64 = "0x" i:$( hex+ )
{ i64::from_str_radix(i, 16).unwrap() }
pub raw_basedinteger -> i64 = b:$( validbase ) "r" i:$( alphanumeric+ )
{ i64::from_str_radix(i, b.parse::<u32>().unwrap()).unwrap() }
pub raw_integer -> i64 = i:$( sign? digit+ ) !("." / ([eE]))
{ i.parse::<i64>().unwrap() }
pub raw_float -> OrderedFloat<f64> = f:$(sign? digit+ ("." digit+)? ([eE] sign? digit+)?)
{ OrderedFloat(f.parse::<f64>().unwrap()) }
pub bigint -> SpannedValue = v:raw_bigint { SpannedValue::BigInteger(v) }
pub octalinteger -> SpannedValue = v:raw_octalinteger { SpannedValue::Integer(v) }
pub hexinteger -> SpannedValue = v:raw_hexinteger { SpannedValue::Integer(v) }
pub basedinteger -> SpannedValue = v:raw_basedinteger { SpannedValue::Integer(v) }
pub integer -> SpannedValue = v:raw_integer { SpannedValue::Integer(v) }
pub float -> SpannedValue = v:raw_float { SpannedValue::Float(v) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
number -> SpannedValue = ( bigint / basedinteger / hexinteger / octalinteger / integer / float )
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Correctly parse and unescape quotes etc. inside EDN strings. (#434) (#589)
2018-03-15 14:13:27 +00:00
// TODO: standalone characters: \<char>, \newline, \return, \space and \tab.
string_special_char -> &'input str = "\\" $([\\"ntr])
string_normal_chars -> &'input str = $([^"\\]+)
// This is what we need to do in order to unescape. We can't just match the entire string slice:
// we get a Vec<&str> from rust-peg, where some of the parts might be unescaped special characters,
// and we join it together to form an output string.
// E.g., input = r#"\"foo\\\\bar\""#
// output = [quote, "foo", backslash, "bar", quote]
// result = r#""foo\\bar""#
// For the typical case, string_normal_chars will match multiple, leading to a single-element vec.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub raw_text -> String = "\"" t:((string_special_char / string_normal_chars)*) "\""
{ t.join(&"").to_string() }
pub text -> SpannedValue
= v:raw_text { SpannedValue::Text(v) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
// RFC 3339 timestamps. #inst "1985-04-12T23:20:50.52Z"
// We accept an arbitrary depth of decimals.
// Note that we discard the timezone information -- all times are translated to UTC.
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_string -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#inst" whitespace+ "\"" d:$( [0-9]*<4> "-" [0-2][0-9] "-" [0-3][0-9]
"T"
[0-2][0-9] ":" [0-5][0-9] ":" [0-6][0-9]
("." [0-9]+)?
[edn] Round-trip instants. (#686) (#687) r=rnewman First, the parser had a small grouping bug where-by it wouldn't parse Z as timezone correctly. Second, we weren't printing instants in the format that we parse.
2018-05-11 09:11:04 +00:00
("Z" / (("+" / "-") [0-2][0-9] ":" [0-5][0-9]))
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
)
"\"" {?
DateTime::parse_from_rfc3339(d)
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
.map(|t| t.with_timezone(&Utc))
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
.map_err(|_| "invalid datetime") // Oh, rustpeg.
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_micros -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#instmicros" whitespace+ d:$( digit+ ) {
let micros = d.parse::<i64>().unwrap();
let seconds: i64 = micros / 1000000;
let nanos: u32 = ((micros % 1000000).abs() as u32) * 1000;
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc.timestamp(seconds, nanos)
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_millis -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#instmillis" whitespace+ d:$( digit+ ) {
let millis = d.parse::<i64>().unwrap();
let seconds: i64 = millis / 1000;
let nanos: u32 = ((millis % 1000).abs() as u32) * 1000000;
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc.timestamp(seconds, nanos)
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst -> SpannedValue = t:(inst_millis / inst_micros / inst_string)
{ SpannedValue::Instant(t) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
uuid_string -> Uuid =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"\"" u:$( [a-f0-9]*<8> "-" [a-f0-9]*<4> "-" [a-f0-9]*<4> "-" [a-f0-9]*<4> "-" [a-f0-9]*<12> ) "\"" {
Uuid::parse_str(u).expect("this is a valid UUID string")
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub uuid -> SpannedValue = "#uuid" whitespace+ u:uuid_string
{ SpannedValue::Uuid(u) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
namespace_divider = "."
namespace_separator = "/"
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// TODO: Be more picky here
Unify and generalize keywords and symbols parsing Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:52:34 +00:00
// Keywords follow the rules of symbols, except they can (and must) begin with :
// e.g. :fred or :my/fred. See https://github.com/edn-format/edn#keywords
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
symbol_char_initial = [a-zA-Z0-9*!_?$%&=<>]
symbol_char_subsequent = [a-zA-Z0-9*!_?$%&=<>-]
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Unify and generalize keywords and symbols parsing Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:52:34 +00:00
symbol_namespace = symbol_char_initial symbol_char_subsequent* (namespace_divider symbol_char_subsequent+)*
Support a limited set of '.'-prefixed non-keyword symbols. (#352) r=nalexander This commit allows `.` and `...` to parse correctly as `PlainSymbol`. Tests in edn, query-translator, and the top level have been added.
2017-03-06 22:55:14 +00:00
symbol_name = ( symbol_char_initial+ symbol_char_subsequent* )
plain_symbol_name = symbol_name / "..." / "."
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
keyword_prefix = ":"
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub symbol -> SpannedValue =
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
Support a limited set of '.'-prefixed non-keyword symbols. (#352) r=nalexander This commit allows `.` and `...` to parse correctly as `PlainSymbol`. Tests in edn, query-translator, and the top level have been added.
2017-03-06 22:55:14 +00:00
n:$(plain_symbol_name)
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
{ SpannedValue::from_symbol(ns, n) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("symbol")
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub keyword -> SpannedValue =
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
keyword_prefix
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
n:$(symbol_name)
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
{ SpannedValue::from_keyword(ns, n) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("keyword")
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub list -> SpannedValue = "(" __ v:(value)* __ ")"
{ SpannedValue::List(LinkedList::from_iter(v)) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub vector -> SpannedValue = "[" __ v:(value)* __ "]"
{ SpannedValue::Vector(v) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub set -> SpannedValue = "#{" __ v:(value)* __ "}"
{ SpannedValue::Set(BTreeSet::from_iter(v)) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
pair -> (ValueAndSpan, ValueAndSpan) =
Update rustpeg to latest version and follow new syntax and formatting rules Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:36:53 +00:00
k:(value) v:(value) {
(k, v)
}
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub map -> SpannedValue = "{" __ v:(pair)* __ "}"
{ SpannedValue::Map(BTreeMap::from_iter(v)) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// It's important that float comes before integer or the parser assumes that
// floats are integers and fails to parse
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
pub value -> ValueAndSpan =
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
__ start:#position v:(nil / nan / infinity / boolean / number / inst / uuid / text / keyword / symbol / list / vector / map / set) end:#position __ {
ValueAndSpan {
inner: v,
span: Span::new(start, end)
}
Update rustpeg to latest version and follow new syntax and formatting rules Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:36:53 +00:00
}
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("value")
edn: Bound values by optional whitespace; treat comma as whitespace.
2017-01-17 19:26:45 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
atom -> ValueAndSpan
= v:value {? if v.is_atom() { Ok(v) } else { Err("expected atom") } }
edn: Bound values by optional whitespace; treat comma as whitespace.
2017-01-17 19:26:45 +00:00
// Clojure (and thus EDN) regards commas as whitespace, and thus the two-element vectors [1 2] and
// [1,,,,2] are equivalent, as are the maps {:a 1, :b 2} and {:a 1 :b 2}.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
whitespace = #quiet<[ \r\n\t,]>
comment = #quiet<";" [^\r\n]* [\r\n]?>
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
edn: Allow comments. EDN supports only one type of comment: initiated by ; and lasting until the end of the current line or the end of the input stream.
2017-01-17 19:25:31 +00:00
__ = (whitespace / comment)*
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
// Transaction entity parser starts here.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub op -> OpType
= ":db/add" { OpType::Add }
/ ":db/retract" { OpType::Retract }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
raw_keyword -> Keyword =
keyword_prefix
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
n:$(symbol_name) {
match ns {
Some(ns) => Keyword::namespaced(ns, n),
None => Keyword::plain(n),
}
}
/ #expected("keyword")
raw_forward_keyword -> Keyword
= v:raw_keyword {? if v.is_forward() { Ok(v) } else { Err("expected :forward or :forward/keyword") } }
raw_backward_keyword -> Keyword
= v:raw_keyword {? if v.is_backward() { Ok(v) } else { Err("expected :_backword or :backward/_keyword") } }
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_namespaced_keyword -> Keyword
Combine NamespacedKeyword and Keyword. (#689) r=nalexander * Make properties on NamespacedKeyword/NamespacedSymbol private * Use only a single String for NamespacedKeyword/NamespacedSymbol * Review comments. * Remove unsafe code in namespaced_name. Benchmarking shows approximately zero change. * Allow the types of ns and name to differ when constructing a NamespacedName. * Make symbol namespaces optional. * Normalize names of keyword/symbol constructors. This will make the subsequent refactor much less painful. * Use expect not unwrap. * Merge Keyword and NamespacedKeyword.
2018-05-11 16:52:17 +00:00
= keyword_prefix ns:$(symbol_namespace) namespace_separator n:$(symbol_name) { Keyword::namespaced(ns, n) }
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
/ #expected("namespaced keyword")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_forward_namespaced_keyword -> Keyword
= v:raw_namespaced_keyword {? if v.is_forward() { Ok(v) } else { Err("expected namespaced :forward/keyword") } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_backward_namespaced_keyword -> Keyword
= v:raw_namespaced_keyword {? if v.is_backward() { Ok(v) } else { Err("expected namespaced :backward/_keyword") } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
entid -> EntidOrIdent
= v:( raw_basedinteger / raw_hexinteger / raw_octalinteger / raw_integer ) { EntidOrIdent::Entid(v) }
/ v:raw_namespaced_keyword { EntidOrIdent::Ident(v) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
forward_entid -> EntidOrIdent
= v:( raw_basedinteger / raw_hexinteger / raw_octalinteger / raw_integer ) { EntidOrIdent::Entid(v) }
/ v:raw_forward_namespaced_keyword { EntidOrIdent::Ident(v) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("forward entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
backward_entid -> EntidOrIdent
= v:raw_backward_namespaced_keyword { EntidOrIdent::Ident(v.to_reversed()) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("backward entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
lookup_ref -> LookupRef<ValueAndSpan>
= "(" __ "lookup-ref" __ a:(entid) __ v:(value) __ ")" { LookupRef { a: AttributePlace::Entid(a), v } }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("lookup-ref")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
tx_function -> TxFunction
Combine NamespacedKeyword and Keyword. (#689) r=nalexander * Make properties on NamespacedKeyword/NamespacedSymbol private * Use only a single String for NamespacedKeyword/NamespacedSymbol * Review comments. * Remove unsafe code in namespaced_name. Benchmarking shows approximately zero change. * Allow the types of ns and name to differ when constructing a NamespacedName. * Make symbol namespaces optional. * Normalize names of keyword/symbol constructors. This will make the subsequent refactor much less painful. * Use expect not unwrap. * Merge Keyword and NamespacedKeyword.
2018-05-11 16:52:17 +00:00
= "(" __ n:$(symbol_name) __ ")" { TxFunction { op: PlainSymbol::plain(n) } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
entity_place -> EntityPlace<ValueAndSpan>
= v:raw_text { EntityPlace::TempId(TempId::External(v)) }
/ v:entid { EntityPlace::Entid(v) }
/ v:lookup_ref { EntityPlace::LookupRef(v) }
/ v:tx_function { EntityPlace::TxFunction(v) }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
value_place_pair -> (EntidOrIdent, ValuePlace<ValueAndSpan>)
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= k:(entid) __ v:(value_place) { (k, v) }
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
map_notation -> MapNotation<ValueAndSpan>
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= "{" __ kvs:(value_place_pair*) __ "}" { kvs.into_iter().collect() }
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
value_place -> ValuePlace<ValueAndSpan>
= __ v:lookup_ref __ { ValuePlace::LookupRef(v) }
/ __ v:tx_function __ { ValuePlace::TxFunction(v) }
/ __ "[" __ vs:(value_place*) __ "]" __ { ValuePlace::Vector(vs) }
/ __ v:map_notation __ { ValuePlace::MapNotation(v) }
/ __ v:atom __ { ValuePlace::Atom(v) }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
pub entity -> Entity<ValueAndSpan>
= __ "[" __ op:(op) __ e:(entity_place) __ a:(forward_entid) __ v:(value_place) __ "]" __ { Entity::AddOrRetract { op, e: e, a: AttributePlace::Entid(a), v: v } }
/ __ "[" __ op:(op) __ e:(value_place) __ a:(backward_entid) __ v:(entity_place) __ "]" __ { Entity::AddOrRetract { op, e: v, a: AttributePlace::Entid(a), v: e } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
/ __ map:map_notation __ { Entity::MapNotation(map) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("entity")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
pub entities -> Vec<Entity<ValueAndSpan>>
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= __ "[" __ es:(entity*) __ "]" __ { es }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
// Query parser starts here.
//
// We expect every rule except the `raw_*` rules to eat whitespace
// (with `__`) at its start and finish. That means that every string
// pattern (say "[") should be bracketed on either side with either a
// whitespace-eating rule or an explicit whitespace eating `__`.
query_function -> query::QueryFunction
= __ n:$(symbol_name) __ {? query::QueryFunction::from_symbol(&PlainSymbol::plain(n)).ok_or("expected query function") }
fn_arg -> query::FnArg
= v:value {? query::FnArg::from_value(&v).ok_or("expected query function argument") }
/ __ "[" args:fn_arg+ "]" __ { query::FnArg::Vector(args) }
find_elem -> query::Element
= __ v:variable __ { query::Element::Variable(v) }
/ __ "(" __ "the" v:variable ")" __ { query::Element::Corresponding(v) }
/ __ "(" __ "pull" var:variable "[" patterns:pull_attribute+ "]" __ ")" __ { query::Element::Pull(query::Pull { var, patterns }) }
/ __ "(" func:query_function args:fn_arg* ")" __ { query::Element::Aggregate(query::Aggregate { func, args }) }
find_spec -> query::FindSpec
= f:find_elem "." __ { query::FindSpec::FindScalar(f) }
/ fs:find_elem+ { query::FindSpec::FindRel(fs) }
/ __ "[" f:find_elem __ "..." __ "]" __ { query::FindSpec::FindColl(f) }
/ __ "[" fs:find_elem+ "]" __ { query::FindSpec::FindTuple(fs) }
pull_attribute -> query::PullAttributeSpec
= __ "*" __ { query::PullAttributeSpec::Wildcard }
/ __ k:raw_forward_namespaced_keyword __ alias:(":as" __ alias:raw_forward_keyword __ { alias })? {
let attribute = query::PullConcreteAttribute::Ident(::std::rc::Rc::new(k));
let alias = alias.map(|alias| ::std::rc::Rc::new(alias));
query::PullAttributeSpec::Attribute(
query::NamedPullAttribute {
attribute,
alias: alias,
})
}
limit -> query::Limit
= __ v:variable __ { query::Limit::Variable(v) }
/ __ n:(raw_octalinteger / raw_hexinteger / raw_basedinteger / raw_integer) __ {?
if n > 0 {
Ok(query::Limit::Fixed(n as u64))
} else {
Err("expected positive integer")
}
}
order -> query::Order
= __ "(" __ "asc" v:variable ")" __ { query::Order(query::Direction::Ascending, v) }
/ __ "(" __ "desc" v:variable ")" __ { query::Order(query::Direction::Descending, v) }
/ v:variable { query::Order(query::Direction::Ascending, v) }
pattern_value_place -> query::PatternValuePlace
= v:value {? query::PatternValuePlace::from_value(&v).ok_or("expected pattern_value_place") }
pattern_non_value_place -> query::PatternNonValuePlace
= v:value {? query::PatternNonValuePlace::from_value(&v).ok_or("expected pattern_non_value_place") }
pattern -> query::WhereClause
= __ "["
src:src_var?
e:pattern_non_value_place
a:pattern_non_value_place
v:pattern_value_place?
tx:pattern_non_value_place?
"]" __
{?
let v = v.unwrap_or(query::PatternValuePlace::Placeholder);
let tx = tx.unwrap_or(query::PatternNonValuePlace::Placeholder);
// Pattern::new takes care of reversal of reversed
// attributes: [?x :foo/_bar ?y] turns into
// [?y :foo/bar ?x].
//
// This is a bit messy: the inner conversion to a Pattern can
// fail if the input is something like
//
// ```edn
// [?x :foo/_reversed 23.4]
// ```
//
// because
//
// ```edn
// [23.4 :foo/reversed ?x]
// ```
//
// is nonsense. That leaves us with a nested optional, which we unwrap here.
query::Pattern::new(src, e, a, v, tx)
.map(query::WhereClause::Pattern)
.ok_or("expected pattern")
}
// TODO: this shouldn't be checked at parse time.
rule_vars -> BTreeSet<query::Variable>
= vs:variable+ {?
let given = vs.len();
let set: BTreeSet<query::Variable> = vs.into_iter().collect();
if given != set.len() {
Err("expected unique variables")
} else {
Ok(set)
}
}
or_pattern_clause -> query::OrWhereClause
= clause:where_clause { query::OrWhereClause::Clause(clause) }
or_and_clause -> query::OrWhereClause
= __ "(" __ "and" clauses:where_clause+ ")" __ { query::OrWhereClause::And(clauses) }
or_where_clause -> query::OrWhereClause
= or_pattern_clause
/ or_and_clause
or_clause -> query::WhereClause
= __ "(" __ "or" clauses:or_where_clause+ ")" __ {
query::WhereClause::OrJoin(query::OrJoin::new(query::UnifyVars::Implicit, clauses))
}
or_join_clause -> query::WhereClause
= __ "(" __ "or-join" __ "[" vars:rule_vars "]" clauses:or_where_clause+ ")" __ {
query::WhereClause::OrJoin(query::OrJoin::new(query::UnifyVars::Explicit(vars), clauses))
}
not_clause -> query::WhereClause
= __ "(" __ "not" clauses:where_clause+ ")" __ {
query::WhereClause::NotJoin(query::NotJoin::new(query::UnifyVars::Implicit, clauses))
}
not_join_clause -> query::WhereClause
= __ "(" __ "not-join" __ "[" vars:rule_vars "]" clauses:where_clause+ ")" __ {
query::WhereClause::NotJoin(query::NotJoin::new(query::UnifyVars::Explicit(vars), clauses))
}
type_annotation -> query::WhereClause
= __ "[" __ "(" __ "type" var:variable __ ty:raw_keyword __ ")" __ "]" __ {
query::WhereClause::TypeAnnotation(
query::TypeAnnotation {
value_type: ty,
variable: var,
})
}
pred -> query::WhereClause
= __ "[" __ "(" func:query_function args:fn_arg* ")" __ "]" __ {
query::WhereClause::Pred(
query::Predicate {
operator: func.0,
args: args,
})
}
pub where_fn -> query::WhereClause
= __ "[" __ "(" func:query_function args:fn_arg* ")" __ binding:binding "]" __ {
query::WhereClause::WhereFn(
query::WhereFn {
operator: func.0,
args: args,
binding,
})
}
where_clause -> query::WhereClause
// Right now we only support patterns and predicates. See #239 for more.
= pattern
/ or_join_clause
/ or_clause
/ not_join_clause
/ not_clause
/ type_annotation
/ pred
/ where_fn
query_part -> query::QueryPart
= __ ":find" fs:find_spec { query::QueryPart::FindSpec(fs) }
/ __ ":in" in_vars:variable+ { query::QueryPart::InVars(in_vars) }
/ __ ":limit" l:limit { query::QueryPart::Limit(l) }
/ __ ":order" os:order+ { query::QueryPart::Order(os) }
/ __ ":where" ws:where_clause+ { query::QueryPart::WhereClauses(ws) }
/ __ ":with" with_vars:variable+ { query::QueryPart::WithVars(with_vars) }
Review comments.
2018-06-04 22:21:27 +00:00
pub parse_query -> query::ParsedQuery
= __ "[" qps:query_part+ "]" __ {? query::ParsedQuery::from_parts(qps) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
variable -> query::Variable
= v:value {? query::Variable::from_value(&v).ok_or("expected variable") }
src_var -> query::SrcVar
= v:value {? query::SrcVar::from_value(&v).ok_or("expected src_var") }
variable_or_placeholder -> query::VariableOrPlaceholder
= v:variable { query::VariableOrPlaceholder::Variable(v) }
/ __ "_" __ { query::VariableOrPlaceholder::Placeholder }
binding -> query::Binding
= __ "[" __ "[" vs:variable_or_placeholder+ "]" __ "]" __ { query::Binding::BindRel(vs) }
/ __ "[" v:variable "..." __ "]" __ { query::Binding::BindColl(v) }
/ __ "[" vs:variable_or_placeholder+ "]" __ { query::Binding::BindTuple(vs) }
/ v:variable { query::Binding::BindScalar(v) }
Reference in a new issue Copy permalink