greg/mentat
1
0
Fork 0
Code Issues 294 Pull requests Projects Releases 9 Packages Wiki Activity
mentat/edn/src/edn.rustpeg

492 lines
18 KiB
Text
Raw Normal View History

Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/* -*- comment-start: "//"; -*- */
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
/* vim: set filetype=rust.rustpeg */
// Copyright 2016 Mozilla
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not use
// this file except in compliance with the License. You may obtain a copy of the
// License at http://www.apache.org/licenses/LICENSE-2.0
// Unless required by applicable law or agreed to in writing, software distributed
// under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
// CONDITIONS OF ANY KIND, either express or implied. See the License for the
// specific language governing permissions and limitations under the License.
use std::collections::{BTreeSet, BTreeMap, LinkedList};
use std::iter::FromIterator;
Parse and display EDN values for NaN, +Infinity and -Infinity. Fixes #232 (#238) r=victorporof
2017-02-03 18:14:23 +00:00
use std::f64::{NAN, INFINITY, NEG_INFINITY};
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
use chrono::{
DateTime,
TimeZone,
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
};
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
use num::BigInt;
use ordered_float::OrderedFloat;
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
use uuid::Uuid;
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
use entities::*;
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
use query;
use query::FromValue;
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
use symbols::*;
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
use types::{SpannedValue, Span, ValueAndSpan};
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// Goal: Be able to parse https://github.com/edn-format/edn
// Also extensible to help parse http://docs.datomic.com/query.html
// Debugging hint: test using `cargo test --features peg/trace -- --nocapture`
// to trace where the parser is failing
// TODO: Support tagged elements
// TODO: Support discard
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub nil -> SpannedValue = "nil" { SpannedValue::Nil }
pub nan -> SpannedValue = "#f" whitespace+ "NaN" { SpannedValue::Float(OrderedFloat(NAN)) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub infinity -> SpannedValue = "#f" whitespace+ s:$(sign) "Infinity"
{ SpannedValue::Float(OrderedFloat(if s == "+" { INFINITY } else { NEG_INFINITY })) }
Parse and display EDN values for NaN, +Infinity and -Infinity. Fixes #232 (#238) r=victorporof
2017-02-03 18:14:23 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub boolean -> SpannedValue
= "true" { SpannedValue::Boolean(true) }
/ "false" { SpannedValue::Boolean(false) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
digit = [0-9]
Add octal, hexadecimal, and arbitrary base integers to the EDN parser. Fixes #277. r=rnewman (#286)
2017-02-11 00:03:35 +00:00
alphanumeric = [0-9a-zA-Z]
octaldigit = [0-7]
validbase = [3][0-6] / [12][0-9] / [2-9]
hex = [0-9a-fA-F]
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
sign = [+-]
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub raw_bigint -> BigInt = b:$( sign? digit+ ) "N"
{ b.parse::<BigInt>().unwrap() }
pub raw_octalinteger -> i64 = "0" i:$( octaldigit+ )
{ i64::from_str_radix(i, 8).unwrap() }
pub raw_hexinteger -> i64 = "0x" i:$( hex+ )
{ i64::from_str_radix(i, 16).unwrap() }
pub raw_basedinteger -> i64 = b:$( validbase ) "r" i:$( alphanumeric+ )
{ i64::from_str_radix(i, b.parse::<u32>().unwrap()).unwrap() }
pub raw_integer -> i64 = i:$( sign? digit+ ) !("." / ([eE]))
{ i.parse::<i64>().unwrap() }
pub raw_float -> OrderedFloat<f64> = f:$(sign? digit+ ("." digit+)? ([eE] sign? digit+)?)
{ OrderedFloat(f.parse::<f64>().unwrap()) }
pub bigint -> SpannedValue = v:raw_bigint { SpannedValue::BigInteger(v) }
pub octalinteger -> SpannedValue = v:raw_octalinteger { SpannedValue::Integer(v) }
pub hexinteger -> SpannedValue = v:raw_hexinteger { SpannedValue::Integer(v) }
pub basedinteger -> SpannedValue = v:raw_basedinteger { SpannedValue::Integer(v) }
pub integer -> SpannedValue = v:raw_integer { SpannedValue::Integer(v) }
pub float -> SpannedValue = v:raw_float { SpannedValue::Float(v) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
number -> SpannedValue = ( bigint / basedinteger / hexinteger / octalinteger / integer / float )
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Correctly parse and unescape quotes etc. inside EDN strings. (#434) (#589)
2018-03-15 14:13:27 +00:00
// TODO: standalone characters: \<char>, \newline, \return, \space and \tab.
string_special_char -> &'input str = "\\" $([\\"ntr])
string_normal_chars -> &'input str = $([^"\\]+)
// This is what we need to do in order to unescape. We can't just match the entire string slice:
// we get a Vec<&str> from rust-peg, where some of the parts might be unescaped special characters,
// and we join it together to form an output string.
// E.g., input = r#"\"foo\\\\bar\""#
// output = [quote, "foo", backslash, "bar", quote]
// result = r#""foo\\bar""#
// For the typical case, string_normal_chars will match multiple, leading to a single-element vec.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub raw_text -> String = "\"" t:((string_special_char / string_normal_chars)*) "\""
{ t.join(&"").to_string() }
pub text -> SpannedValue
= v:raw_text { SpannedValue::Text(v) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
// RFC 3339 timestamps. #inst "1985-04-12T23:20:50.52Z"
// We accept an arbitrary depth of decimals.
// Note that we discard the timezone information -- all times are translated to UTC.
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_string -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#inst" whitespace+ "\"" d:$( [0-9]*<4> "-" [0-2][0-9] "-" [0-3][0-9]
"T"
[0-2][0-9] ":" [0-5][0-9] ":" [0-6][0-9]
("." [0-9]+)?
[edn] Round-trip instants. (#686) (#687) r=rnewman First, the parser had a small grouping bug where-by it wouldn't parse Z as timezone correctly. Second, we weren't printing instants in the format that we parse.
2018-05-11 09:11:04 +00:00
("Z" / (("+" / "-") [0-2][0-9] ":" [0-5][0-9]))
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
)
"\"" {?
DateTime::parse_from_rfc3339(d)
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
.map(|t| t.with_timezone(&Utc))
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
.map_err(|_| "invalid datetime") // Oh, rustpeg.
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_micros -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#instmicros" whitespace+ d:$( digit+ ) {
let micros = d.parse::<i64>().unwrap();
let seconds: i64 = micros / 1000000;
let nanos: u32 = ((micros % 1000000).abs() as u32) * 1000;
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc.timestamp(seconds, nanos)
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst_millis -> DateTime<Utc> =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"#instmillis" whitespace+ d:$( digit+ ) {
let millis = d.parse::<i64>().unwrap();
let seconds: i64 = millis / 1000;
let nanos: u32 = ((millis % 1000).abs() as u32) * 1000000;
Update some dependencies. (#492) r=etoop * Update some dependencies. * Update rusqlite to 0.12. * Update error-chain to a forked version that implements Sync. * Fix some compiler warnings. * Remove unused imports in tests. * Parse errors no longer naturally print with the expected symbol.
2017-11-21 16:24:08 +00:00
Utc.timestamp(seconds, nanos)
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
inst -> SpannedValue = t:(inst_millis / inst_micros / inst_string)
{ SpannedValue::Instant(t) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
uuid_string -> Uuid =
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
"\"" u:$( [a-f0-9]*<8> "-" [a-f0-9]*<4> "-" [a-f0-9]*<4> "-" [a-f0-9]*<4> "-" [a-f0-9]*<12> ) "\"" {
Uuid::parse_str(u).expect("this is a valid UUID string")
}
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub uuid -> SpannedValue = "#uuid" whitespace+ u:uuid_string
{ SpannedValue::Uuid(u) }
UUIDs and instants. Fixes #44, #45, #426, #427. (#438) r=nalexander * Pre: unused import in translate.rs. * Part 2: take a dependency on rusqlite for query arguments. * Part 1: flatten V2 schema into V1. Add UUID and URI. Bump expected ident and bootstrap datom count in tests. * Part 5: parse edn::Value::Uuid. * Part 3: extend ValueType and TypedValue to include Uuid. * Part 4: add Uuid to query arguments. * Part 6: extend db to support Uuid. * Part 8: add a tx-parser test for #f NaN and #uuid. * Part 7: parse and algebrize UUIDs in queries. * Part 1: parse #inst in EDN and throughout query engine. * Part 3: handle instants in db. * Part 2: instants never matches integers in queries. * Part 4: use DateTime for tx_instants. * Add a test for adding and querying UUIDs and instants. * Review comments.
2017-04-29 03:11:55 +00:00
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
namespace_divider = "."
namespace_separator = "/"
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// TODO: Be more picky here
Unify and generalize keywords and symbols parsing Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:52:34 +00:00
// Keywords follow the rules of symbols, except they can (and must) begin with :
// e.g. :fred or :my/fred. See https://github.com/edn-format/edn#keywords
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
symbol_char_initial = [a-zA-Z0-9*!_?$%&=<>]
symbol_char_subsequent = [a-zA-Z0-9*!_?$%&=<>-]
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Unify and generalize keywords and symbols parsing Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:52:34 +00:00
symbol_namespace = symbol_char_initial symbol_char_subsequent* (namespace_divider symbol_char_subsequent+)*
Support a limited set of '.'-prefixed non-keyword symbols. (#352) r=nalexander This commit allows `.` and `...` to parse correctly as `PlainSymbol`. Tests in edn, query-translator, and the top level have been added.
2017-03-06 22:55:14 +00:00
symbol_name = ( symbol_char_initial+ symbol_char_subsequent* )
plain_symbol_name = symbol_name / "..." / "."
Read EDN keywords and symbols as rich types. Fixes #154. r=nalexander
2017-01-11 21:51:34 +00:00
keyword_prefix = ":"
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub symbol -> SpannedValue =
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
Support a limited set of '.'-prefixed non-keyword symbols. (#352) r=nalexander This commit allows `.` and `...` to parse correctly as `PlainSymbol`. Tests in edn, query-translator, and the top level have been added.
2017-03-06 22:55:14 +00:00
n:$(plain_symbol_name)
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
{ SpannedValue::from_symbol(ns, n) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("symbol")
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub keyword -> SpannedValue =
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
keyword_prefix
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
n:$(symbol_name)
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
{ SpannedValue::from_keyword(ns, n) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("keyword")
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub list -> SpannedValue = "(" __ v:(value)* __ ")"
{ SpannedValue::List(LinkedList::from_iter(v)) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub vector -> SpannedValue = "[" __ v:(value)* __ "]"
{ SpannedValue::Vector(v) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub set -> SpannedValue = "#{" __ v:(value)* __ "}"
{ SpannedValue::Set(BTreeSet::from_iter(v)) }
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
pair -> (ValueAndSpan, ValueAndSpan) =
Update rustpeg to latest version and follow new syntax and formatting rules Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:36:53 +00:00
k:(value) v:(value) {
(k, v)
}
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
pub map -> SpannedValue = "{" __ v:(pair)* __ "}"
{ SpannedValue::Map(BTreeMap::from_iter(v)) }
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
// It's important that float comes before integer or the parser assumes that
// floats are integers and fails to parse
Add a span component to edn::Value, r=ncalexan Signed-off-by: Victor Porof <victor.porof@gmail.com>
2017-02-14 15:43:32 +00:00
pub value -> ValueAndSpan =
Speed up EDN parser (fixes #445) (#581) r=nalexander Fixes from @kevinmehall. * Prefer character sets over backtracking in the EDN parser. * Avoid duplicate effort when parsing floats in the EDN parser. * Clean up duplicate position tracking code. This turns out to have little performance impact, but makes the grammar much cleaner. * Fix EDN work to pass tests with correct numeric precedence.
2018-03-06 04:33:51 +00:00
__ start:#position v:(nil / nan / infinity / boolean / number / inst / uuid / text / keyword / symbol / list / vector / map / set) end:#position __ {
ValueAndSpan {
inner: v,
span: Span::new(start, end)
}
Update rustpeg to latest version and follow new syntax and formatting rules Signed-off-by: Victor Porof <vporof@mozilla.com>
2017-02-02 10:36:53 +00:00
}
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("value")
edn: Bound values by optional whitespace; treat comma as whitespace.
2017-01-17 19:26:45 +00:00
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
atom -> ValueAndSpan
= v:value {? if v.is_atom() { Ok(v) } else { Err("expected atom") } }
edn: Bound values by optional whitespace; treat comma as whitespace.
2017-01-17 19:26:45 +00:00
// Clojure (and thus EDN) regards commas as whitespace, and thus the two-element vectors [1 2] and
// [1,,,,2] are equivalent, as are the maps {:a 1, :b 2} and {:a 1 :b 2}.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
whitespace = #quiet<[ \r\n\t,]>
comment = #quiet<";" [^\r\n]* [\r\n]?>
Implement a basic EDN parser. (#149) r=rnewman,bgrins,nalexander The parser mostly works and has a decent test suite. It parses all the queries issued by the Tofino UAS, with some caveats. Known flaws: * No support for tagged elements, comments, discarded elements or "'". * Incomplete support for escaped characters in strings and the range of characters that are allowed in keywords and symbols. * Possible whitespace handling problems.
2017-01-06 15:15:57 +00:00
edn: Allow comments. EDN supports only one type of comment: initiated by ; and lasting until the end of the current line or the end of the input stream.
2017-01-17 19:25:31 +00:00
__ = (whitespace / comment)*
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
// Transaction entity parser starts here.
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
pub op -> OpType
= ":db/add" { OpType::Add }
/ ":db/retract" { OpType::Retract }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
raw_keyword -> Keyword =
keyword_prefix
ns:( sns:$(symbol_namespace) namespace_separator { sns })?
n:$(symbol_name) {
match ns {
Some(ns) => Keyword::namespaced(ns, n),
None => Keyword::plain(n),
}
}
/ #expected("keyword")
raw_forward_keyword -> Keyword
= v:raw_keyword {? if v.is_forward() { Ok(v) } else { Err("expected :forward or :forward/keyword") } }
raw_backward_keyword -> Keyword
= v:raw_keyword {? if v.is_backward() { Ok(v) } else { Err("expected :_backword or :backward/_keyword") } }
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_namespaced_keyword -> Keyword
Combine NamespacedKeyword and Keyword. (#689) r=nalexander * Make properties on NamespacedKeyword/NamespacedSymbol private * Use only a single String for NamespacedKeyword/NamespacedSymbol * Review comments. * Remove unsafe code in namespaced_name. Benchmarking shows approximately zero change. * Allow the types of ns and name to differ when constructing a NamespacedName. * Make symbol namespaces optional. * Normalize names of keyword/symbol constructors. This will make the subsequent refactor much less painful. * Use expect not unwrap. * Merge Keyword and NamespacedKeyword.
2018-05-11 16:52:17 +00:00
= keyword_prefix ns:$(symbol_namespace) namespace_separator n:$(symbol_name) { Keyword::namespaced(ns, n) }
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
/ #expected("namespaced keyword")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_forward_namespaced_keyword -> Keyword
= v:raw_namespaced_keyword {? if v.is_forward() { Ok(v) } else { Err("expected namespaced :forward/keyword") } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
[edn] Pre: Rename keyword to namespaced_keyword. The `Keyword` type evolved to become more general: we now use the one type for both :regular and :name/spaced keywords. This changes reflects the new generality.
2018-05-29 21:01:46 +00:00
raw_backward_namespaced_keyword -> Keyword
= v:raw_namespaced_keyword {? if v.is_backward() { Ok(v) } else { Err("expected namespaced :backward/_keyword") } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
entid -> EntidOrIdent
= v:( raw_basedinteger / raw_hexinteger / raw_octalinteger / raw_integer ) { EntidOrIdent::Entid(v) }
/ v:raw_namespaced_keyword { EntidOrIdent::Ident(v) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
forward_entid -> EntidOrIdent
= v:( raw_basedinteger / raw_hexinteger / raw_octalinteger / raw_integer ) { EntidOrIdent::Entid(v) }
/ v:raw_forward_namespaced_keyword { EntidOrIdent::Ident(v) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("forward entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
backward_entid -> EntidOrIdent
= v:raw_backward_namespaced_keyword { EntidOrIdent::Ident(v.to_reversed()) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("backward entid")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
lookup_ref -> LookupRef<ValueAndSpan>
= "(" __ "lookup-ref" __ a:(entid) __ v:(value) __ ")" { LookupRef { a: AttributePlace::Entid(a), v } }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("lookup-ref")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
tx_function -> TxFunction
Combine NamespacedKeyword and Keyword. (#689) r=nalexander * Make properties on NamespacedKeyword/NamespacedSymbol private * Use only a single String for NamespacedKeyword/NamespacedSymbol * Review comments. * Remove unsafe code in namespaced_name. Benchmarking shows approximately zero change. * Allow the types of ns and name to differ when constructing a NamespacedName. * Make symbol namespaces optional. * Normalize names of keyword/symbol constructors. This will make the subsequent refactor much less painful. * Use expect not unwrap. * Merge Keyword and NamespacedKeyword.
2018-05-11 16:52:17 +00:00
= "(" __ n:$(symbol_name) __ ")" { TxFunction { op: PlainSymbol::plain(n) } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
entity_place -> EntityPlace<ValueAndSpan>
Part 4: Prepare EDN `Entity` type for interning tempids during parsing. This is all part of moving the entity builder away from building term instances and toward building entity instances. One of the nice things that the existing term interface does is allow consumers to use lightweight reference counted tempid handles; I don't want to lose that, so we'll build it into the entity data structures directly.
2018-07-02 17:33:43 +00:00
= v:raw_text { EntityPlace::TempId(TempId::External(v).into()) }
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
/ v:entid { EntityPlace::Entid(v) }
/ v:lookup_ref { EntityPlace::LookupRef(v) }
/ v:tx_function { EntityPlace::TxFunction(v) }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Rename or delete things so that there is only one type named Entid (#768) * Delete the (apparently unused) EntId * Rename edn's Entid to EntidOrIdent to avoid confusion with the Entid that's actually an i64 * Fix travis beta bustage (This is actually unrelated to entids, but is a trivial fix nonetheless)
2018-06-26 23:34:18 +00:00
value_place_pair -> (EntidOrIdent, ValuePlace<ValueAndSpan>)
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= k:(entid) __ v:(value_place) { (k, v) }
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
map_notation -> MapNotation<ValueAndSpan>
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= "{" __ kvs:(value_place_pair*) __ "}" { kvs.into_iter().collect() }
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
value_place -> ValuePlace<ValueAndSpan>
= __ v:lookup_ref __ { ValuePlace::LookupRef(v) }
/ __ v:tx_function __ { ValuePlace::TxFunction(v) }
/ __ "[" __ vs:(value_place*) __ "]" __ { ValuePlace::Vector(vs) }
/ __ v:map_notation __ { ValuePlace::MapNotation(v) }
/ __ v:atom __ { ValuePlace::Atom(v) }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
pub entity -> Entity<ValueAndSpan>
= __ "[" __ op:(op) __ e:(entity_place) __ a:(forward_entid) __ v:(value_place) __ "]" __ { Entity::AddOrRetract { op, e: e, a: AttributePlace::Entid(a), v: v } }
/ __ "[" __ op:(op) __ e:(value_place) __ a:(backward_entid) __ v:(entity_place) __ "]" __ { Entity::AddOrRetract { op, e: v, a: AttributePlace::Entid(a), v: e } }
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
/ __ map:map_notation __ { Entity::MapNotation(map) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
/ #expected("entity")
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
Generalize Entity by value type. (#701) (#691) r=rnewman * Part 3: Parameterize Entity by value type. This isn't quite right, because after parsing, we shouldn't care about` `edn::ValueAndSpan`, we should care only about edn::Value. However, I think we can drop `ValueAndSpan` entirely if we just use `rust-peg` (and its simpler error messages) rather than a mix of `rust-peg` and `combine`. In any case, this paves the way to transacting `Entity<TypedValue>`, which is a nice step towards building general entities. * Part 1: Add AttributePlace. * Part 2: Name other places EntityPlace and ValuePlace. Now we're consistent and closer to self-documenting. Both matter more as we expose `Entity` as the thing to build for programmatic usage. * Part 4: Allow Ident and TempId in ValuePlace. The parser will never produce these, since determining whether an integer/keyword or string is an ident or a tempid, respectively, in the value place requires the schema. But a builder that produces `Entity` instances directly will want to produce these.
2018-05-15 07:43:07 +00:00
pub entities -> Vec<Entity<ValueAndSpan>>
Use rust-peg for tx parsing. There are few reasons to do this: - it's difficult to add symbol interning to combine-based parsers like tx-parser -- literally every type changes to reflect the interner, and that means every convenience macro we've built needs to chagne. It's trivial to add interning to rust-peg-based parsers. - combine has rolled forward to 3.2, and I spent a similar amount of time investigating how to upgrade tx-parser (to take advantage of the new parser! macros in combine that I think are necessary for adapting to changing types) as I did just converting to rust-peg. - it's easy to improve the error messages in rust-peg, where-as I have tried twice to improve the nested error messages in combine and am stumped. - it's roughly 4x faster to parse strings directly as opposed to edn::ValueAndSpan, and it'll be even better when we intern directly.
2018-05-07 17:32:28 +00:00
= __ "[" __ es:(entity*) __ "]" __ { es }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
// Query parser starts here.
//
// We expect every rule except the `raw_*` rules to eat whitespace
// (with `__`) at its start and finish. That means that every string
// pattern (say "[") should be bracketed on either side with either a
// whitespace-eating rule or an explicit whitespace eating `__`.
query_function -> query::QueryFunction
= __ n:$(symbol_name) __ {? query::QueryFunction::from_symbol(&PlainSymbol::plain(n)).ok_or("expected query function") }
fn_arg -> query::FnArg
= v:value {? query::FnArg::from_value(&v).ok_or("expected query function argument") }
/ __ "[" args:fn_arg+ "]" __ { query::FnArg::Vector(args) }
find_elem -> query::Element
= __ v:variable __ { query::Element::Variable(v) }
/ __ "(" __ "the" v:variable ")" __ { query::Element::Corresponding(v) }
/ __ "(" __ "pull" var:variable "[" patterns:pull_attribute+ "]" __ ")" __ { query::Element::Pull(query::Pull { var, patterns }) }
/ __ "(" func:query_function args:fn_arg* ")" __ { query::Element::Aggregate(query::Aggregate { func, args }) }
find_spec -> query::FindSpec
= f:find_elem "." __ { query::FindSpec::FindScalar(f) }
/ fs:find_elem+ { query::FindSpec::FindRel(fs) }
/ __ "[" f:find_elem __ "..." __ "]" __ { query::FindSpec::FindColl(f) }
/ __ "[" fs:find_elem+ "]" __ { query::FindSpec::FindTuple(fs) }
pull_attribute -> query::PullAttributeSpec
= __ "*" __ { query::PullAttributeSpec::Wildcard }
/ __ k:raw_forward_namespaced_keyword __ alias:(":as" __ alias:raw_forward_keyword __ { alias })? {
let attribute = query::PullConcreteAttribute::Ident(::std::rc::Rc::new(k));
let alias = alias.map(|alias| ::std::rc::Rc::new(alias));
query::PullAttributeSpec::Attribute(
query::NamedPullAttribute {
attribute,
alias: alias,
})
}
limit -> query::Limit
= __ v:variable __ { query::Limit::Variable(v) }
/ __ n:(raw_octalinteger / raw_hexinteger / raw_basedinteger / raw_integer) __ {?
if n > 0 {
Ok(query::Limit::Fixed(n as u64))
} else {
Err("expected positive integer")
}
}
order -> query::Order
= __ "(" __ "asc" v:variable ")" __ { query::Order(query::Direction::Ascending, v) }
/ __ "(" __ "desc" v:variable ")" __ { query::Order(query::Direction::Descending, v) }
/ v:variable { query::Order(query::Direction::Ascending, v) }
pattern_value_place -> query::PatternValuePlace
= v:value {? query::PatternValuePlace::from_value(&v).ok_or("expected pattern_value_place") }
pattern_non_value_place -> query::PatternNonValuePlace
= v:value {? query::PatternNonValuePlace::from_value(&v).ok_or("expected pattern_non_value_place") }
pattern -> query::WhereClause
= __ "["
src:src_var?
e:pattern_non_value_place
a:pattern_non_value_place
v:pattern_value_place?
tx:pattern_non_value_place?
"]" __
{?
let v = v.unwrap_or(query::PatternValuePlace::Placeholder);
let tx = tx.unwrap_or(query::PatternNonValuePlace::Placeholder);
// Pattern::new takes care of reversal of reversed
// attributes: [?x :foo/_bar ?y] turns into
// [?y :foo/bar ?x].
//
// This is a bit messy: the inner conversion to a Pattern can
// fail if the input is something like
//
// ```edn
// [?x :foo/_reversed 23.4]
// ```
//
// because
//
// ```edn
// [23.4 :foo/reversed ?x]
// ```
//
// is nonsense. That leaves us with a nested optional, which we unwrap here.
query::Pattern::new(src, e, a, v, tx)
.map(query::WhereClause::Pattern)
.ok_or("expected pattern")
}
// TODO: this shouldn't be checked at parse time.
rule_vars -> BTreeSet<query::Variable>
= vs:variable+ {?
let given = vs.len();
let set: BTreeSet<query::Variable> = vs.into_iter().collect();
if given != set.len() {
Err("expected unique variables")
} else {
Ok(set)
}
}
or_pattern_clause -> query::OrWhereClause
= clause:where_clause { query::OrWhereClause::Clause(clause) }
or_and_clause -> query::OrWhereClause
= __ "(" __ "and" clauses:where_clause+ ")" __ { query::OrWhereClause::And(clauses) }
or_where_clause -> query::OrWhereClause
= or_pattern_clause
/ or_and_clause
or_clause -> query::WhereClause
= __ "(" __ "or" clauses:or_where_clause+ ")" __ {
query::WhereClause::OrJoin(query::OrJoin::new(query::UnifyVars::Implicit, clauses))
}
or_join_clause -> query::WhereClause
= __ "(" __ "or-join" __ "[" vars:rule_vars "]" clauses:or_where_clause+ ")" __ {
query::WhereClause::OrJoin(query::OrJoin::new(query::UnifyVars::Explicit(vars), clauses))
}
not_clause -> query::WhereClause
= __ "(" __ "not" clauses:where_clause+ ")" __ {
query::WhereClause::NotJoin(query::NotJoin::new(query::UnifyVars::Implicit, clauses))
}
not_join_clause -> query::WhereClause
= __ "(" __ "not-join" __ "[" vars:rule_vars "]" clauses:where_clause+ ")" __ {
query::WhereClause::NotJoin(query::NotJoin::new(query::UnifyVars::Explicit(vars), clauses))
}
type_annotation -> query::WhereClause
= __ "[" __ "(" __ "type" var:variable __ ty:raw_keyword __ ")" __ "]" __ {
query::WhereClause::TypeAnnotation(
query::TypeAnnotation {
value_type: ty,
variable: var,
})
}
pred -> query::WhereClause
= __ "[" __ "(" func:query_function args:fn_arg* ")" __ "]" __ {
query::WhereClause::Pred(
query::Predicate {
operator: func.0,
args: args,
})
}
pub where_fn -> query::WhereClause
= __ "[" __ "(" func:query_function args:fn_arg* ")" __ binding:binding "]" __ {
query::WhereClause::WhereFn(
query::WhereFn {
operator: func.0,
args: args,
binding,
})
}
where_clause -> query::WhereClause
// Right now we only support patterns and predicates. See #239 for more.
= pattern
/ or_join_clause
/ or_clause
/ not_join_clause
/ not_clause
/ type_annotation
/ pred
/ where_fn
query_part -> query::QueryPart
= __ ":find" fs:find_spec { query::QueryPart::FindSpec(fs) }
/ __ ":in" in_vars:variable+ { query::QueryPart::InVars(in_vars) }
/ __ ":limit" l:limit { query::QueryPart::Limit(l) }
/ __ ":order" os:order+ { query::QueryPart::Order(os) }
/ __ ":where" ws:where_clause+ { query::QueryPart::WhereClauses(ws) }
/ __ ":with" with_vars:variable+ { query::QueryPart::WithVars(with_vars) }
Review comments.
2018-06-04 22:21:27 +00:00
pub parse_query -> query::ParsedQuery
= __ "[" qps:query_part+ "]" __ {? query::ParsedQuery::from_parts(qps) }
Part 4: Parse queries with `rust-peg`. There's an unfortunate conflation here between implementing the query parser in `rust-peg` and moving some validation that now happens at parse time to happen later. The result is that we introduce `ParsedFindQuery` as a less-processed `FindQuery`, and that we only use string errors (which is all `rust-peg` supports) instead of the structured errors in query-parser's errors module. The next commit will address this, on the road to removing the `query-parser` module entirely.
2018-05-28 22:27:19 +00:00
variable -> query::Variable
= v:value {? query::Variable::from_value(&v).ok_or("expected variable") }
src_var -> query::SrcVar
= v:value {? query::SrcVar::from_value(&v).ok_or("expected src_var") }
variable_or_placeholder -> query::VariableOrPlaceholder
= v:variable { query::VariableOrPlaceholder::Variable(v) }
/ __ "_" __ { query::VariableOrPlaceholder::Placeholder }
binding -> query::Binding
= __ "[" __ "[" vs:variable_or_placeholder+ "]" __ "]" __ { query::Binding::BindRel(vs) }
/ __ "[" v:variable "..." __ "]" __ { query::Binding::BindColl(v) }
/ __ "[" vs:variable_or_placeholder+ "]" __ { query::Binding::BindTuple(vs) }
/ v:variable { query::Binding::BindScalar(v) }
Reference in a new issue Copy permalink