Initial experiments with serialized testing

Cargo.toml

@@ -15,6 +15,7 @@ members=[
     "lrpar/examples/clone_param",
     "lrtable",
     "nimbleparse",
+    "grammar_testing",
 ]
 resolver = "2"
 
@@ -48,3 +49,4 @@ proc-macro2 = "1.0"
 prettyplease = "0.2.31"
 syn = "2.0"
 yaml-rust2 = "0.10.1"
+ron = "0.10"

doc/src/SUMMARY.md

@@ -14,12 +14,14 @@
   - [grmtools parsing idioms](parsing_idioms.md)
   - [Error recovery](errorrecovery.md)
   - [An AST evaluator](ast_example.md)
+  - [Testing](testing.md)
 - [Rust Editions](editions.md)
 - [The individual libraries and tools](libsandtools.md)
   - [lrpar](lrpar.md)
   - [lrlex](lrlex.md)
   - [nimbleparse](nimbleparse.md)
   - [cfgrammar](cfgrammar.md)
   - [lrtable](lrtable.md)
+  - [grammar_testing](grammar_testing.md)
   - [third party](thirdparty.md)
 - [Other Rust parsing tools](othertools.md)

doc/src/grammar_testing.md

@@ -0,0 +1,6 @@
+# `grammar_testing`
+
+`grammar_testing` ([crate](https://crates.io/crates/grammar_testing);
+[source](https://github.com/softdevteam/grmtools/tree/master/grammar_testing)) is a library
+that facilitates testing of grammars. It's primary purpose is to define an standardize serialization formats
+so tests can be performed from tools like `nimbleparse` as well as user specified build scripts.

doc/src/testing.md

@@ -0,0 +1,165 @@
+# Testing (experimental)
+
+grmtools testing facilities are currently experimental, under development, and subject to change.
+Not all features described in this are section currently implemented (in fact most are incomplete).
+Currently this text primarily serves as a design document for testing features and feedback is welcome.
+
+## Testing raw input
+
+### Example lexer
+🚀
+```lex
+%%
+[a-z] "character"
+\" '"'
+[ \n\t] ;
+```
+
+
+### Example grammar
+🚀 
+```yacc
+%grmtools{
+    yacckind: Original(NoAction),
+    test_files: "input*.txt"
+}
+
+%%
+start: "character";
+```
+
+
+### Input text
+🚀 
+Contents specified via `test_files`:
+```text
+a
+```
+
+## Specifying multiple globs
+🚧
+It would be nice if you could specify multiple globs:
+```
+test_files: ["*.txt", "*.grmtest"],
+```
+
+## Testing with serialized formats
+
+🚧
+When specifying `test_files` the `grmtest` extension is treated specially.
+
+### Example Grammar for serializated test data.
+🚧
+By specifying the `grmtest` extension for the `test_files` value you enable serialized
+test data including output expectations for abstract syntax trees and error text.
+
+```yacc
+    test_files "*.grmtest"
+```
+
+
+### Serialized test input
+🚧
+The `grmtest` file deserializes to a struct with many optional fields.
+It is a [`ron`](https://crates.io/crates/ron) using the [Implicit Some](https://github.com/ron-rs/ron/blob/master/docs/extensions.md#implicit_some) extension.
+This allows you to omit most of the values, but has the downside that typos may fall back to default values.
+
+#### Fields 
+🚧
+* `input: String` field is required, and specifies the input to the parser.
+* `pass: bool` defaults to None, only Some if explicitly specified.
+* `ast: Option<grammar_testing::ASTRepr>` if present specifies the expected output see [Serializing with nimbleparse] for generating ast output.
+* `errors: Option<Vec<String>>` if present specifies the expected error text, without error recovery.
+
+#### Methods
+* `should_pass(&self)` Returns the value of the `pass` field when it is `Some` otherwise returns whether the `errors` field is `None` or it's inner value is empty.
+
+#### Example .grmtest file
+🚧
+```grmtest
+(
+   input: "a",
+   ast: ("start", [
+        ("character", "a"),
+    ]),
+)
+```
+
+
+### Serializing with nimbleparse
+
+#### ast output
+* 🚀 Using the `-s` option to nimbleparse to produce a serialized ast.
+* 🚧 `-s ast`
+
+```console
+$ echo "a" | nimbleparse -s testing.l testing.y -
+("start", [
+    ("character", "a"),
+])
+```
+
+
+#### grmtest output
+🚧 `-s grmtest` 
+
+```console
+$ echo "a" | nimbleparse -s grmtest testing.l testing.y -
+(
+   input: "a\n",
+   ast: ("start", [
+    ("character", "a"),
+    ])
+)
+```
+
+
+## Testing multiple inputs
+🚧 
+Specify a plural extension `test_files: "*.grmtests` to allow a vector of test inputs.
+
+```grmtests
+[
+    (input: "a"),
+    (input: "b"),
+    (input: "a", ast: ("start", [("character", "a")])),
+    (input: "abc", pass: false),
+]
+```
+
+
+## Escaping and raw strings
+
+Because grammars often contain quotation marks, it is convenient to support rusts `raw string` format.
+By default the serializer outputs raw strings.
+
+🚀
+```grmtest
+$ echo "\"" | ./target/debug/nimbleparse -s testing.{l,y} -
+("start", [
+    (r#"""#, r#"""#),
+])
+```
+
+🚧
+```
+[
+  // Raw strings, and escaped characters are usable interchangably.
+  ( input: r#"""#, ast:("start", [(r#"""#, "\"")])),
+  ( input: "\"", ast:("start", [("\"", r#"""#)])),
+]
+```
+
+
+## Running tests
+🚀
+
+If using build.rs/`CTParserBuilder` just cargo build.
+If no arguments are specified `nimbleparse testing.l testing.y` then if `test_files` is specified it will be used by default.
+
+## Manual Lexer limitations
+🚧
+
+Currently grmtools testing facilities are limited to being used with the combination of `lrlex`,
+and `lrpar`.  There currently isn't any mechanism in place for testing when using a manual lexer.
+Doing so requires using unstable/hidden/undocumented aspects of the API.

grammar_testing/Cargo.toml

@@ -0,0 +1,8 @@
+[package]
+name = "grammar_testing"
+version = "0.0.1"
+edition = "2024"
+
+[dependencies]
+ron.workspace = true
+serde =  { workspace = true, features = ["derive"] }