Better documentation for `filters` key

[ash-m]

Summary

• What does a name filter refer to?
• Plural tags + --help make valid configuration values for filter options ambiguous (string only? array?)
• It should be noted that multiple command-line filter options use AND conditions only making it only useful for multiple exclusions.
• Docs omit quote marks around strings with leading @-sign which is invalid YAML.
• Append note on suite tags to suggest reading organization as well.

Elaboration

The docs do mention a name filter, but don't really tell you what it is. The thoughts that come to mind are:

• A filename or part of a filename? (makes the most sense to me)
• A suite name? (makes no sense to me)
• The prose after Feature:? (aligns with how roles are defined; really though?)
• Scenario prose? (implied since that's often what matches against regex)

behat --help is a little better:

      --name=NAME                        Only executeCall the feature elements which match part
                                         of the given name or regex. (multiple values allowed)
      --tags=TAGS                        Only executeCall the features or scenarios with tags
                                         matching tag filter expression. (multiple values allowed)

Which begs the question, what is a "feature element"?

There is also some ambiguity in allowed values. The (multiple values allowed) as well as the fact that tags is plural would suggest the mapping value can be an array of tags in the configuration, but you will get:

In TagFilter.php line 33:

  trim(): Argument Behat/Behat#1 ($string) must be of type string, array given

The "multiple values allowed" note would lead one to expect filters are applied with an OR condition, (ie you might think --tags=@wip --tags=@important would run features with either tag) but they appear to be applied with an AND condition. This makes exclusionary tags work properly and with the word "filter" (like, "filter these away"), but when you provide a negation option you expect things to work differently.

Also, documentation pages use invalid YAML since scalars with a leading reserved key must be quoted. These pages include:

• https://behat.org/en/latest/user_guide/configuration/suites.html#suite-filters
• https://behat.org/en/latest/user_guide/organizing.html

The note on suites.html referring users to identifying.html should also probably callout organizing.html since it describes more explicitly how to use tags, rather than just how to configure them.

Conclusion

I'm not claiming this should be a feature or is a bug, (even if it is), but it should be noted in the documentation what valid values are, and an example usage for name would be nice. If the AND condition is intended, then that should be noted too.

[carlos-granados]

Moved to the documentation repository where it better belongs