diff --git a/.gitbook/assets/markdownlint-example.png b/.gitbook/assets/markdownlint-example.png new file mode 100644 index 000000000..35cb39bc5 Binary files /dev/null and b/.gitbook/assets/markdownlint-example.png differ diff --git a/.gitbook/assets/vale-example-error.png b/.gitbook/assets/vale-example-error.png new file mode 100644 index 000000000..e57ecbb92 Binary files /dev/null and b/.gitbook/assets/vale-example-error.png differ diff --git a/.gitbook/assets/vale-example-warning.png b/.gitbook/assets/vale-example-warning.png new file mode 100644 index 000000000..460c7579b Binary files /dev/null and b/.gitbook/assets/vale-example-warning.png differ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f4a019db6..b2f1ed5d0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,193 +1,158 @@ + + + # Contributing to Fluent Bit docs -First of all, thanks for taking the time to read this guide. The fact that you're -here means you're interested in contributing to Fluent Bit, and we highly appreciate -your time. +First of all, thanks for taking the time to read this guide. The fact that you're here means you're interested in contributing to Fluent Bit, and we greatly appreciate your time. -This repository contains the files for the -[Fluent Bit documentation library](https://docs.fluentbit.io/). Keeping these docs -separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) -helps reduce the number of commits to the Fluent Bit source code and makes it -easier to maintain both projects. +This repository contains the files for the [Fluent Bit documentation library](https://docs.fluentbit.io/). Keeping these docs separate from the [main Fluent Bit repository](https://github.com/fluent/fluent-bit) helps reduce the number of commits to the Fluent Bit source code and makes it easier to maintain both projects. -Fluent Bit has a group of dedicated maintainers who oversee this repository, -including several technical writers. These writers will review any pull requests -you open, so don't be afraid to contribute—even if you're not a writer by trade. -Your suggestions are valuable, and we'll help you wrangle any stray commas. +Fluent Bit has a group of dedicated maintainers who oversee this repository, including several technical writers. These writers will review any pull requests you open, so don't be afraid to contribute, even if you're not a writer by trade. Your suggestions are valuable, and we'll help you wrangle any stray commas. -## GitBook +## :star: Quick tips -The Fluent Bit docs library is built and hosted through -[GitBook](https://docs.gitbook.com/). Unfortunately, GitBook doesn't support -local previews for contributors, but a Fluent Bit maintainer with a dedicated GitBook -account can verify that things are formatted correctly after you open a new pull -request. +Before you contribute to the Fluent Bit docs, review these critical tips: -Each `.md` file in this repository is a single page. You can use -[standard Markdown syntax](https://docs.gitbook.com/content-editor/editing-content/markdown) -to edit existing pages, or create a new `.md` file to add an additional page to -the docs library. If you create a new page, you'll also need to update -[GitBook's `SUMMARY.md` file](https://docs.gitbook.com/integrations/git-sync/content-configuration#structure) -(or request that a maintainer to update it for you). +- [Sign off](#sign-off-your-git-commits) your Git commits. +- Use [soft line wraps](#line-wraps) in Markdown files. +- To link between pages, use [absolute file paths](#links). +- Review the results of [linters](#linters) for style and formatting guidance. -## Workflow +## Review process -After you open a pull request in this repository, a Fluent Bit maintainer will review -it, triage it, add comments or suggestions as needed, and then merge it. After -your changes are successfully merged into `master`, the docs site will update -within a few minutes. +After you open a pull request in this repository, a Fluent Bit maintainer will review it, add comments or suggestions as needed, and then merge it. After your changes are successfully merged into `master`, the docs site will update within a few minutes. -### Stale pull requests +### Request review without merging -If you open a pull request that requires ongoing discussion or review, the Fluent Bit -maintainers will add a `waiting-for-user` to your pull request. This tag means that -we're blocked from moving forward until you reply. To keep contributions from going -stale, we'll wait 45 days for your response, but we might close the pull request if we -don't hear back from you by then. +If you're contributing documentation for a Fluent Bit feature that's still in development, ask a maintainer to add the `waiting-on-code-merge` label to your pull request. This lets other maintainers know that your changes aren't ready to merge yet, even if they were approved. -## Submit a contribution +### Stale pull requests -When you open a pull request, make your changes against `master`, which is the -active development branch. If your contribution also applies to the latest -stable version, submit another PR for that versioned branch. However, if -submitting multiple PRs at the same time adds too much complexity, you can instead -create a single PR against `master` and specify that your changes need to be -**backported** to other branches; one of our maintainers will take care of that -process on your behalf. +If you open a pull request that requires extended discussion or review, the Fluent Bit maintainers will add a `waiting-for-user` label to your pull request. This label means that we're blocked from moving forward until you reply. To keep contributions from going stale, we'll wait 45 days for your response, after which we might close the pull request if we don't hear back from you. -All contributions must be made **first** against [master branch](https://github.com/fluent/fluent-bit-docs/tree/master) which is the active development branch, and then if the contribution also applies for the current stable branch, submit another PR for that specific branch, if submitting another PR adds some complexity, specify in the first PR as a comment (for master branch) that it needs to be backported. One of the maintainers will take care of that process. +## Pass DCO checks -As a contributor, we'll ask you to follow a few best practices related to Git: +To pass this repository's [DCO checks](https://github.com/apps/dco), you'll need to adhere to the following Git practices. -### One file per commit +### Set your email in Git -Each commit you make should only modify one file or interface, following the same -practice as commits to the Fluent Bit source code. +To sign commits properly, you must first set your email address in your local Git environment. This should be the same email address associated with your GitHub account. -### Commit subjects +For more information, refer to GitHub's guide to [setting your commit email address in Git](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address#setting-your-commit-email-address-in-git). -Use descriptive commit subjects that describe which file or interface you're -modifying. +### Sign off your Git commits -For example, if you're modifying the Syslog output plugin doc, whose file is -located at [pipeline/outputs/syslog.md](https://github.com/fluent/fluent-bit-docs/blob/master/pipeline/outputs/syslog.md), this would be a descriptive commit subject: +You must sign off your commits to certify your identity as the commit author. A commit is signed off when its commit message looks like this: -`pipeline: outputs: syslog: fix grammar in examples` +```text +pipeline: outputs: syslog: fix grammar in examples +Signed-off-by: Tux +``` -Because this commit is prefixed with the relevant file's path, it helps the maintainers -understand and prioritize your contribution. +For faster signing, you can use the `-s` flag: -### Set your email in Git +`git commit -a -s -m "pipeline: outputs: syslog: fix grammar in examples"` -Make sure your email address is configured in your local Git environment. This -should be the same email address associated with your GitHub account. +> :bulb: If you're using VSCode, the [**Git: Always Sign Off**](https://github.com/microsoft/vscode/issues/83096#issuecomment-545350047) option appends a `Signed-off-by:` message to all of your commits. -For more information, refer to GitHub's guide to -[setting your commit email address in Git](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address#setting-your-commit-email-address-in-git). +### Fix "DCO is missing" errors -### Sign off your commits +If a DCO error blocks your pull request from merging, refer to this guide about [how to add sign-offs retroactively](https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md#how-to-add-sign-offs-retroactively). -You must sign off your commits to certify your identity as the commit author. If -you don't sign off your commits, the CI system will flag the pull request with a -[DCO](https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md) -error and prevent your pull request from merging. +## Style guidelines -To prevent DCO errors, refer to the following guide about -[signing your commits properly](https://github.com/src-d/guide/blob/master/developer-community/fix-DCO.md). +The Fluent Bit documentation follows the [Google developer documentation style guide](https://developers.google.com/style) for most topics related to grammar and style. We don't expect you to memorize these style rules, but the maintainers who review your pull request might suggest changes accordingly. -> :bulb: For faster signing, you can use the `-s` flag in Git: -> -> `git commit -a -s -m "pipeline: outputs: syslog: fix grammar in examples"` -> -> If you're using VSCode, enable the -> [**Git: Always Sign Off**](https://github.com/microsoft/vscode/issues/83096#issuecomment-545350047) -> setting, which automatically appends a `Signed-off-by:` message to your commits. +The active [linters](#linters) in this repository flag certain style errors and, in some cases, explain how to fix them. -## Style guidelines +## Formatting guidelines -The Fluent Bit maintainers refer to the -[Google developer documentation style guide](https://developers.google.com/style) -for most topics related to grammar, style, and formatting. We don't expect you -to memorize these style rules, but the technical writer who reviews your pull -request can suggest changes accordingly. +The Fluent Bit docs library is built and hosted through [GitBook](https://docs.gitbook.com/). Unfortunately, GitBook doesn't support local previews for contributors, but a Fluent Bit maintainer with a dedicated GitBook account can verify that things are formatted correctly after you open a new pull request. -### URLs +### Links -When cross-linking to a page in this repository, use a full absolute path whenever -possible. For example: +When cross-linking between in this repository, use a full absolute path whenever possible. For example: ```text [LTSV](../pipeline/parsers/ltsv.md) and [Logfmt](../pipeline/parsers/logfmt.md) ``` -## Additional testing +### Line wraps + +When GitBook renders pages, it treats all newlines literally, which means hard line wraps in Markdown files create awkward line breaks in the Fluent Bit docs site. Due to this, docs contributions must use soft line wraps. + +
+:white_check_mark: Example: soft line wraps + +```text +Soft-wrapped text uses newlines only to mark the end of a paragraph. From the perspective of a text editor, this means each paragraph looks like a single, unbroken line of text. + +After two newlines, another paragraph begins. +``` + +
+ +
+:no_entry_sign: Example: hard line wraps + +```text +Hard-wrapped text uses newlines +in the middle of sentences and +paragraphs. + +This can make text easier for +humans to read, but GitBook +renders hard line wraps +awkwardly. +``` + +
+ +### Quotes + +By default, Google Docs and Microsoft Word turn standard straight quotes into "smart" +curly quotes. If you copy-paste from one of these tools, you must correct the quotes back to straight quotes. You can also turn off smart quotes in [Google Docs](https://support.google.com/docs/thread/217182974/can-i-turn-smart-quotes-off-in-a-google-doc?hl=en) or [Microsoft Word](https://support.microsoft.com/en-us/office/smart-quotes-in-word-and-powerpoint-702fc92e-b723-4e3d-b2cc-71dedaf2f343) to prevent this problem. + +### Table of contents + +When you create a new `.md` file for a new page, you must add an entry to this repository's [`SUMMARY.md` file](https://github.com/fluent/fluent-bit-docs/blob/master/SUMMARY.md) (or ask a maintainer to add it on your behalf). If you don't update `SUMMARY.md`, the new page won't appear in the table of contents on the Fluent Bit docs site. + +Similarly, if you update the `# h1` title header of an existing page, be sure to update that page's `SUMMARY.md` entry to match. `SUMMARY.md` entries takes precedence over in-page headers, which means that if you update a page's `# h1` title without updating `SUMMARY.md`, the unchanged `SUMMARY.md` title will persist in both the rendered page and the table of contents. + +## Linters -Fluent Bit uses additional tests for style and consistency. Your maintainers -suggest installing and applying the suggestions from these linters. +This repository runs linters as GitHub Actions for each pull request. If a linter finds errors or makes suggested changes, you can view these results in the **Files changed** tab. + +
+:mag: Examples: linter results + +![An example of a warning-level Vale result.](/.gitbook/assets/vale-example-warning.png) + +![An example of an error-level Vale result.](/.gitbook/assets/vale-example-error.png) + +![An example of a Markdownlint result.](/.gitbook/assets/markdownlint-example.png) + +
### Vale -The Fluent Bit maintainers use the [Vale](https://vale.sh/docs/) plugin, which lints -the prose of your pull requests and adds suggestions to improve style and clarity. -There is a [VSCode plugin for Vale](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) -which outputs suggestions to the problems pane in the IDE. - -The Fluent Bit repository runs Vale as a GitHub Action on updated text in new pull -requests. Vale suggestions, errors, and warnings will display in GitHub on the -**Files changed** page. - -[See the Vale tests for Fluent Bit](https://github.com/fluent/fluent-bit-docs/tree/master/vale-styles). - -Most Vale tests are at the `suggestion` level and won't block merging. - -The following tests are at `error` level and will cause a test failure: - -- [AmSpelling](https://developers.google.com/style/word-list) -- [Ampersand](https://developers.google.com/style/word-list#ampersand) -- [Don'tUse](https://github.com/fluent/fluent-bit-docs/blob/master/vale-styles/FluentBit/DontUse.yml) -- [Emdash](https://github.com/errata-ai/Google/blob/master/Google/EmDash.yml) -- [Endash](https://github.com/errata-ai/Google/blob/master/Google/EmDash.yml) -- [Exclamation](https://github.com/errata-ai/Google/blob/master/Google/Exclamation.yml) -- [Gender](https://developers.google.com/style/pronouns#gender-neutral-pronouns) -- [GenderBias](https://developers.google.com/style/inclusive-documentation) -- [Hints](https://github.com/fluent/fluent-bit-docs/blob/master/vale-styles/FluentBit/Hints.yml) using `>`. -- [HeadingPunctuation](https://developers.google.com/style/capitalization#capitalization-in-titles-and-headings) -- [Latin](https://developers.google.com/style/abbreviations) -- [LyHyphens](https://developers.google.com/style/hyphens) -- [MayMightCan](https://github.com/fluent/fluent-bit-docs/blob/master/vale-styles/FluentBit/MayMightCan.yml) -- [NonStandardQuotes](https://github.com/fluent/fluent-bit-docs/blob/master/vale-styles/FluentBit/NonStandardQuotes.yml): - [Use standard quotes](https://developers.google.com/style/quotation-marks#straight-and-curly-quotation-marks). - By default, Google Docs and Microsoft Word turn standard straight quotes into "smart" - curly quotes. If you copy-paste from one of these tools, you must correct the quotes - back to straight quotes. You can also turn off smart quotes - in [Google Docs](https://support.google.com/docs/thread/217182974/can-i-turn-smart-quotes-off-in-a-google-doc?hl=en) - or [Microsoft Word](https://support.microsoft.com/en-us/office/smart-quotes-in-word-and-powerpoint-702fc92e-b723-4e3d-b2cc-71dedaf2f343) - to prevent this problem. -- [Optional plurals](https://developers.google.com/style/plurals-parentheses) -- [Ordinal](https://developers.google.com/style/numbers) -- [Periods](https://developers.google.com/style/abbreviations) -- [Ranges](https://developers.google.com/style/hyphens) -- [Repetition](https://github.com/errata-ai/vale/blob/v3/testdata/styles/Markup/Repetition.yml): - Checks for the same word used twice in succession. -- [Slang](https://developers.google.com/style/abbreviations) -- [Spacing](https://developers.google.com/style/sentence-spacing) - -The following tests are at a `warning` level and won't prevent merging: - -- [Ampersand](https://developers.google.com/style/word-list#ampersand) -- [First person](https://developers.google.com/style/pronouns#personal-pronouns) -- [Possessives](https://developers.google.com/style/possessives) -- [Simplicity](https://developers.google.com/style/word-list#easy) +[Vale](https://vale.sh/docs/) lints prose for style and clarity. In addition to reviewing the results of each Vale test in GitHub, you can use the [Vale plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) to view errors and suggestions locally. + +Vale tests for the Fluent Bit docs are stored in the [`/vale-styles`](https://github.com/fluent/fluent-bit-docs/tree/master/vale-styles) folder. Most Vale tests are at the `suggestion` or `warning` level and won't block pull requests from merging. However, tests at the `error` level will block merging until the associated issue is fixed. ### Markdownlint -[Markdownlint](https://github.com/markdownlint/markdownlint) checks markdown in a -file and makes suggestions for improvements. Most markdownlint tests are enabled. -[See the configuration file](https://github.com/fluent/fluent-bit-docs/blob/master/.markdownlint.json). +[Markdownlint](https://github.com/markdownlint/markdownlint) lints Markdown formatting and makes suggestions for improvements. In addition to reviewing the results of each test in GitHub, you can use the [markdownlint plugin for VSCode](https://marketplace.visualstudio.com/items?itemName=DavidAnson.vscode-markdownlint) to view suggestions locally. + +Markdownlint tests for the Fluent Bit docs are stored in [`markdownlint.json`](https://github.com/fluent/fluent-bit-docs/blob/master/.markdownlint.json). These tests don't block pull requests from merging. + +## Additional resources + +For general guidance about writing documentation, consult the following resources: -The Fluent Bit documentation repository is using a GitHub action to check for invalid -markdown based on the configuration file. This action might provide review -suggestions, but doesn't block merging. +- [Open Source Technical Documentation Essentials (LFC111)](https://training.linuxfoundation.org/training/open-source-technical-documentation-essentials-lfc111/) +- [Creating Effective Documentation for Developers (LFC112)](https://training.linuxfoundation.org/training/creating-effective-documentation-for-developers-lfc112/) +- [Google Technical Writing Courses for Engineers](https://developers.google.com/tech-writing) -Line wrap tests are off due to a GitBook decision that hard wraps the rendered page -on line wraps in the markdown. This leads to rendered pages with awkward line wraps. + + diff --git a/README.md b/README.md index 070a46708..b658e7cb4 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces - Wasm: [Wasm Filter Plugins](development/wasm-filter-plugins.md) or [Wasm Input Plugins](development/wasm-input-plugins.md) - Write [Filters in Lua](pipeline/filters/lua.md) or [Output plugins in Golang](development/golang-output-plugins.md) - [Monitoring](administration/monitoring.md): Expose internal metrics over HTTP in JSON and [Prometheus](https://prometheus.io/) format -- [Stream Processing](stream-processing/introduction.md): Perform data selection and transformation using simple SQL queries +- [Stream Processing](stream-processing/introduction.md): Perform data selection and transformation using basic SQL queries - Create new streams of data using query results - Aggregation windows - Data analysis and prediction: Time series forecasting diff --git a/pipeline/outputs/opensearch.md b/pipeline/outputs/opensearch.md index 35c7d481b..f4e03d313 100644 --- a/pipeline/outputs/opensearch.md +++ b/pipeline/outputs/opensearch.md @@ -246,7 +246,7 @@ Without this you will see errors like: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400} ``` -### Fluent-Bit and Amazon OpenSearch serverless +### Fluent Bit and Amazon OpenSearch serverless Amazon OpenSearch Serverless is an offering that eliminates your need to manage OpenSearch clusters. All existing Fluent Bit OpenSearch output plugin options work with OpenSearch Serverless. For Fluent Bit, you must specify the service name as `aoss` (Amazon OpenSearch Serverless) when you enable `AWS_Auth`: diff --git a/pipeline/processors/conditional-processing.md b/pipeline/processors/conditional-processing.md index 3af5ef63d..eaf1ca0e4 100644 --- a/pipeline/processors/conditional-processing.md +++ b/pipeline/processors/conditional-processing.md @@ -11,7 +11,6 @@ You can turn a standard processor into a conditional processor by adding a `cond {% hint style="info" %} - Only [YAML configuration files](../../administration/configuring-fluent-bit/yaml/README.md) support processors, including conditional processors. - - Conditional processing isn't supported if you're using a [filter as a processor](../filters). {% endhint %}