Skip to content

Pipeline: input: exec: style #1722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 10, 2025
Merged

Pipeline: input: exec: style #1722

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 53 additions & 44 deletions pipeline/inputs/exec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,42 +1,50 @@
# Exec

The **exec** input plugin, allows to execute external program and collects event logs.
The _Exec_ input plugin lets you execute external programs and collects event logs.

**WARNING**: Because this plugin invokes commands via a shell, its inputs are
subject to shell metacharacter substitution. Careless use of untrusted input in
command arguments could lead to malicious command execution.
{% hint style="warning" %}

This plugin invokes commands using a shell. Its inputs are subject to shell metacharacter substitution. Careless use of untrusted input in command arguments could lead to malicious command execution.

{% end hint %}

## Container support

This plugin will not function in all the distroless production images as it needs a functional `/bin/sh` which is not present.
The debug images use the same binaries so even though they have a shell, there is no support for this plugin as it is compiled out.
This plugin needs a functional `/bin/sh` and won't function in all the distroless production images.

The debug images use the same binaries so even though they have a shell, there is no support for this plugin as it's compiled out.

## Configuration Parameters
## Configuration parameters

The plugin supports the following configuration parameters:

| Key | Description |
| :--- | :--- |
| Command | The command to execute, passed to [popen(...)](https://man7.org/linux/man-pages/man3/popen.3.html) without any additional escaping or processing. May include pipelines, redirection, command-substitution, etc. |
| Parser | Specify the name of a parser to interpret the entry as a structured message. |
| Interval\_Sec | Polling interval \(seconds\). |
| Interval\_NSec | Polling interval \(nanosecond\). |
| Buf\_Size | Size of the buffer \(check [unit sizes](../../administration/configuring-fluent-bit/unit-sizes.md) for allowed values\) |
| Oneshot | Only run once at startup. This allows collection of data precedent to fluent-bit's startup (bool, default: false) |
| Exit\_After\_Oneshot | Exit as soon as the one-shot command exits. This allows the exec plugin to be used as a wrapper for another command, sending the target command's output to any fluent-bit sink(s) then exiting. (bool, default: false) |
| Propagate\_Exit\_Code | When exiting due to Exit\_After\_Oneshot, cause fluent-bit to exit with the exit code of the command exited by this plugin. Follows [shell conventions for exit code propagation](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). (bool, default: false) |
| Threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). Default: `false`. |

## Getting Started
| `Command` | The command to execute, passed to [popen](https://man7.org/linux/man-pages/man3/popen.3.html) without any additional escaping or processing. Can include pipelines, redirection, command-substitution, or other information. |
| `Parser` | Specify the name of a parser to interpret the entry as a structured message. |
| `Interval_Sec` | Polling interval (seconds). |
| `Interval_NSec` | Polling interval (nanosecond). |
| `Buf_Size` | Size of the buffer. See [unit sizes](../../administration/configuring-fluent-bit/unit-sizes.md) for allowed values. |
| `Oneshot` | Only run once at startup. This allows collection of data precedent to Fluent Bit startup (Boolean, default: `false`). |
| `Exit_After_Oneshot` | Exit as soon as the one-shot command exits. This allows the `exec` plugin to be used as a wrapper for another command, sending the target command's output to any Fluent Bit sink, then exits. (Boolean, default: `false`). |
| `Propagate_Exit_Code` | When exiting due to `Exit_After_Oneshot`, cause Fluent Bit to exit with the exit code of the command exited by this plugin. Follows [shell conventions for exit code propagation](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). (Boolean, default: `false`). |
| `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). Default: `false`. |

## Get started

You can run the plugin from the command line or through the configuration file:

### Command Line
### Command line

The following example will read events from the output of _ls_.

```bash
$ fluent-bit -i exec -p 'command=ls /var/log' -o stdout
fluent-bit -i exec -p 'command=ls /var/log' -o stdout
```

which should return something like the following:

```text
Fluent Bit v1.x.x
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
Expand All @@ -53,12 +61,13 @@ Fluent Bit v1.x.x
[6] exec.0: [1521622010.013497225, {"exec"=>"anaconda.storage.log"}]
```

### Configuration File
### Configuration file

In your main configuration file append the following _Input_ & _Output_ sections:
In your main configuration file append the following `Input` and `Output` sections:

{% tabs %}
{% tab title="fluent-bit.conf" %}

```python
[INPUT]
Name exec
Expand All @@ -73,9 +82,11 @@ In your main configuration file append the following _Input_ & _Output_ sections
Name stdout
Match *
```

{% endtab %}

{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
Expand All @@ -91,17 +102,18 @@ pipeline:
- name: stdout
match: '*'
```

{% endtab %}
{% endtabs %}

## Use as a command wrapper

To use `fluent-bit` with the `exec` plugin to wrap another command, use the
`Exit_After_Oneshot` and `Propagate_Exit_Code` options, e.g.:
To use Fluent Bit with the `exec` plugin to wrap another command, use the `Exit_After_Oneshot` and `Propagate_Exit_Code` options:

{% tabs %}
{% tab title="fluent-bit.conf" %}
```

```python
[INPUT]
Name exec
Tag exec_oneshot_demo
Expand All @@ -114,9 +126,11 @@ To use `fluent-bit` with the `exec` plugin to wrap another command, use the
Name stdout
Match *
```

{% endtab %}

{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
Expand All @@ -131,12 +145,13 @@ pipeline:
- name: stdout
match: '*'
```

{% endtab %}
{% endtabs %}

`fluent-bit` will output
Fluent Bit will output:

```
```text
[0] exec_oneshot_demo: [[1681702172.950574027, {}], {"exec"=>"count: 1"}]
[1] exec_oneshot_demo: [[1681702173.951663666, {}], {"exec"=>"count: 2"}]
[2] exec_oneshot_demo: [[1681702174.953873724, {}], {"exec"=>"count: 3"}]
Expand All @@ -149,27 +164,22 @@ pipeline:
[9] exec_oneshot_demo: [[1681702181.965852990, {}], {"exec"=>"count: 10"}]
```

then exit with exit code 1.
then exits with exit code 1.

Translation of command exit code(s) to `fluent-bit` exit code follows
[the usual shell rules for exit code handling](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html).
Like with a shell, there is no way to differentiate between the command exiting
on a signal and the shell exiting on a signal, and no way to differentiate
between normal exits with codes greater than 125 and abnormal or signal exits
reported by fluent-bit or the shell. Wrapped commands should use exit codes
between 0 and 125 inclusive to allow reliable identification of normal exit.
If the command is a pipeline, the exit code will be the exit code of the last
command in the pipeline unless overridden by shell options.
Translation of command exit codes to Fluent Bit exit code follows [the usual shell rules for exit code handling](https://www.gnu.org/software/bash/manual/html_node/Exit-Status.html). Like with a shell, there is no way to differentiate between the command exiting on a signal and the shell exiting on a signal. Similarly, there is no way to differentiate between normal exits with codes greater than `125` and abnormal or signal exits reported by Fluent Bit or the shell. Wrapped commands should use exit codes between `0` and `125` inclusive to allow reliable identification of normal exit. If the command is a pipeline, the exit code will be the exit code of the last command in the pipeline unless overridden by shell options.

### Parsing command output

By default the `exec` plugin emits one message per command output line, with a
single field `exec` containing the full message. Use the `Parser` directive to
specify the name of a parser configuration to use to process the command input.
By default the `exec` plugin emits one message per command output line, with a single field `exec` containing the full message. Use the `Parser` directive to specify the name of a parser configuration to use to process the command input.

### Security concerns

**Take great care with shell quoting and escaping when wrapping commands**.
{% hint style="warning" %}

Take great care with shell quoting and escaping when wrapping commands**.

{% endhint %}

A script like

```bash
Expand All @@ -186,11 +196,10 @@ exec fluent-bit \
can ruin your day if someone passes it the argument
`$(rm -rf /my/important/files; echo "deleted your stuff!")'`

The above script would be safer if written with:
The previous script would be safer if written with:

```bash
-p command='echo '"$(printf '%q' "$@")" \
```

... but it's generally best to avoid dynamically generating the command or
handling untrusted arguments to it at all.
It's generally best to avoid dynamically generating the command or handling untrusted arguments.