diff --git a/pipeline/outputs/tcp-and-tls.md b/pipeline/outputs/tcp-and-tls.md index 72ceb242c..a4f266cc9 100644 --- a/pipeline/outputs/tcp-and-tls.md +++ b/pipeline/outputs/tcp-and-tls.md @@ -30,6 +30,8 @@ The following parameters are available to configure a secure channel connection | `tls.key_file` | Absolute path to private Key file. | _none_ | | `tls.key_passwd` | Optional password for `tls.key_file` file. | _none_ | +## Getting started + ### Command line #### JSON format @@ -102,3 +104,130 @@ Which return results like: (ExtType(code=0, data=b'b\n5\xc65\x05\x14\xac'), {'cpu_p': 0.1875, 'user_p': 0.125, 'system_p': 0.0625, 'cpu0.p_cpu': 0.0, 'cpu0.p_user': 0.0, 'cpu0.p_system': 0.0, 'cpu1.p_cpu': 0.0, 'cpu1.p_user': 0.0, 'cpu1.p_system': 0.0, 'cpu2.p_cpu': 1.0, 'cpu2.p_user': 0.0, 'cpu2.p_system': 1.0, 'cpu3.p_cpu': 0.0, 'cpu3.p_user': 0.0, 'cpu3.p_system': 0.0, 'cpu4.p_cpu': 0.0, 'cpu4.p_user': 0.0, 'cpu4.p_system': 0.0, 'cpu5.p_cpu': 0.0, 'cpu5.p_user': 0.0, 'cpu5.p_system': 0.0, 'cpu6.p_cpu': 0.0, 'cpu6.p_user': 0.0, 'cpu6.p_system': 0.0, 'cpu7.p_cpu': 0.0, 'cpu7.p_user': 0.0, 'cpu7.p_system': 0.0, 'cpu8.p_cpu': 0.0, 'cpu8.p_user': 0.0, 'cpu8.p_system': 0.0, 'cpu9.p_cpu': 1.0, 'cpu9.p_user': 1.0, 'cpu9.p_system': 0.0, 'cpu10.p_cpu': 0.0, 'cpu10.p_user': 0.0, 'cpu10.p_system': 0.0, 'cpu11.p_cpu': 0.0, 'cpu11.p_user': 0.0, 'cpu11.p_system': 0.0, 'cpu12.p_cpu': 0.0, 'cpu12.p_user': 0.0, 'cpu12.p_system': 0.0, 'cpu13.p_cpu': 0.0, 'cpu13.p_user': 0.0, 'cpu13.p_system': 0.0, 'cpu14.p_cpu': 0.0, 'cpu14.p_user': 0.0, 'cpu14.p_system': 0.0, 'cpu15.p_cpu': 0.0, 'cpu15.p_user': 0.0, 'cpu15.p_system': 0.0}) ... ``` + +### Configuration file + +The following is a basic setup to forward logs over TCP: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: cpu + tag: cpu + + outputs: + - name: tcp + match: '*' + host: 192.168.10.12 + port: 5170 + format: json_lines +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name cpu + Tag cpu + +[OUTPUT] + Name tcp + Match * + Host 192.168.10.12 + Port 5170 + Format json_lines +``` + +{% endtab %} +{% endtabs %} + +## Examples + +### Raw message key + +When `raw_message_key` is set, the plugin sends the value of the specified key as a raw message instead of applying formatting, causing the `format` property to be ignored: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: tail + path: /var/log/app.log + tag: app + + outputs: + - name: tcp + match: '*' + host: 127.0.0.1 + port: 5170 + raw_message_key: $message +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + Name tail + Path /var/log/app.log + Tag app + +[OUTPUT] + Name tcp + Match * + Host 127.0.0.1 + Port 5170 + Raw_Message_Key $message +``` + +{% endtab %} +{% endtabs %} + +The `$` sign is required to signal the record accessor to extract the value from the specified key - this can be a top-level or nested field: + +{% tabs %} +{% tab title="Top-level field" %} + +Example log: + +```json +{ + "timestamp": "2026-07-05T14:32:07Z", + "message": "<34>Jul 5 14:32:07 myhost app: Cache key S2VlcCBjYWxtIGFuZCBsb2cgb24=" +} +``` + +Setting `raw_message_key` to `$message` will yield: + +```text +<34>Jul 5 14:32:07 myhost app: Cache key S2VlcCBjYWxtIGFuZCBsb2cgb24= +``` + +{% endtab %} +{% tab title="Nested field" %} + +Example log: + +```json +{ + "timestamp": "2026-07-05T14:32:07Z", + "message": { + "original": "<34>Jul 5 14:32:07 myhost app: Cache key S2VlcCBjYWxtIGFuZCBsb2cgb24=" + } +} +``` + +Setting `raw_message_key` to `$message['original']` will yield: + +```text +<34>Jul 5 14:32:07 myhost app: Cache key S2VlcCBjYWxtIGFuZCBsb2cgb24= +``` + +{% endtab %} +{% endtabs %} diff --git a/pipeline/outputs/udp.md b/pipeline/outputs/udp.md index 9eaf48852..5e9adaaaa 100644 --- a/pipeline/outputs/udp.md +++ b/pipeline/outputs/udp.md @@ -146,7 +146,7 @@ After Fluent Bit connects, you should see output in JSON format: ### Raw message format -When `raw_message_key` is set, the plugin sends the value of the specified key as a raw message instead of formatting it as JSON. This is for when you want to send pre-formatted messages: +When `raw_message_key` is set, the plugin sends the value of the specified key as a raw message instead of applying formatting, causing the `format` property to be ignored: {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -163,7 +163,7 @@ pipeline: match: '*' host: 127.0.0.1 port: 5170 - raw_message_key: message + raw_message_key: $message ``` {% endtab %} @@ -180,7 +180,50 @@ pipeline: Match * Host 127.0.0.1 Port 5170 - Raw_Message_Key message + Raw_Message_Key $message +``` + +{% endtab %} +{% endtabs %} + +The `$` sign is required to signal the record accessor to extract the value from the specified key - this can be a top-level or nested field: + +{% tabs %} +{% tab title="Top-level field" %} + +Example log: + +```json +{ + "timestamp": "2026-07-05T14:32:07Z", + "message": "<34>Jul 5 14:32:07 myhost kernel: Payload detected: 54 42 50 20 77 61 73 20 68 65 72 65" +} +``` + +Setting `raw_message_key` to `$message` will yield: + +```text +<34>Jul 5 14:32:07 myhost kernel: Payload detected: 54 42 50 20 77 61 73 20 68 65 72 65 +``` + +{% endtab %} +{% tab title="Nested field" %} + +Example log: + +```json +{ + "timestamp": "2026-07-05T14:32:07Z", + "message": { + "original": "<34>Jul 5 14:32:07 myhost kernel: Payload detected: 54 42 50 20 77 61 73 20 68 65 72 65" + } +} +``` + +Setting `raw_message_key` to `$message['original']` will yield: + +```text +<34>Jul 5 14:32:07 myhost kernel: Payload detected: 54 42 50 20 77 61 73 20 68 65 72 65 ``` {% endtab %}