logfmt.mq

A logfmt parser implemented as an mq module.

Features

• Parse key=value pairs from a single line or multiple lines
• Quoted values with spaces (key="hello world")
• Bare keys without values (treated as true)
• Skip blank lines and # comments
• Stringify dicts back to logfmt format (single line or multi-line)

Installation

cp logfmt.mq ~/.local/mq/config/

HTTP Import

mq -I raw 'import "github.com/harehare/logfmt.mq" | logfmt::logfmt_parse(.)' app.log

API

Function	Description
logfmt_parse_line(input)	Parse a single logfmt line into a dict
logfmt_parse(input)	Parse multiple lines, return an array of dicts
logfmt_stringify_line(input)	Stringify a single dict into a logfmt line
logfmt_stringify(input)	Stringify an array of dicts into a multi-line logfmt string

Stringify rules

• true booleans become bare keys (fatal instead of fatal=true)
• false booleans emit key=false
• Values containing spaces or double-quotes are double-quoted, with inner quotes escaped (msg="hello world")

Example

Given app.log:

time=2024-01-15T12:00:00Z level=info msg="request received" method=GET path=/api/users status=200 duration=42ms
time=2024-01-15T12:00:01Z level=error msg="db connection failed" retries=3 fatal
time=2024-01-15T12:00:02Z level=info msg="request received" method=POST path=/api/items status=201 duration=87ms

# Parse all lines
mq -I raw 'import "logfmt" | logfmt::logfmt_parse(.)' app.log

# Filter error lines
mq -I raw 'import "logfmt" | logfmt::logfmt_parse(.) | filter(fn(e): e["level"] == "error";)' app.log
# => [{"time":"2024-01-15T12:00:01Z","level":"error","msg":"db connection failed","retries":"3","fatal":true}]

# Extract all status codes
mq -I raw 'import "logfmt" | logfmt::logfmt_parse(.) | map(fn(e): e["status"];)' app.log
# => ["200", "201"]

# Parse a single line
mq -I raw 'import "logfmt" | logfmt::logfmt_parse_line(.)' <<< 'level=info msg="hello world" ok'
# => {"level":"info","msg":"hello world","ok":true}

# Stringify a dict into a logfmt line
mq -n 'import "logfmt" | logfmt::logfmt_stringify_line({"level":"info","msg":"hello world","ok":true})'
# => level=info msg="hello world" ok

# Stringify an array of dicts into multiple logfmt lines
mq -n 'import "logfmt" | logfmt::logfmt_stringify([{"level":"info","status":"200"},{"level":"error","fatal":true}])'
# => level=info status=200
# => level=error fatal

# Round-trip: parse then re-stringify
mq -I raw 'import "logfmt" | logfmt::logfmt_stringify(logfmt::logfmt_parse(.))' app.log

Compatibility

Requires mq v0.5 or later.

License

MIT