Release notes

[rn: skip]

What does this PR do?

Adds a gauge to pipeline queue compression metrics to indicate the encode goal

Why is it important/What is the impact to the user?

When a user is analyzing the cost and benefit of queue compression, including the current setting helps contextualize the values.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

1. create an ndjson file example-input.ndjson:

   {"this":"that","banana":["orange",{"elephant":1}]}
   {"event":{"original":"this is an original event"}}
   

2. add my feed utility to your path
3. Invoke Logstash with options that activate the PQ and compression, feeding your input to stdin:

   bin/logstash --log.level=info \
   -Squeue.type=persisted \
   -Squeue.drain=true \
   -Squeue.compression=balanced \
   --config.string 'input { stdin { codec => json_lines } } output { sink {} }' \
   < <(feed example-input.ndjson)
   

4. while that runs, query the HTTP API:

   curl --silent 'localhost:9600/_node/stats?pretty=true' | jq .pipelines.main.queue.compression
   

5. observe that the encode.goal matches the provided queue.compression setting.

Related issues

Pared off of: #18227

Use cases

Screenshots

Logs