kubernetes#container_logs: update docs for GZIP promoted to GA on 9.3

[AndersonQ]

Proposed commit message

    kubernetes#container_logs: update docs on ingesting rotated log
    
    Update the container logs docs on ingesting rotated logs, including
    GZIP-compressed logs, with the new `compression` configuration, for 9.3+,  and
    mark `gzip_experimental` as for 9.2.x.

Checklist

• [ ] I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• [ ] I have verified that Kibana version constraints are current according to guidelines.
• ~~[ ] I have verified that any added dashboard complies with Kibana's Dashboard good practices ~~

Author's Checklist

• check the docs are rendered as expected

How to test this PR locally

• Create an elastic stack
• setup a kind cluster

kind create cluster

• start some flog containers:

kubectl apply -f ./flog.yaml

flog.yaml:

apiVersion: batch/v1
kind: Job
metadata:
  name: flog-log-generator
spec:
  template:
    spec:
      containers:
        - name: flog-unstructured-cont-rot
          image: mingrammer/flog
          #          too small "-d" won't give kubelet time to rotate the files
          args: ["-t", "stdout", "-d", "1us", "-l"]
      restartPolicy: OnFailure
  backoffLimit: 10

• build, then install the integration

cd packages/kubernetes
elastic-package build -v
elastic-package install -v

• grab the agent curl -O -L https://snapshots.elastic.co/9.3.0-40d73df7/downloads/beats/elastic-agent/elastic-agent-9.3.0-SNAPSHOT-linux-x86_64.tar.gz
• extract tar -xf elastic-agent-9.3.0-SNAPSHOT-linux-x86_64.tar.gz
• deploy the agent to the cluster following the instructions in kibana, but use the package you djust downloaded. If you see the error InvalidImageName, check here to see hot to fix it
• following the new instructions from this PR, add the k8s integration from the fleet UI
• check all is normal
• ensure logs are being ingested. use filter log.file.path : *flog*. Ensure the logs come from /var/log/pods/ and there are files with .log.gz extension

Related issues

• Closes [kubernetes] Adjust kubernetes integration for new compression config #16221

## Screenshots

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

Package kubernetes 👍(1) 💚(0) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
container_logs	250000	200000	-50000 (-20%)	💔

To see the full report comment with /test benchmark fullreport

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[pierrehilbert]

cc @bturquet

[bturquet]

@gizas can you hep with reviewing the PR please ?

[gizas]

Wondering if version: "^8.15.0 || ^9.3.0" and update the above values as per comment https://github.com/elastic/integrations/pull/16581/files#r2627363467

[AndersonQ]

We haven't done that for filestream. The idea is the integration still works with older versions and making it would basically start to bound a integration version to a specific release. It'd make difficult to add changes that are backwards compatible.

[gizas]

Indeed. Then you should have marked it as Breaking change in this case. So users to be aware that something might break in case of upgrade.

[AndersonQ]

there is no breaking change. If they upgrade nothing will happen. It's just updating the docs guiding them if they want to ingest rotated logs.

[gizas]

NW I had in mid that you would have changed the default configuration and pin a specific version to be availalble. But now is clear

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: db0e11c

History

• 💚 Build #35618 succeeded b7a00d8
• 💚 Build #35577 succeeded cbf67d4
• 💔 Build #35575 failed 3ac45a2

cc @AndersonQ

[elastic-vault-github-plugin-prod]

Package kubernetes - 1.83.0 containing this change is available at https://epr.elastic.co/package/kubernetes/1.83.0/