Adds a new dlq_custom_codes option to customize DLQ codes

[andsel]

Release notes

Adds dlq_custom_codes option to customize the list of HTTP codes that are considered eligible for DLQ.

What does this PR do?

Introduced a new configuration option, named dlq_custom_codes which accepts a list of HTTP error codes.
The list is to be considered as an addition to the already existing codes that the plugin uses (DLQ_CODES, DOC_SUCCESS_CODES, DOC_CONFLICT_CODE defined in module Common). If the list has an intersection with those error codes then it's reported as a configuration error.

Why is it important/What is the impact to the user?

This is a feature requested in #697, in some cases some error codes returned by Elasticsearch should be handled as DLQ messaged and not retried for ever. For example writing to a read only index report a 403, the index status would never change automatically so it's best to move these events to DLQ for a later reprocessing.
Under normal conditions the user shouldn't customize this list, but it's an helper in some circumstances.

Checklist

• My code follows the style guidelines of this project
• [ ] I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• test a version of the plugin with a real ES

How to test this PR locally

• launch an ES instance, for example docker-compose -f es_and_kb.yml up
  es_and_kb.txt
• from http://localhost:5601 Kibana UI (credential: elastic, changeme) create and index and put into read only:

PUT /test_index
PUT test_index/_settings 
{ 
  "index.blocks.read_only" : true 
}

• configure a pipeline to write into:

input {
	stdin {}
}

output {
	elasticsearch {
		index => "test_index"
		hosts => "http://localhost:9200"
		user => "elastic"
		password => "changeme"
		dlq_custom_codes => [403]
	}
}	

• enable DLQ in config/logstash.yml :

dead_letter_queue.enable: true

• start the pipeline and generate an input and stop
• check that the string you typed is present in DLQ

strings data/dead_letter_queue/main/1.log

Related issues

• Fixes Make 403 response configurable for going into DLQ #697
• Related [Meta, 8.4, 8.5] Logstash DLQ Improvements for 8.x elastic/logstash#10493

Use cases

HTTP error codes used to decide if an event index reply has to be moved into the DLQ are 400 and 404. These codes are hard coded, this PR aims the user to customize this list extending other than 400 and 404.

[yaauie]

It looks like we already handle 200/201 as explicit success, 400/404 as DLQ, and 409 as conflict, and are declaring it a configuration error to re-use any of these.

If a user can't use this to intercept 409 conflicts, or 404's attempting to update records that don't exist, what error codes do we expect? Only authorization-related ones like 403? Others? Do we have real-world use-cases driving this feature?

[andsel]

This PR aim to solve request #697 which explicitly state that 403 error code could be eventually added as a DLQ trigger code.
While intersection with existing DLQ codes (400 and 404) could be seen as not necessarily a conflict, because both trigger the same feature, using error codes already configured as success (200/201) document conflict error (409) generates confusion in the precedence order when the code process responses.

Suppose that also 409 is customized as a dlq_custom_code what should take precedence? Processing it as a failed action or as DLQ event?

To keep the things simple I imagined to keep out of the set the codes that already has a meaning in the plugin.

[roaksoax]

@yaauie seems that this needs your feedback?

[yaauie]

Yes. I can unblock this PR.

We can consider a later feature to enable this explicit custom-code routing to also intercept 400/404/409 that are already routed by the implementation, and if and when we do so we have a path to do it in a non-breaking way.

[kaisecheng]

Once the CI is fixed, I can approve.

[kaisecheng]

LGTM. CI red due to missing 7.17.7-SNAPSHOT docker image

[yaauie]

LGTM 👍🏼

[yaauie]

Yes. I can unblock this PR.

We can consider a later feature to enable this explicit custom-code routing to also intercept 400/404/409 that are already routed by the implementation, and if and when we do so we have a path to do it in a non-breaking way.

[yaauie]

I've merged in upstream and resolved conflicting version bump in changelog/gemspec, and will merge/release on successful CI.