move async finish_register to bottom of register to avoid race condition

[jsvd]

This PR fixes a race condition between the thread running register and the one running finish_register.

The issue can be observed by having 20-30 simple pipelines in pipelines.yml like:

input {
 java_generator { eps => 1.1 }
}
output {
  elasticsearch {
    user => elastic
    password => ".."
    data_stream => true
  }

- pipeline.id: test_0
  pipeline.workers: 1
  pipeline.batch.size: 1
  path.config: "/tmp/logstash-8.7.0/cfg"

ruby -ryaml -e '30.times.each {|i| puts "- pipeline.id: test_#{i}\n  pipeline.workers: 1\n  pipeline.batch.size: 1\n  path.config: \"/tmp/logstash-8.7.0/cfg\""}' > config/pipelines.yml

Run Logstash w/ pipeline reloading:

bin/logstash -r

Changing the pipeline config a few times and letting Logstash reload will eventually show one or more pipelines overwriting the index with "ecs-logstash":

[2023-04-20T12:08:33,181][WARN ][logstash.outputs.elasticsearch][test_0] Overwriting supplied index logs-generic-default with rollover alias ecs-logstash

Putting finish_register at the end of register ensures there are no concurrent instructions manipulating @index and other internal state variables.

fixes #1126

[kaisecheng]

LGTM

[andsel]

LGTM

wasn't able to reproduce the original issue on local host, but the code changes seems wise.

[jsvd]

For reliable reproduceability:

/tmp/test/logstash-8.7.0
> bin/logstash-plugin update logstash-output-elasticsearch
> vim ./vendor/bundle/jruby/2.6.0/gems/logstash-output-elasticsearch-11.15.0-java/lib/logstash/outputs/elasticsearch.rb

1. Add sleep 1 before the if on line https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/main/lib/logstash/outputs/elasticsearch.rb#L341
2. Change line https://github.com/logstash-plugins/logstash-output-elasticsearch/blob/main/lib/logstash/outputs/elasticsearch.rb#L364 to (sleep 3; setup_ilm) if ilm_in_use?

[andsel]

@jsvd tested as you suggested and it was reproducible, confirming that the PR fixes it.

[jsvd]

@logstashmachine bump patch