Skip to content

Improve log messages for data stream support checks #1109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

edmocosta
Copy link
Contributor

@edmocosta edmocosta commented Jan 12, 2023

What this PR does?

  • Refactored the check_data_stream_config! method to reduce the cyclomatic complexity.
  • Changed the data_stream_default log levels from debug to info. Those messages are helpful to users understanding the data stream auto-configuration mechanism/default and troubleshooting possibly incompatible settings.
  • Logs were being printed twice due to a race condition on the data_stream_config? method, caused by the register function.

Closes #1070

@@ -28,7 +28,6 @@ def self.included(base)
base.extend(Validator)
end

# @note assumes to be running AFTER {after_successful_connection} completed, due ES version checks
Copy link
Contributor Author

@edmocosta edmocosta Jan 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment looks to be true only for the assert_es_version_supports_data_streams method (which does the connected elastic search version validation).
Also, before this PR, there were no guarantees that this method would be invoked only AFTER the @after_successful_connection_thread was completed, the register method was also invoking it, making it prone to race conditions.

@edmocosta edmocosta marked this pull request as ready for review January 12, 2023 17:18
@edmocosta edmocosta marked this pull request as draft January 12, 2023 17:20
@edmocosta edmocosta marked this pull request as ready for review January 12, 2023 17:40
@edmocosta edmocosta requested review from andsel, jsvd and yaauie January 12, 2023 17:40
Copy link
Contributor

@andsel andsel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactoring.
I've left a doubt on the usage of #data_stream_config? because on the method signature is explicitly cited that has to be used after #after_successful_connection is invoked.

Left a couple of suggestions I would like to know your ideas

Inverted the ECS check ifs to cut off the else and use the extracted methods.
@edmocosta edmocosta force-pushed the improve-data-stream-support-messages branch from e761370 to 96f5fc4 Compare January 16, 2023 16:12
@edmocosta edmocosta requested a review from andsel January 16, 2023 17:39
Copy link
Contributor

@andsel andsel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes here.
My major concern in this PR is the lack of testing for the new validation logic.
We should have a few tests describing the new permutations of settings and the validations being added here.

@edmocosta
Copy link
Contributor Author

Thanks for the changes here. My major concern in this PR is the lack of testing for the new validation logic. We should have a few tests describing the new permutations of settings and the validations being added here.

Hey @jsvd, thanks for reviewing! :)

I'm probably missing something, but this PR should not be adding/changing any new validation logic.
Except for the log messages text, It should behave exactly the same and should be already covered by the existing tests.
We did some refactoring on the check_data_stream_config! method, moving code around, and a small change suggested on this PR thread, which should not affect the existing logic.

Could you please provide more details on that?

@jsvd
Copy link
Member

jsvd commented Jan 23, 2023

Fair enough, I just happened to be playing around with the PR and removed the check for settings incompatible with explicitly enabling data streams (here), and didn't cause any tests to fail in the data_stream_support_spec.rb. That said I realize it's not an issue with your refactoring, it was a missing test case already.

Overall, after some extra manual testing I'm good with merging.

Copy link
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@edmocosta edmocosta merged commit 6da4aec into logstash-plugins:main Jan 23, 2023
@edmocosta edmocosta deleted the improve-data-stream-support-messages branch January 23, 2023 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

'action' doesn't default to 'create' (as datastreams is default)
4 participants