Skip to content

Deprecates option in favor of silence_errors_in_log #1068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

andsel
Copy link
Contributor

@andsel andsel commented Feb 17, 2022

Release notes

Deprecates failure_type_logging_whitelist setting renaming it silence_errors_in_log

What does this PR do?

Introduce a new setting named silence_errors_in_log with exact same behavior of failure_type_logging_whitelist. It print a deprecation log message if failure_type_logging_whitelist is used.

Why is it important/What is the impact to the user?

Rename a setting that's generally confusing the user to a more meaningful name.

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • Run against a real ES instance

How to test this PR locally

  • launch an ES instance, for example docker-compose -f es_and_kb.yml up
    es_and_kb.txt
  • from http://localhost:5601 Kibana UI (credential: elastic, changeme) create and index and put into read only:
PUT /test_index
PUT test_index/_settings 
{ 
  "index.blocks.read_only" : true 
}
  • configure a pipeline to write into:
input {
	generator {
		message => '{"name": "John", "surname": "Doe"}'
		count => 1
		codec => json
	}
}

output {
	elasticsearch {
               # failure_type_logging_whitelist => ["cluster_block_exception"]
	       # silence_errors_in_log => ["cluster_block_exception"]
		index => "test_index"
		hosts => "http://localhost:9200"
		user => "elastic"
		password => "changeme"
		dlq_custom_codes => [403]
	}
}	
  • enable the failure_type_logging_whitelist and check that a deprecation log is present in logs/logstash-deprecation.log. After the deprecation log the work as expected and doesn't log the content of the 403 responses from Elasticsearch
  • comment failure_type_logging_whitelist and decomment the silence_errors_in_log and verify the feature work as expected (no log of error responses)

Related issues

Use cases

Screenshots

Logs

@andsel andsel added the on hold label Feb 17, 2022
@andsel andsel changed the title Deprecates option in favor of Deprecates option in favor of log_silenced_errors Feb 17, 2022
@andsel andsel force-pushed the feature/rename_whitelist_log_errors_options branch from 95374ff to cf4c630 Compare August 18, 2022 12:46
@andsel andsel marked this pull request as ready for review August 25, 2022 13:57
Copy link
Contributor

@kaisecheng kaisecheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original name failure_type_logging_whitelist is not too bad and is widely adopted. I am not sure whether the rename gives a better value to users. The bottom line is it is a better name

[source,ruby]
output {
elasticsearch {
log_silenced_errors => ["document_already_exists_exception"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit hard for me to understand log_silenced_errors means not to log xxx error. Maybe silence_error_in_log?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sound better, maybe silenced_errors_in_log ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds better. let's see how @karenzone judge on silence_errors_in_log or silenced_errors_in_log

@andsel andsel requested a review from kaisecheng August 26, 2022 12:19
@andsel andsel changed the title Deprecates option in favor of log_silenced_errors Deprecates option in favor of silenced_errors_in_log Aug 26, 2022
@andsel andsel changed the title Deprecates option in favor of silenced_errors_in_log Deprecates option in favor of silence_errors_in_log Aug 26, 2022
@andsel andsel requested a review from kaisecheng August 29, 2022 06:49
Copy link
Contributor

@kaisecheng kaisecheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Glad to see the example in doc and thanks for changing the name

@andsel andsel merged commit f697c20 into logstash-plugins:main Aug 30, 2022
@@ -332,6 +332,7 @@ This plugin supports the following configuration options plus the
| <<plugins-{type}s-{plugin}-index>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-keystore>> |a valid filesystem path|No
| <<plugins-{type}s-{plugin}-keystore_password>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-silence_errors_in_log>> |<<array,array>>|No
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be in alphabetical order

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use correct deprecation format throughout

Set the Elasticsearch errors in the whitelist that you don't want to log.
A useful example is when you want to skip all 409 errors
which are `document_already_exists_exception`.
NOTE: Deprecated, refer to <<plugins-{type}s-{plugin}-silence_errors_in_log>>.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use correct deprecation format

}
}

NOTE: Deprecates <<plugins-{type}s-{plugin}-failure_type_logging_whitelist>>.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use deprecation format

@karenzone
Copy link
Contributor

@kilfoyle and I will make these changes and create a PR as part of a knowledge share exercise. @andsel, we'll tag you for review when we're ready.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants