Skip to content

Conversation

@ternaus
Copy link
Owner

@ternaus ternaus commented Mar 29, 2025

Summary by Sourcery

Enhance docstring parsing with improved type validation and error handling

New Features:

  • Add option to collect type validation errors instead of raising exceptions
  • Improve handling of Returns section validation

Enhancements:

  • Refactor docstring parsing to separate validation and processing logic
  • Add more detailed error messages for type annotation issues
  • Improve nested type validation for collection types

Tests:

  • Add comprehensive test cases for Returns section validation
  • Extend test coverage for type validation edge cases

Chores:

  • Bump package version to 0.0.7

@ternaus ternaus requested a review from Copilot March 29, 2025 00:07
@sourcery-ai
Copy link

sourcery-ai bot commented Mar 29, 2025

Reviewer's Guide by Sourcery

This pull request introduces comprehensive validation for the 'Returns' section in Google-style docstrings, including type annotation checks and handling of 'None' returns. It also enhances type validation to catch bare collection types within nested type annotations. Error handling in parse_google_docstring has been refactored to optionally collect errors instead of immediately raising them.

Sequence diagram for parsing docstring with type validation

sequenceDiagram
  participant Parser as GoogleDocstringParser
  participant Extractor as _extract_sections
  participant DocstringParser as parse
  participant ArgsProcessor as _process_args_with_validation
  participant ReturnsProcessor as _process_returns_with_validation
  participant ReferencesProcessor as _process_references_section

  Parser->Extractor: _extract_sections(docstring)
  Extractor-->Parser: sections
  Parser->DocstringParser: parse(docstring)
  DocstringParser-->Parser: parsed
  Parser->ArgsProcessor: _process_args_with_validation(sections, parsed, result, validate_types, collect_errors)
  ArgsProcessor-->Parser: Updates result with Args
  Parser->ReturnsProcessor: _process_returns_with_validation(sections, result, validate_types, collect_errors)
  ReturnsProcessor-->Parser: Updates result with Returns
  Parser->ReferencesProcessor: _process_references_section(sections, result)
  ReferencesProcessor-->Parser: Updates result with References
  Parser-->Parser: Updates result with other sections
  Parser-->Parser: Removes errors key if no errors
  Parser-->User: Returns result
Loading

Updated class diagram for google_docstring_parser

classDiagram
    class parse_google_docstring {
        +docstring: str
        +validate_types: bool
        +collect_errors: bool
        +parse_google_docstring(docstring: str, validate_types: bool, collect_errors: bool)
        +Returns: dict[str, Any]
    }
    note for parse_google_docstring "Added collect_errors parameter"

    class _process_returns_with_validation {
        +sections: dict[str, str]
        +result: dict[str, Any]
        +validate_types: bool
        +collect_errors: bool
        +_process_returns_with_validation(sections: dict[str, str], result: dict[str, Any], validate_types: bool, collect_errors: bool)
        +Returns: None
    }

    class _validate_type_with_error_handling {
        +type_str: str
        +result: dict[str, Any]
        +collect_errors: bool
        +_validate_type_with_error_handling(type_str: str, result: dict[str, Any], collect_errors: bool)
        +Returns: None
    }

    class _process_args_with_validation {
        +sections: dict[str, str]
        +parsed: Docstring
        +result: dict[str, Any]
        +validate_types: bool
        +collect_errors: bool
        +_process_args_with_validation(sections: dict[str, str], parsed: Docstring, result: dict[str, Any], validate_types: bool, collect_errors: bool)
        +Returns: None
    }

    class _process_references_section {
        +sections: dict[str, str]
        +result: dict[str, Any]
        +_process_references_section(sections: dict[str, str], result: dict[str, Any])
        +Returns: None
    }
Loading

Updated class diagram for type_validation

classDiagram
    class _is_bare_collection_in_nested_type {
        +token: str
        +tokens: list[str]
        +i: int
        +bracket_stack: list[str]
        +_is_bare_collection_in_nested_type(token: str, tokens: list[str], i: int, bracket_stack: list[str])
        +Returns: bool
    }
Loading

File-Level Changes

Change Details Files
Added comprehensive validation for the 'Returns' section in Google-style docstrings, including type annotation checks and handling of 'None' returns.
  • Implemented _process_returns_with_validation to handle 'Returns' section processing with type validation.
  • Added _validate_type_with_error_handling to validate type annotations and handle errors.
  • Modified _process_returns_section to support 'None' returns and return a string 'None' in that case.
  • Updated parse_google_docstring to include collect_errors parameter, allowing errors to be collected instead of raised.
  • Added error handling for invalid return types in check_returns_type in tools/check_docstrings.py.
  • Added tests for validating 'Returns' sections in test_docstring_checker/test_check_docstrings.py.
  • Added tests for checking return types in test_docstring_checker/test_docstring_validation.py.
google_docstring_parser/google_docstring_parser.py
tests/test_docstring_checker/test_check_docstrings.py
tests/test_docstring_checker/test_docstring_validation.py
tools/check_docstrings.py
Enhanced type validation to catch bare collection types within nested type annotations.
  • Added _is_bare_collection_in_nested_type to check for bare collections in nested types.
  • Modified _check_tokens_for_collection_type_usage to use the new function.
  • Added tests for invalid nested types in tests/test_type_validation_edge_cases.py.
google_docstring_parser/type_validation.py
tests/test_type_validation_edge_cases.py
Refactored error handling in parse_google_docstring to optionally collect errors instead of immediately raising them.
  • Added a collect_errors parameter to parse_google_docstring.
  • Initialized an errors list in the result dictionary when collect_errors is True.
  • Appended errors to the errors list instead of raising them when collect_errors is True.
  • Removed the errors key from the result dictionary if no errors were collected.
google_docstring_parser/google_docstring_parser.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a pre-commit check for return type annotations in docstrings and refactors the type validation workflow. Key changes include:

  • Updating check_returns_type in tools/check_docstrings.py to validate the Returns section.
  • Modifying tests to validate error collection instead of raising, and adjusting expected error messages.
  • Refactoring functions in google_docstring_parser/google_docstring_parser.py to include error collection (via a new collect_errors flag) for both argument and returns type validations.

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tools/check_docstrings.py Simplifies check_returns_type and adds a walrus operator for conditional assignment.
tests/test_type_validation_edge_cases.py Adjusts tests to assert error messages instead of expecting exceptions.
tests/test_type_validation.py Updates call signatures to include the collect_errors flag.
tests/test_docstring_checker/test_docstring_validation.py Adds parametrized tests for various Returns section scenarios.
tests/test_docstring_checker/test_check_docstrings.py Tests overall error reporting in the docstring checking tool.
pyproject.toml Bumps the version and updates linting ignores for consistency.
google_docstring_parser/type_validation.py Refines error messaging for bare collection types.
google_docstring_parser/google_docstring_parser.py Refactors several functions to support error collection in type validation.
Comments suppressed due to low confidence (2)

google_docstring_parser/google_docstring_parser.py:375

  • [nitpick] Consider expanding the docstring for _validate_type_with_error_handling to explicitly describe the behavior when collect_errors is True versus when it is False, so users clearly understand whether errors are collected or raised.
def _validate_type_with_error_handling(type_str: str, result: dict[str, Any], collect_errors: bool) -> None:

google_docstring_parser/google_docstring_parser.py:476

  • [nitpick] The function name _process_returns_section is very similar to _process_returns_with_validation; consider renaming one (for example, to _parse_returns_section) to clearly differentiate their responsibilities.
def _process_returns_section(sections: dict[str, str], *, validate_types: bool) -> dict[str, str] | str:

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @ternaus - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider adding a changelog entry to document the new feature and any breaking changes.
  • The introduction of collect_errors adds complexity; ensure its usage is consistent and well-understood throughout the codebase.
Here's what I looked at during the review
  • 🟢 General issues: all looks good
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 1 issue found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@ternaus ternaus merged commit 1110796 into main Mar 29, 2025
17 checks passed
@ternaus ternaus deleted the fix_pre-commit branch March 30, 2025 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants