Skip to content

Add cursor abstraction over search result pagination #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jodal merged 3 commits into from
Jan 29, 2024
Merged

Add cursor abstraction over search result pagination #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1c22ffd
Add Cursor class to make result pagination easier
jodal Jan 28, 2024
1d2dcf7
Add search and cursor usage to the quickstart guide
jodal Jan 28, 2024
edd80f0
Update copyright notices
jodal Jan 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ the parts of the API that there is an actual demand for.

Copyright
2019 [Otovo ASA](https://www.otovo.com/),
2023 [Cardboard AS](https://cardboard.inc/).
2023-2024 [Cardboard AS](https://cardboard.inc/).

Licensed under the
[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
11 changes: 9 additions & 2 deletions docs/api.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,12 @@ Query objects
:members:
:exclude-members: model_config, model_fields

Response objects
----------------
Pagination objects
------------------

.. autoclass:: brreg.enhetsregisteret.Cursor
:members:
:exclude-members: model_config, model_fields

.. autoclass:: brreg.enhetsregisteret.Page
:members:
Expand All @@ -58,6 +62,9 @@ Response objects
:members:
:exclude-members: model_config, model_fields

Response objects
----------------

.. autoclass:: brreg.enhetsregisteret.Enhet
:members:
:exclude-members: model_config, model_fields
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
"""Sphinx configuration file."""

project = "python-brreg"
author = "Stein Magnus Jodal"
copyright = f"2019 Otovo ASA, 2023 {author}" # noqa: A001
author = "Cardboard AS"
copyright = f"2019 Otovo ASA, 2023-2024 {author}" # noqa: A001

extensions = [
"sphinx.ext.autodoc",
Expand Down
6 changes: 3 additions & 3 deletions docs/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,9 @@ Project resources
License
=======

``python-brreg`` is copyright
Copyright
2019 `Otovo ASA <https://www.otovo.com/>`_,
2023 Stein Magnus Jodal and contributors.
2023-2024 `Cardboard AS <https://cardboard.inc/>`_.

``python-brreg`` is licensed under the
Licensed under the
`Apache License, Version 2.0 <https://www.apache.org/licenses/LICENSE-2.0>`_.
83 changes: 68 additions & 15 deletions docs/quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,94 @@
Quickstart
==========

The following examples takes you through typical use cases with the
:mod:`brreg` library.
The following examples takes you through typical use cases, showing how to use
the :mod:`brreg` library to query Enhetsregisteret.


Querying Enhetsregisteret
=========================
Create a client
===============

To query Enhetsregisteret, you need to create a :class:`brreg.enhetsregisteret.Client` instance:
To query Enhetsregisteret, you need to create a
:class:`brreg.enhetsregisteret.Client` instance:

>>> from brreg.enhetsregisteret import Client
>>> client = Client()
>>>

The client instance will ensure that the HTTP connection is reused across requests.
The client instance will ensure that the HTTP connection is reused across
requests.


Details about a specific organization
=====================================

To get details about an organization ("enhet") given its organization number:

>>> enhet = client.get_enhet('915501680')
>>> enhet = client.get_enhet('930070556')
>>> enhet.organisasjonsnummer
'915501680'
'930070556'
>>> enhet.navn
'OTOVO ASA'
'CARDBOARD AS'
>>> enhet.organisasjonsform
Organisasjonsform(kode='ASA', beskrivelse='Allmennaksjeselskap')
Organisasjonsform(kode='AS', beskrivelse='Aksjeselskap', utgaatt=None)
>>> enhet.forretningsadresse
Adresse(adresse=['Torggata 7'], postnummer='0181', poststed='OSLO', kommunenummer='0301', kommune='OSLO', landkode='NO', land='Norge')
Adresse(adresse=['Grensen 13'], postnummer='0159', poststed='OSLO', kommunenummer='0301', kommune='OSLO', landkode='NO', land='Norge')
>>>

To get details of a suborganization ("underenhet") given its organization number:

>>> underenhet = client.get_underenhet('915659683')
>>> underenhet = client.get_underenhet('930090069')
>>> underenhet.organisasjonsnummer
'915659683'
'930090069'
>>> underenhet.antall_ansatte
91
5
>>> underenhet.beliggenhetsadresse
Adresse(adresse=['Torggata 7'], postnummer='0181', poststed='OSLO', kommunenummer='0301', kommune='OSLO', landkode='NO', land='Norge')
Adresse(adresse=['Grensen 13'], postnummer='0159', poststed='OSLO', kommunenummer='0301', kommune='OSLO', landkode='NO', land='Norge')
>>>

To get details of roles ("roller") given an organization number:

>>> rollegrupper = client.get_roller('930070556')
>>> [rg.type.beskrivelse for rg in rollegrupper]
['Daglig leder/ adm.direktør', 'Styre', 'Regnskapsfører']
>>> rollegrupper[2].roller[0].enhet
RolleEnhet(organisasjonsnummer='914549140', organisasjonsform=Organisasjonsform(kode='AS', beskrivelse='Aksjeselskap', utgaatt=None), navn=['SYNEGA REGNSKAP AS'], er_slettet=False)
>>>


Searching for organizations
===========================

To search for organizations ("enheter"):

>>> from brreg.enhetsregisteret import EnhetQuery
>>> cursor = client.search_enhet(EnhetQuery(navn='cardboard'))
>>>

The search result is paginated, which you can see by looking at the available page numbers:

>>> list(cursor.page_numbers)
[0]
>>>

The cursor has two iterators, ``cursor.pages`` to iterate over the pages:

>>> page = next(cursor.pages)
>>> page.page_number
0
>>> page.total_items
2
>>> page.total_pages
1
>>> [enhet.navn for enhet in page.items]
['CARDBOARD AS', 'RECYCLING AND TRADING OF SCRAP METAL AND CARDBOARD PAPER V/ABIKAR']
>>>

And ``cursor.items`` to iterate over all items in all pages:

>>> [enhet.navn for enhet in cursor.items]
['CARDBOARD AS', 'RECYCLING AND TRADING OF SCRAP METAL AND CARDBOARD PAPER V/ABIKAR']
>>>

As long as you keep using the same cursor object each page is fetched only once,
no matter how many times you iterate over the pages or items.
3 changes: 2 additions & 1 deletion src/brreg/enhetsregisteret/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""

from brreg.enhetsregisteret._client import Client
from brreg.enhetsregisteret._pagination import EnhetPage, Page, UnderenhetPage
from brreg.enhetsregisteret._pagination import Cursor, EnhetPage, Page, UnderenhetPage
from brreg.enhetsregisteret._queries import EnhetQuery, Query, UnderenhetQuery
from brreg.enhetsregisteret._responses import (
Adresse,
Expand Down Expand Up @@ -38,6 +38,7 @@
# From _client module:
"Client",
# From _pagination module:
"Cursor",
"EnhetPage",
"Page",
"UnderenhetPage",
Expand Down
12 changes: 7 additions & 5 deletions src/brreg/enhetsregisteret/_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import httpx

from brreg import BrregError, BrregRestError
from brreg.enhetsregisteret._pagination import EnhetPage, UnderenhetPage
from brreg.enhetsregisteret._pagination import Cursor, EnhetPage, UnderenhetPage
from brreg.enhetsregisteret._responses import (
Enhet,
RolleGruppe,
Expand Down Expand Up @@ -141,7 +141,7 @@ def get_roller(
def search_enhet(
self,
query: EnhetQuery,
) -> EnhetPage:
) -> Cursor[Enhet, EnhetQuery]:
"""Search for :class:`Enhet` that matches the given query.

:param query: The search query.
Expand All @@ -157,12 +157,13 @@ def search_enhet(
},
)
res.raise_for_status()
return EnhetPage.model_validate_json(res.content)
page = EnhetPage.model_validate_json(res.content)
return Cursor(self.search_enhet, query, page)

def search_underenhet(
self,
query: UnderenhetQuery,
) -> UnderenhetPage:
) -> Cursor[Underenhet, UnderenhetQuery]:
"""Search for :class:`Underenhet` that matches the given query.

:param query: The search query.
Expand All @@ -178,7 +179,8 @@ def search_underenhet(
},
)
res.raise_for_status()
return UnderenhetPage.model_validate_json(res.content)
page = UnderenhetPage.model_validate_json(res.content)
return Cursor(self.search_underenhet, query, page)


@contextmanager
Expand Down
67 changes: 66 additions & 1 deletion src/brreg/enhetsregisteret/_pagination.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
from typing import Generic, List, TypeVar
from __future__ import annotations

from typing import (
Callable,
Dict,
Generic,
Iterator,
List,
TypeVar,
)

from pydantic import (
AliasPath,
BaseModel,
Field,
)

from brreg.enhetsregisteret._queries import Query
from brreg.enhetsregisteret._responses import Enhet, Underenhet

__all__ = [
Expand All @@ -15,6 +25,7 @@


T = TypeVar("T", bound=BaseModel)
Q = TypeVar("Q", bound=Query)


class Page(BaseModel, Generic[T]):
Expand Down Expand Up @@ -44,6 +55,60 @@ class Page(BaseModel, Generic[T]):
)


class Cursor(Generic[T, Q]):
"""Cursor for iterating over multiple pages of items."""

_operation: Callable[[Q], Cursor[T, Q]]
_query: Q
_pages: Dict[int, Page[T]]
_current_page_number: int

#: Iterate over all page numbers in this cursor.
page_numbers: range

def __init__(
self,
operation: Callable[[Q], Cursor[T, Q]],
query: Q,
page: Page[T],
) -> None:
self._operation = operation
self._query = query
self._pages = {page.page_number: page}
# Expose the empty first page, even if it says the totalt number of pages is 0.
self.page_numbers = range(max(1, page.total_pages))

def get_page(self, page_number: int) -> Page[T] | None:
"""Get a page by its 0-indexed page number."""
if page_number not in self.page_numbers:
return None

if page_number not in self._pages:
# We need to fetch the page.
new_cursor = self._operation(
self._query.model_copy(update={"page": page_number})
)
new_page = new_cursor.get_page(page_number)
assert new_page is not None
self._pages[page_number] = new_page

return self._pages[page_number]

@property
def pages(self) -> Iterator[Page[T]]:
"""Iterator over all pages in this cursor."""
for page_number in self.page_numbers:
page = self.get_page(page_number)
assert page is not None
yield page

@property
def items(self) -> Iterator[T]:
"""Iterator over all items in this cursor."""
for page in self.pages:
yield from page.items


class EnhetPage(Page[Enhet]):
"""Response type for enhet search."""

Expand Down
Loading