apiary

package module
v0.0.0-...-666a001 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 12, 2025 License: MIT Imports: 19 Imported by: 0

README

Apiary: The RRCHNM Data API

This repository provides an API to access data stored in a PostgreSQL database. It is a component of American Religious Ecologies, America's Public Bible, Death by Numbers and other projects at the Roy Rosenzweig Center for History and New Media. The API is intended for use by RRCHNM projects and is not general purpose, but we provide the source code in case it is useful. You can read more about the rationale for this piece of RRCHNM's infrastructure on our website.

Documentation

Documentation for the various endpoints and parameters to them can be found on the Go package website. Handlers are all methods on the main server type, so you can find any endpoint specific documentation on that type.

Note that the root of the API lists out all the endpoints and sample URLs, and may be more useful than the documentation for understanding how to use the API.

Configuration

Set the following environment variables to configure the server:

  • APIARY_DB (default: none). A connection string to the database. An example connection string looks like this: postgres://username:password@host:portnum/databasename.
  • APIARY_INTERFACE (default: 0.0.0.0). The interface to serve the API on.
  • APIARY_PORT (default: 8090). The port to serve the API on.
  • APIARY_LOGGING (default: on). If logging is on, then access logs will be written to stdout in the Apache Common Log format. Errors and status messages will always be written to stderr.

Compiling or running a container

There is a Makefile in the root of the repository that can be used for compiling and for running the service locally.

  • make build will build the binary.
  • make install will build the binary and install it under the name apiary to your $GOPATH.
  • make serve will serve the API locally.
  • make docker-build will create a Docker container for the API.
  • make docker-serve will create the container and run it locally via Docker.
  • make serve-ghcr. If you just need to run the Data API locally, it may be most convenient to just run a Docker container served from the GitHub Container Registry. There are versions that are tagged with each of the GitHub branches that have been pushed, so that you can try the development version from your current branch. You still need to set the environment variables.

Testing

You can run make test to run integration tests with the database. Or run go test -v ./... to get more verbose tests.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsValidBillType 

func IsValidBillType(billType string) bool

IsValidBillType checks if the provided bill type is valid

func IsValidCountType 

func IsValidCountType(countType string) bool

IsValidCountType checks if the provided count type is valid

Types

type APBIndexItem 

type APBIndexItem struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
	Count     int    `json:"count"`
}

APBIndexItem is an entry in one of the different indexes to verses

type APBIndexItemText 

type APBIndexItemText struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
}

APBIndexItemText is an entry in one of the different indexes to verses, with the reference and the text of the verse.

type APBIndexItemWithYear 

type APBIndexItemWithYear struct {
	Reference string `json:"reference"`
	Text      string `json:"text"`
	Count     int    `json:"count"`
	Peak      int    `json:"peak"`
}

APBIndexItemWithYear is an index item with the peak year

type APIParameters 

type APIParameters struct {
	StartYear  int
	EndYear    int
	StartWeek  int
	EndWeek    int
	Parish     []int
	BillType   string
	CountType  string
	Missing    *bool
	Illegible  *bool
	Sort       string
	Limit      int
	Offset     int
	Page       int
	Cursor     string
	CursorYear int
	CursorWeek int
	CursorName string
}

func (*APIParameters) GetQueryOptions 

func (p *APIParameters) GetQueryOptions() QueryOptions

type BibleBook 

type BibleBook struct {
	Book  string `json:"book"`
	Part  string `json:"part"`
	Order int    `json:"order"`
}

BibleBook describes a book of the Bible and which part of the Bible it is in.

type BibleSimilarityEdge 

type BibleSimilarityEdge struct {
	A string `json:"source"`
	B string `json:"target"`
	N int    `json:"n"`
}

BibleSimilarityEdge describes an edge between two parts of the Bible

type CatholicDiocese 

type CatholicDiocese struct {
	City             string    `json:"city"`
	State            string    `json:"state"`
	Country          string    `json:"country"`
	Rite             string    `json:"rite"`
	YearErected      int64     `json:"year_erected"`
	YearMetropolitan NullInt64 `json:"year_metropolitan"`
	YearDestroyed    NullInt64 `json:"year_destroyed"`
	Lon              float32   `json:"lon"`
	Lat              float32   `json:"lat"`
}

CatholicDiocese describes a diocese of the Roman Catholic Church.

type CatholicDiocesesPerDecade 

type CatholicDiocesesPerDecade struct {
	Decade int64 `json:"decade"`
	Count  int64 `json:"n"`
}

CatholicDiocesesPerDecade shows how many dioceses were established in North America per year.

type Causes 

type Causes struct {
	Name string `json:"name"`
}

Causes describes a cause of death.

type Christenings 

type Christenings struct {
	Name string `json:"name"`
	ID   int    `json:"id"`
}

Christenings describes a christening location.

type ChristeningsByYear 

type ChristeningsByYear struct {
	Christening  string     `json:"christening"`
	TotalCount   NullInt64  `json:"count"`
	WeekNumber   NullInt64  `json:"week_number"`
	StartDay     NullInt64  `json:"start_day"`
	StartMonth   NullString `json:"start_month"`
	EndDay       NullInt64  `json:"end_day"`
	EndMonth     NullString `json:"end_month"`
	Year         int        `json:"year"`
	BillType     NullString `json:"bill_type"`
	SplitYear    string     `json:"split_year"`
	TotalRecords int        `json:"totalrecords"`
}

ChristeningsByYear describes a christening's description, total count, week number, week ID, year, and bill type.

type CityMembership 

type CityMembership struct {
	Year           int       `json:"year"`
	Group          string    `json:"group"`
	City           string    `json:"city"`
	State          string    `json:"state"`
	Denominations  int       `json:"denominations"`
	Churches       int       `json:"churches"`
	Members        int       `json:"members"`
	Population1926 NullInt64 `json:"population_1926"`
	Lon            float64   `json:"lon"`
	Lat            float64   `json:"lat"`
}

CityMembership gives the membership (and population) statistics for some aggregation of denominations in a given year.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

The Config type stores configuration which is read from environment variables.

type DeathCauses 

type DeathCauses struct {
	Death            string     `json:"death"`
	BillType         string     `json:"bill_type"`
	Count            NullInt64  `json:"count"`
	Definition       NullString `json:"definition"`
	DefinitionSource NullString `json:"definition_source"`
	WeekID           string     `json:"week_id"`
	WeekNumber       NullInt64  `json:"week_number"`
	StartDay         NullInt64  `json:"start_day"`
	StartMonth       NullString `json:"start_month"`
	EndDay           NullInt64  `json:"end_day"`
	EndMonth         NullString `json:"end_month"`
	Year             NullInt64  `json:"year"`
	SplitYear        NullString `json:"split_year"`
	TotalRecords     int        `json:"totalrecords"`
}

DeathCauses returns a list of causes of death with a count of deaths for each cause and related metadata.

type DeathsAPIParameters 

type DeathsAPIParameters struct {
	StartYear int
	EndYear   int
	Death     []string
	Sort      string
}

type Denomination 

type Denomination struct {
	Name           string     `json:"name"`
	ShortName      string     `json:"short_name"`
	DenominationID NullString `json:"denomination_id"`
	FamilyCensus   NullString `json:"family_census"`
	FamilyRelec    string     `json:"family_relec"`
}

Denomination describes a denomination's names and various systems of classification.

type DenominationFamily 

type DenominationFamily struct {
	Name string `json:"name"`
}

DenominationFamily describes a group of denominations. There can be different ways of categorizing denominations.

type Endpoint 

type Endpoint struct {
	Name     string       `json:"name"`
	URL      string       `json:"path"`
	Examples []ExampleURL `json:"examples,omitempty"`
}

Endpoint describes an endpoint available in this API and provides a sample path.

type ExampleURL 

type ExampleURL struct {
	URL     string `json:"url"`
	Purpose string `json:"purpose"`
}

ExampleURL provides an example URL to a different way of querying the API for any given endpoint.

type LocationInfo 

type LocationInfo struct {
	PlaceID    int     `json:"place_id"`
	City       string  `json:"city"`
	County     string  `json:"county"`
	State      string  `json:"state"`
	CountyAHCB string  `json:"county_ahcb"`
	MapName    string  `json:"map_name"`
	Lon        float64 `json:"lon"`
	Lat        float64 `json:"lat"`
}

LocationInfo provides basic information about cities, counties, and states for use in dropdowns and other UI elements.

type NullInt64 

type NullInt64 struct {
	sql.NullInt64
}

NullInt64 embeds the sql.NullInt64 type, so that it can be extended to change the JSON marshaling.

func (NullInt64) MarshalJSON 

func (v NullInt64) MarshalJSON() ([]byte, error)

MarshalJSON marshals a null integer as `{"int": null}` rather than using an object inside the field. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

func (*NullInt64) UnmarshalJSON 

func (v *NullInt64) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals a null integer represented in JSON as `{"int": null}` into our embedded type that allows nulls. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

type NullString 

type NullString struct {
	sql.NullString
}

NullString embeds the sql.NullString type, so that it can be extended to change the JSON marshaling.

func (NullString) MarshalJSON 

func (v NullString) MarshalJSON() ([]byte, error)

MarshalJSON marshals a null string as `{"string": null}` rather than using an object inside the field. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

func (*NullString) UnmarshalJSON 

func (v *NullString) UnmarshalJSON(data []byte) error

UnmarshalJSON unmarshals a null string represented in JSON as `{"string": null}` into our embedded type that allows nulls. See https://stackoverflow.com/questions/33072172/how-can-i-work-with-sql-null-values-and-json-in-a-good-way

type PaginatedResponse 

type PaginatedResponse struct {
	Data       []ParishByYear `json:"data"`
	NextCursor *string        `json:"next_cursor,omitempty"`
	HasMore    bool           `json:"has_more"`
}

type Parish 

type Parish struct {
	ParishID       int        `json:"id"`
	Name           string     `json:"name"`
	CanonicalName  string     `json:"canonical_name"`
	BillSubunit    NullString `json:"subunit"`
	FoundationYear NullString `json:"foundation_year"`
	Notes          NullString `json:"notes"`
}

Parish describes a parish name, canonical name, and unique ID.

type ParishByYear 

type ParishByYear struct {
	CanonicalName    string     `json:"name"`
	BillType         string     `json:"bill_type"`
	CountType        string     `json:"count_type"`
	Count            NullInt64  `json:"count"`
	StartDay         NullInt64  `json:"start_day"`
	StartMonth       NullString `json:"start_month"`
	EndDay           NullInt64  `json:"end_day"`
	EndMonth         NullString `json:"end_month"`
	Year             NullInt64  `json:"year"`
	SplitYear        string     `json:"split_year"`
	WeekNumber       int        `json:"week_number"`
	WeekID           string     `json:"week_id"`
	Missing          *bool      `json:"missing"`
	Illegible        *bool      `json:"illegible"`
	Source           NullString `json:"source"`
	UniqueIdentifier NullString `json:"unique_identifier"`
	TotalRecords     int        `json:"totalrecords"`
	Parish           *Parish    `json:"parish,omitempty"`
}

ParishByYear describes a parish's canoncial name, count type, total count, start day, start month, end day, end month, year, week number, and week ID.

type ParishYearlySummary 

type ParishYearlySummary struct {
	Year        int    `json:"year"`
	ParishName  string `json:"parish_name"`
	TotalBuried int    `json:"total_buried"`
	TotalPlague *int   `json:"total_plague"`
}

ParishYearlySummary represents total counts by parish and year for small multiple visualizations

type Place 

type Place struct {
	PlaceID int    `json:"place_id"`
	Place   string `json:"place"`
	MapName string `json:"map_name"`
}

Place represents a place with ID and name

type PlaceCounty 

type PlaceCounty struct {
	CountyAHCB string `json:"county_ahcb"`
	County     string `json:"name"`
}

PlaceCounty represents a county with ID and name

type PlaceDetails 

type PlaceDetails struct {
	PlaceID    int    `json:"place_id"`
	Place      string `json:"place"`
	MapName    string `json:"map_name"`
	County     string `json:"county"`
	CountyAHCB string `json:"county_ahcb"`
	State      string `json:"state"`
}

PlaceDetails represents details about a place

type PresbyteriansByYear 

type PresbyteriansByYear struct {
	Year     int `json:"year"`
	Members  int `json:"members"`
	Churches int `json:"churches"`
}

PresbyteriansByYear holds aggregate data on Presbyterian membership and churches.

type QueryBuilder 

type QueryBuilder struct {
	Query  string
	Params []interface{}
}

QueryBuilder holds the query string and parameters

func NewQueryBuilder 

func NewQueryBuilder() *QueryBuilder

NewQueryBuilder creates a new query builder

func (*QueryBuilder) AddParam 

func (qb *QueryBuilder) AddParam(value interface{}) string

AddParam adds a parameter and returns the placeholder ($1, $2, etc.) We do this to prevent injection problems.

type QueryOptions 

type QueryOptions struct {
	Limit  int
	Offset int
}

type Server 

type Server struct {
	Server *http.Server
	DB     *pgxpool.Pool
	Router *mux.Router
	Config Config
	Cache  *cache.Client
}

The Server type shares access to the database.

func NewServer 

func NewServer(ctx context.Context) *Server

NewServer creates a new Server and connects to the database or fails trying.

func (*Server) AHCBCountiesByIDHandler 

func (s *Server) AHCBCountiesByIDHandler() http.HandlerFunc

AHCBCountiesByIDHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by county ID (or IDs if given a comma-separated string of values).

func (*Server) AHCBCountiesByStateCodeHandler 

func (s *Server) AHCBCountiesByStateCodeHandler() http.HandlerFunc

AHCBCountiesByStateCodeHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by state code (or state codes if given a comma-separated string of values).

func (*Server) AHCBCountiesByStateTerrIDHandler 

func (s *Server) AHCBCountiesByStateTerrIDHandler() http.HandlerFunc

AHCBCountiesByStateTerrIDHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date and by state/territory ID (or IDs if given a comma-separated string of values).

func (*Server) AHCBCountiesHandler 

func (s *Server) AHCBCountiesHandler() http.HandlerFunc

AHCBCountiesHandler returns a GeoJSON FeatureCollection containing counties from AHCB. The handler will get the county boundaries for a particular date.

func (*Server) AHCBStatesHandler 

func (s *Server) AHCBStatesHandler() http.HandlerFunc

AHCBStatesHandler returns a GeoJSON FeatureCollection containing states from AHCB. The handler will get the county boundaries for a particular date.

func (*Server) APBBibleBooksHandler 

func (s *Server) APBBibleBooksHandler() http.HandlerFunc

APBBibleBooksHandler returns the books of the Bible (in the KJV).

func (*Server) APBBibleSimilarityHandler 

func (s *Server) APBBibleSimilarityHandler() http.HandlerFunc

APBBibleSimilarityHandler returns the information about the network of similarities within the Bible.

func (*Server) APBBibleTrendHandler 

func (s *Server) APBBibleTrendHandler() http.HandlerFunc

APBBibleTrendHandler returns the rates of quotation per year for a verse.

func (*Server) APBIndexAllHandler 

func (s *Server) APBIndexAllHandler() http.HandlerFunc

APBIndexAllHandler returns basically all available verses in their biblical order.

func (*Server) APBIndexBiblicalOrderHandler 

func (s *Server) APBIndexBiblicalOrderHandler() http.HandlerFunc

APBIndexBiblicalOrderHandler returns verses in their biblical order.

func (*Server) APBIndexChronologicalHandler 

func (s *Server) APBIndexChronologicalHandler() http.HandlerFunc

APBIndexChronologicalHandler returns verses in chronological order by their peak.

func (*Server) APBIndexFeaturedHandler 

func (s *Server) APBIndexFeaturedHandler() http.HandlerFunc

APBIndexFeaturedHandler returns featured verses for APB.

func (*Server) APBIndexTopHandler 

func (s *Server) APBIndexTopHandler() http.HandlerFunc

APBIndexTopHandler returns top verses for APB.

func (*Server) APBVerseHandler 

func (s *Server) APBVerseHandler() http.HandlerFunc

APBVerseHandler returns information about a verse, and other verses which are related to it, if any.

func (*Server) APBVerseQuotationsHandler 

func (s *Server) APBVerseQuotationsHandler() http.HandlerFunc

APBVerseQuotationsHandler returns the instances of quotations for a verse.

func (*Server) APBVerseTrendHandler 

func (s *Server) APBVerseTrendHandler() http.HandlerFunc

APBVerseTrendHandler returns the rates of quotation per year for a verse.

func (*Server) BillsHandler 

func (s *Server) BillsHandler() http.HandlerFunc

BillsHandler returns the bills for a given range of years. It expects a start year and an end year. It returns a JSON array of ParishByYear objects.

func (*Server) BillsShapefilesHandler 

func (s *Server) BillsShapefilesHandler() http.HandlerFunc

BillsShapefilesHandler returns a GeoJSON FeatureCollection containing parish polygons joined with the bills data. It accepts filtering by year, bill_type, count_type, etc.

func (*Server) CacheTest 

func (s *Server) CacheTest() http.HandlerFunc

CacheTest returns the time the application started and the time that the handler was last run. So if the result is cached, then one would expect that the time the handler was last run would remain the same.

func (*Server) CatholicDiocesesHandler 

func (s *Server) CatholicDiocesesHandler() http.HandlerFunc

CatholicDiocesesHandler returns a JSON array of Catholic dioceses. Though the spatial data is stored in the database as a geometry, it is returned as simple lon/lat coordinates because that is easiest to process in the visualizations.

func (*Server) CatholicDiocesesPerDecadeHandler 

func (s *Server) CatholicDiocesesPerDecadeHandler() http.HandlerFunc

CatholicDiocesesPerDecadeHandler returns a JSON array of the number of dioceses established in North America per year.

func (*Server) ChristeningsHandler 

func (s *Server) ChristeningsHandler() http.HandlerFunc

ChristeningsHandler returns the christenings for a given range of years. It expects a start year and end year as query parameters. Optional query parameters: id (location filter), bill-type (general/weekly filter).

func (*Server) CountiesInState 

func (s *Server) CountiesInState() http.HandlerFunc

CountiesInState returns a list of all the counties in a state, with IDs from AHCB.

func (*Server) DeathCausesHandler 

func (s *Server) DeathCausesHandler() http.HandlerFunc

DeathCausesHandler returns a JSON array of causes of death. The list of causes depends on whether a user has provided a comma-separated list of causes. If no list is provided, it returns the entire list of causes.

func (*Server) EndpointsHandler 

func (s *Server) EndpointsHandler() http.HandlerFunc

EndpointsHandler describes the endpoints that are available in this API, with sample URLs to show how the API works.

func (*Server) ListCausesHandler 

func (s *Server) ListCausesHandler() http.HandlerFunc

func (*Server) ListChristeningsHandler 

func (s *Server) ListChristeningsHandler() http.HandlerFunc

ListChristeningsHandler returns a list of parish names and ids where christenings have been recorded.

func (*Server) Middleware 

func (s *Server) Middleware()

Middleware registers the middleware functions that should be used.

func (*Server) NaturalEarthHandler 

func (s *Server) NaturalEarthHandler() http.HandlerFunc

NaturalEarthHandler returns a GeoJSON FeatureCollection containing country polygons by passing location parameters. The available country parameters are: Africa; Antarctica; Asia; Europe; North+America; Oceania; South+America; Seven+seas+(open+ocean)

func (*Server) NotFoundHandler 

func (s *Server) NotFoundHandler() http.HandlerFunc

NotFoundHandler returns 404 errors

func (*Server) ParishShpHandler 

func (s *Server) ParishShpHandler() http.HandlerFunc

ParishShpHandler returns a GeoJSON FeatureCollection containing parish polygons and can receive year, city_cnty, and subunit parameters.

func (*Server) ParishesHandler 

func (s *Server) ParishesHandler() http.HandlerFunc

ParishesHandler returns a list of unique parish IDs and names.

func (*Server) Place 

func (s *Server) Place() http.HandlerFunc

Place returns the details about a populated place.

func (*Server) PlacesInCounty 

func (s *Server) PlacesInCounty() http.HandlerFunc

PlacesInCounty returns a list of all the populated places in a county.

func (*Server) PresbyteriansHandler 

func (s *Server) PresbyteriansHandler() http.HandlerFunc

PresbyteriansHandler returns the aggregate data on Presbyterian memberhsip and churches.

func (*Server) RelCensusCityMembershipHandler 

func (s *Server) RelCensusCityMembershipHandler() http.HandlerFunc

RelCensusCityMembershipHandler returns the statistics for all the cities for a single denomination in a single year. It must be filtered by year and denomination.

func (*Server) RelCensusDenominationFamiliesHandler 

func (s *Server) RelCensusDenominationFamiliesHandler() http.HandlerFunc

RelCensusDenominationFamiliesHandler returns

func (*Server) RelCensusDenominationsHandler 

func (s *Server) RelCensusDenominationsHandler() http.HandlerFunc

RelCensusDenominationsHandler returns the denominations that are available. Optionally, it can be filtered to get just the denominations in a particular family.

func (*Server) RelCensusLocationsHandler 

func (s *Server) RelCensusLocationsHandler() http.HandlerFunc

RelCensusLocationsHandler returns a list of all locations

func (*Server) Routes 

func (s *Server) Routes()

Routes registers the handlers for the URLs that should be served.

func (*Server) Run 

func (s *Server) Run() error

Run starts the API server.

func (*Server) Shutdown 

func (s *Server) Shutdown()

Shutdown closes the connection to the database and shutsdown the server.

func (*Server) StatisticsHandler 

func (s *Server) StatisticsHandler() http.HandlerFunc

func (*Server) TotalBillsHandler 

func (s *Server) TotalBillsHandler() http.HandlerFunc

TotalBillsHandler returns the total number of bills in the database. This number is required for pagination in the web application.

type TotalBills 

type TotalBills struct {
	TotalRecords NullInt64 `json:"total_records"`
}

TotalBills returns to the total number of records in the database.

type Verse 

type Verse struct {
	Reference string   `json:"reference"`
	Text      string   `json:"text"`
	Related   []string `json:"related"`
}

Verse describes the reference and text of a single Bible verse

type VerseQuotation 

type VerseQuotation struct {
	Reference   string  `json:"reference"`
	DocID       string  `json:"docID"`
	Date        string  `json:"date"`
	Probability float32 `json:"probability"`
	Title       string  `json:"title"`
	State       string  `json:"state"`
}

VerseQuotation is a single instance of a quotation

type VerseTrend 

type VerseTrend struct {
	Year                int     `json:"year"`
	N                   int     `json:"n"`
	QuotationRateSmooth float64 `json:"smoothed"`
}

VerseTrend is the rate of quotations in a single year for a single verse in a given corpus. The quotation rate is expressed in quotations per million words; the smoothed rate has the same units, and is a centered three-year rolling average.

type VerseTrendResponse 

type VerseTrendResponse struct {
	Reference string       `json:"reference"`
	Corpus    string       `json:"corpus"`
	Trend     []VerseTrend `json:"trend"`
}

VerseTrendResponse wraps the data with the queries that were made.

type WeeklySummary 

type WeeklySummary struct {
	Year       int `json:"year"`
	WeekNumber int `json:"weekNumber"`
	RowsCount  int `json:"rowsCount"`
}

type YearlySummary 

type YearlySummary struct {
	Year           int `json:"year"`
	WeeksCompleted int `json:"weeksCompleted"`
	RowsCount      int `json:"rowsCount"`
	TotalCount     int `json:"totalCount"`
}

Statistics

Source Files

View all Source files

Directories

Path Synopsis
cmd
apiary command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.