kuha_osmh_repo_handler

Kuha OSMH Repo Handler application.

Serve records from Kuha Document Store throught OSMH protocol.

serve.py

Main entry point for starting OSMH Repo Handler

kuha_osmh_repo_handler.serve.get_app(api_version, app_settings=None)[source]

Setup routes and return initialized Tornado web application.

Parameters
  • api_version (str) – HTTP Api version gets prepended to routes.

  • app_settings (dict or None.) – Settings to store to application.

Returns

Tornado web application.

Return type

tornado.web.Application

kuha_osmh_repo_handler.serve.main()[source]

Application main function.

Parse commandline for settings. Pass settings to kuha_osmh_repo_handler.server.main(). Exit on exceptions propagated at this level.

Returns

exit code, 1 on error, 0 on success.

Return type

int

configure.py

Configure OSMH Repo Handler

kuha_osmh_repo_handler.configure.configure()[source]

Get settings for application configuration.

Declares application spesific configuration options and some common options declared in kuha_common.cli_setup

Configure application with arguments specified in configuration file, environment variables and command line arguments.

Note

Calling this function multiple times will not initiate new settings to be parsed, but will return previously parsed settings instead.

Returns

settings

Return type

argparse.Namespace

handlers.py

Define handlers for responding to HTTP-requests.

note

OSMH protocol only supports HTTP-GET

class kuha_osmh_repo_handler.handlers.BaseHandler(*args, **kwargs)[source]

BaseHandler class for handling OSMH requests.

Derived from kuha_common.server.RequestHandler. Defines common attributes.

async prepare()[source]

Prepare for responding to request.

Set output content type. Initiate kuha_osmh_repo_handler.response.RecordsResponse and kuha_common.query.QueryController as instance attributes.

class kuha_osmh_repo_handler.handlers.ListRecordHeadersHandler(*args, **kwargs)[source]

Handle list responses.

async prepare()[source]

Prepare for each request.

Depending on stream-configuration choose which response callback to use. If streaming is enabled write output once a single record has been built. Otherwise store all records in a list and write output when all records are built.

Note

With streaming enabled memory consumption will be lower since the records will not be gathered in a single list and encoded to JSON all at once. When dealing with large repositories the amount of memory consumed without streaming could be an issue.

async get(record_type=None)[source]

Handles HTTP-GET requests to endpoint.

Parameters

record_type (str or None) – Optional record_type parameter defines if the request limits the list to a certain OSMH type. If the parameter is None, shall output every record in repository. Valid values are defined in handler configuration.

class kuha_osmh_repo_handler.handlers.GetRecordHandler(*args, **kwargs)[source]

Handle responses for single record.

async get(record_type, identifier)[source]

Handle HTTP-GET requests to endpoint.

Gathers the needed information by querying Document Store. Builds the OSMH record response.

Parameters
  • record_type (str) – requested OSMH record type.

  • identifier (str) – requested record identifier.

class kuha_osmh_repo_handler.handlers.ListSupportedRecordTypesHandler(*args, **kwargs)[source]

Handle response to endpoint that lists supported record types.

async get()[source]

Handle HTTP-GET request.

Note

returning object will be a list, so will encode to JSON by using build-in json-module.

class kuha_osmh_repo_handler.handlers.SupportedVersionsHandler(*args, **kwargs)[source]

Handle response to endpoint that lists supported api version.

Note

Currently only single version is supported at a time.

async get()[source]

Handle HTTP-GET request.

Note

returning object will be a list, so will encode to JSON by using build-in json-module.

response.py

Build responses containing OSMH records.

class kuha_osmh_repo_handler.response.RecordsResponse[source]

Response containing records.

Provides methods that can be used as callbacks to kuha_common.query.QueryController. Uses OSMH records defined in kuha_osmh_repo_handler.osmh.records. Stores these records for later processing.

iterate_records()[source]

Iterate throught records gathered to response.

Returns

generator for iterating records.

get_record_appender(record_constructor)[source]

Get record appender function.

Records that are appended are constructed by using the record_constructor parameter.

Returns

callback function for appending records that get constructed by submitting the receiver parameter to given record_constructor.

Return type

functools.partial

get_payload_appender(record_constructor)[source]

Get payload appender function.

Records that are appended are constructed by using the record_constructor parameter. After constructing the record, calls the constructed record-objects get_payload() method.

Returns

callback function for appending records that get constructed by submitting the receiver parameter to given record_constructor.

Return type

functools.partial

get_response()[source]

Build and return the response as a list of dictionary payloads.

Returns

record payloads

Return type

list

get_single_response()[source]

Get single response payload.

Note

After calling this method the payloads list will be empty.

Returns

single record’s payload.

Return type

dict

Raises

ValueError if contained payloads list has more than a single cell.

osmh

Defines OSMH records and payload.

osmh/records.py

Build OSMH payload from Document Store record objects. Provide mapping between these two record formats. Provide Document Store fields for querying.

note

This module has strict dependency to kuha_common.document_store.records

class kuha_osmh_repo_handler.osmh.records.Payload(identifier, last_modified)[source]

Represents OSMH record’s payload.

Provides methods for manipulating the payload. Stores the payload in a dictionary, which can be easilly encoded to JSON.

Example:

>>> from kuha_osmh_repo_handler.osmh.records import Payload
>>> payload = Payload('1', '2017-01-01')
>>> payload.insert_localized_value('study_title', 'en', 'Household Survey')
>>> payload.insert_localized_value('study_title', 'fi', 'Kotitalouskysely')
>>> payload.get() # Indent for better readability
{'identifier': '1',
 'lastModified': '2017-01-01',
 'study_title':
    {'fi': 'Kotitalouskysely',
     'en': 'Household Survey'}
}
Parameters
  • identifier (str) – Record’s OSMH-identifier. Must uniquelly identify the record within other records of the same OSMH record type in the repository.

  • last_modified (str) – timestamp of the last modification made to the record.

Returns

Payload

classmethod join_values(*args)[source]

Join values together using _join_character

Parameters

*args (str) – values to join

classmethod split_value(value)[source]

Split value using _join_character

Parameters

value (str) – value to split

Returns

splitted values

Return type

list

insert(key, value)[source]

Insert a value to payload.

Insert a value for given key to the payload. If the key is not present in the payload, creates one.

Parameters
  • key (str) – payload key for the value.

  • value (str) – value to be inserted.

insert_localized_value(key, locale, value)[source]

Insert a localized value to payload.

Insert value for given locale into the given payload key. If the key is not present in the payload, creates one.

Parameters
  • key (str) – payload key

  • locale (str) – values locale

  • value (str) – payload value

append(key, value, unique=False)[source]

Insert list item to given payload key

If key is not in payload, creates it and inserts a list with a single cell containing value. If parameter unique is True, will not append duplicate values to list.

Parameters
  • key (str) – payload key

  • value (str) – value to insert as list item

  • unique (bool) – whether to keep the list of values unique (no duplicates)

header(osmh_type)[source]

Create record header to payload

Note

Header is common for all record types. The only changing value is the record type.

Parameters

osmh_type (str) – OSMH record type

get()[source]

Return the constructed payload

Returns

OSMH payload

Return type

dict

class kuha_osmh_repo_handler.osmh.records.OSMHRecord(payload)[source]

Abstract Base class for OSMH record.

Use from a subclass.

Provides common properties and methods to be used in OSMH records.

Parameters

payload (Payload) – payload of the record.

Raises

TypeError if subclass does not define class attributes.

abstract property osmh_type

OSMH type. Declare in subclass.

abstract property query_document

Document Store record to query. Declare in subclass.

abstract property relative_queries_for_record

Does the record-response require relative records queried from Dccument Store. Declare in subclass.

abstract static fields_for_header()[source]

Get fields to query that are required to build the record header.

Override in subclass.

abstract static fields_for_record()[source]

Get fields to query that are required to build the record.

Override in subclass.

abstract static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Override in subclass.

classmethod for_header_response(ds_record)[source]

Create a record for response that only contains headers for records.

Parameters

ds_record (Record defined in kuha_common.document_store.records) – Document Store record.

Returns

Instantiated OSMH record object.

classmethod for_record_response(ds_record)[source]

Create record for response containing the actual record.

Parameters

ds_record (Record defined in kuha_common.document_store.records) – Document Store record.

Returns

Instantiated OSHM record object.

classmethod get_query_document()[source]

Return the Document Store record used for Querying.

Returns

Document Store record used for querying.

classmethod requires_relative_queries_for_record()[source]

Does the record require querying for relative records from Document Store to construct the full record response.

Returns

True or False.

Return type

bool

build_header_payload()[source]

Builds the common header payload.

abstract build_record_payload()[source]

Builds the common record payload.

get_payload()[source]

Get the built payload.

Returns

record payload.

Return type

dict

class kuha_osmh_repo_handler.osmh.records.StudyRecord(study)[source]

Represents OSMH Study.

Derived from OSMHRecord.

Parameters

study (kuha_common.document_store.records.Study) – Study from Document Store.

Returns

Instantiated OSMH Study record

Return type

StudyRecord

query_document

alias of kuha_common.document_store.records.Study

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns

Study fields required to build record header.

Return type

list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns

Study fields required to build record header.

Return type

list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters

identifier (str) – study identifier (study number).

Returns

filter to use for query.

Return type

dict

static get_secondary_query_fields_for_record()[source]

Get fields to query that are required to build the relative record (Variable).

Returns

Variable fields.

Return type

list

static get_secondary_query_document()[source]

Get secondary query document (Document Store record).

Returns

Document Store variable record.

Return type

kuha_common.document_store.records.Variable

get_secondary_query_filter_for_record()[source]

Get filter which queries the correct record from Document Store.

Returns

filter to use for query.

Return type

dict

build_relative_record_payload(relative_record)[source]

Build payload for relative record.

Parameters

relative_record (kuha_common.document_store.records.Variable) – Relative record instance.

build_record_payload()[source]

Build payload for record.

class kuha_osmh_repo_handler.osmh.records.VariableRecord(variable)[source]

Represents OSMH Variable.

Derived from OSMHRecord.

Parameters

variable (kuha_common.document_store.records.Variable) – Variable from Document Store.

Returns

Instantiated OSMH Variable record

Return type

VariableRecord

query_document

alias of kuha_common.document_store.records.Variable

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns

Variable fields required to build record header.

Return type

list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns

Variable fields required to build record header.

Return type

list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters

identifier (str) – variable identifier.

Returns

filter to use for query.

Return type

dict

build_record_payload()[source]

Build payload for record.

class kuha_osmh_repo_handler.osmh.records.QuestionRecord(question)[source]

Represents OSMH Question.

Derived from OSMHRecord.

Parameters

question (kuha_common.document_store.records.Question) – Question from Document Store.

Returns

Instantiated OSMH Question record

Return type

QuestionRecord

query_document

alias of kuha_common.document_store.records.Question

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns

Question fields required to build record header.

Return type

list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns

Question fields required to build record header.

Return type

list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters

identifier (str) – question identifier.

Returns

filter to use for query.

Return type

dict

build_record_payload()[source]

Build record payload.

class kuha_osmh_repo_handler.osmh.records.StudyGroupRecord(study_group)[source]

Represents OSMH StudyGroup.

Derived from OSMHRecord.

Parameters

study_group (kuha_common.document_store.records.StudyGroup) – StudyGroup from Document Store.

Returns

Instantiated OSMH StudyGroup record

Return type

StudyGroupRecord

query_document

alias of kuha_common.document_store.records.StudyGroup

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns

StudyGroup fields required to build record header.

Return type

list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns

StudyGroup fields required to build record header.

Return type

list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters

identifier (str) – Study group identifier.

Returns

filter to use for query.

Return type

dict

build_record_payload()[source]

Build record payload.

kuha_osmh_repo_handler.osmh.records.get_osmh_record_for_type(osmh_record_type)[source]

Return the OSMH record class representing osmh_record_type.

Parameters

osmh_record_type (str) – Supported OSMH record type.

Returns

One of the OSMH records defined in this module.

Return type

StudyRecord or VariableRecord or QuestionRecord or StudyGroupRecord