kuha_osmh_repo_handler

Kuha OSMH Repo Handler application.

Serve records from Kuha Document Store throught OSMH protocol.

serve.py

Main entry point for starting OSMH Repo Handler

kuha_osmh_repo_handler.serve.get_app(api_version, app_settings=None)[source]

Setup routes and return initialized Tornado web application.

Parameters

api_version (str) – HTTP Api version gets prepended to routes.
app_settings (dict or None.) – Settings to store to application.

Returns

Tornado web application.

Return type

tornado.web.Application

kuha_osmh_repo_handler.serve.main()[source]

Application main function.

Parse commandline for settings. Pass settings to kuha_osmh_repo_handler.server.main(). Exit on exceptions propagated at this level.

Returns: exit code, 1 on error, 0 on success.
Return type: int

configure.py

Configure OSMH Repo Handler

kuha_osmh_repo_handler.configure.configure()[source]

Get settings for application configuration.

Declares application spesific configuration options and some common options declared in kuha_common.cli_setup

Configure application with arguments specified in configuration file, environment variables and command line arguments.

Note: Calling this function multiple times will not initiate new settings to be parsed, but will return previously parsed settings instead.
Returns: settings
Return type: argparse.Namespace

handlers.py

Define handlers for responding to HTTP-requests.

note: OSMH protocol only supports HTTP-GET

class kuha_osmh_repo_handler.handlers.BaseHandler(*args, **kwargs)[source]

BaseHandler class for handling OSMH requests.

Derived from kuha_common.server.RequestHandler. Defines common attributes.

async prepare()[source]

Prepare for responding to request.

Set output content type. Initiate kuha_osmh_repo_handler.response.RecordsResponse and kuha_common.query.QueryController as instance attributes.

class kuha_osmh_repo_handler.handlers.ListRecordHeadersHandler(*args, **kwargs)[source]

Handle list responses.

async prepare()[source]

Prepare for each request.

Depending on stream-configuration choose which response callback to use. If streaming is enabled write output once a single record has been built. Otherwise store all records in a list and write output when all records are built.

Note: With streaming enabled memory consumption will be lower since the records will not be gathered in a single list and encoded to JSON all at once. When dealing with large repositories the amount of memory consumed without streaming could be an issue.

async get(record_type=None)[source]

Handles HTTP-GET requests to endpoint.

Parameters: record_type (str or None) – Optional record_type parameter defines if the request limits the list to a certain OSMH type. If the parameter is None, shall output every record in repository. Valid values are defined in handler configuration.

class kuha_osmh_repo_handler.handlers.GetRecordHandler(*args, **kwargs)[source]

Handle responses for single record.

async get(record_type, identifier)[source]

Handle HTTP-GET requests to endpoint.

Gathers the needed information by querying Document Store. Builds the OSMH record response.

Parameters

record_type (str) – requested OSMH record type.
identifier (str) – requested record identifier.

class kuha_osmh_repo_handler.handlers.ListSupportedRecordTypesHandler(*args, **kwargs)[source]

Handle response to endpoint that lists supported record types.

async get()[source]

Handle HTTP-GET request.

Note: returning object will be a list, so will encode to JSON by using build-in json-module.

class kuha_osmh_repo_handler.handlers.SupportedVersionsHandler(*args, **kwargs)[source]

Handle response to endpoint that lists supported api version.

Note: Currently only single version is supported at a time.

async get()[source]

Handle HTTP-GET request.

Note: returning object will be a list, so will encode to JSON by using build-in json-module.

response.py

Build responses containing OSMH records.

class kuha_osmh_repo_handler.response.RecordsResponse[source]

Response containing records.

Provides methods that can be used as callbacks to kuha_common.query.QueryController. Uses OSMH records defined in kuha_osmh_repo_handler.osmh.records. Stores these records for later processing.

iterate_records()[source]

Iterate throught records gathered to response.

Returns: generator for iterating records.

get_record_appender(record_constructor)[source]

Get record appender function.

Records that are appended are constructed by using the record_constructor parameter.

Returns: callback function for appending records that get constructed by submitting the receiver parameter to given record_constructor.
Return type: functools.partial

get_payload_appender(record_constructor)[source]

Get payload appender function.

Records that are appended are constructed by using the record_constructor parameter. After constructing the record, calls the constructed record-objects get_payload() method.

Returns: callback function for appending records that get constructed by submitting the receiver parameter to given record_constructor.
Return type: functools.partial

get_response()[source]

Build and return the response as a list of dictionary payloads.

Returns: record payloads
Return type: list

get_single_response()[source]

Get single response payload.

Note: After calling this method the payloads list will be empty.
Returns: single record’s payload.
Return type: dict
Raises: ValueError if contained payloads list has more than a single cell.

osmh

Defines OSMH records and payload.

osmh/records.py

Build OSMH payload from Document Store record objects. Provide mapping between these two record formats. Provide Document Store fields for querying.

note: This module has strict dependency to kuha_common.document_store.records

class kuha_osmh_repo_handler.osmh.records.Payload(identifier, last_modified)[source]

Represents OSMH record’s payload.

Provides methods for manipulating the payload. Stores the payload in a dictionary, which can be easilly encoded to JSON.

Example:

>>> from kuha_osmh_repo_handler.osmh.records import Payload
>>> payload = Payload('1', '2017-01-01')
>>> payload.insert_localized_value('study_title', 'en', 'Household Survey')
>>> payload.insert_localized_value('study_title', 'fi', 'Kotitalouskysely')
>>> payload.get() # Indent for better readability
{'identifier': '1',
 'lastModified': '2017-01-01',
 'study_title':
    {'fi': 'Kotitalouskysely',
     'en': 'Household Survey'}
}

Parameters

identifier (str) – Record’s OSMH-identifier. Must uniquelly identify the record within other records of the same OSMH record type in the repository.
last_modified (str) – timestamp of the last modification made to the record.

Returns

Payload

classmethod join_values(*args)[source]

Join values together using _join_character

Parameters: *args (str) – values to join

classmethod split_value(value)[source]

Split value using _join_character

Parameters: value (str) – value to split
Returns: splitted values
Return type: list

insert(key, value)[source]

Insert a value to payload.

Insert a value for given key to the payload. If the key is not present in the payload, creates one.

Parameters

key (str) – payload key for the value.
value (str) – value to be inserted.

insert_localized_value(key, locale, value)[source]

Insert a localized value to payload.

Insert value for given locale into the given payload key. If the key is not present in the payload, creates one.

Parameters

key (str) – payload key
locale (str) – values locale
value (str) – payload value

append(key, value, unique=False)[source]

Insert list item to given payload key

If key is not in payload, creates it and inserts a list with a single cell containing value. If parameter unique is True, will not append duplicate values to list.

Parameters

key (str) – payload key
value (str) – value to insert as list item
unique (bool) – whether to keep the list of values unique (no duplicates)

header(osmh_type)[source]

Create record header to payload

Note: Header is common for all record types. The only changing value is the record type.
Parameters: osmh_type (str) – OSMH record type

get()[source]

Return the constructed payload

Returns: OSMH payload
Return type: dict

class kuha_osmh_repo_handler.osmh.records.OSMHRecord(payload)[source]

Abstract Base class for OSMH record.

Use from a subclass.

Provides common properties and methods to be used in OSMH records.

Parameters: payload (Payload) – payload of the record.
Raises: TypeError if subclass does not define class attributes.

abstract property osmh_type: OSMH type. Declare in subclass.

abstract property query_document: Document Store record to query. Declare in subclass.

abstract property relative_queries_for_record: Does the record-response require relative records queried from Dccument Store. Declare in subclass.

abstract static fields_for_header()[source]

Get fields to query that are required to build the record header.

Override in subclass.

abstract static fields_for_record()[source]

Get fields to query that are required to build the record.

Override in subclass.

abstract static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Override in subclass.

classmethod for_header_response(ds_record)[source]

Create a record for response that only contains headers for records.

Parameters: ds_record (Record defined in kuha_common.document_store.records) – Document Store record.
Returns: Instantiated OSMH record object.

classmethod for_record_response(ds_record)[source]

Create record for response containing the actual record.

Parameters: ds_record (Record defined in kuha_common.document_store.records) – Document Store record.
Returns: Instantiated OSHM record object.

classmethod get_query_document()[source]

Return the Document Store record used for Querying.

Returns: Document Store record used for querying.

classmethod requires_relative_queries_for_record()[source]

Does the record require querying for relative records from Document Store to construct the full record response.

Returns: True or False.
Return type: bool

build_header_payload()[source]: Builds the common header payload.

abstract build_record_payload()[source]: Builds the common record payload.

get_payload()[source]

Get the built payload.

Returns: record payload.
Return type: dict

class kuha_osmh_repo_handler.osmh.records.StudyRecord(study)[source]

Represents OSMH Study.

Derived from OSMHRecord.

Parameters: study (kuha_common.document_store.records.Study) – Study from Document Store.
Returns: Instantiated OSMH Study record
Return type: StudyRecord

query_document: alias of kuha_common.document_store.records.Study

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns: Study fields required to build record header.
Return type: list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns: Study fields required to build record header.
Return type: list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters: identifier (str) – study identifier (study number).
Returns: filter to use for query.
Return type: dict

static get_secondary_query_fields_for_record()[source]

Get fields to query that are required to build the relative record (Variable).

Returns: Variable fields.
Return type: list

static get_secondary_query_document()[source]

Get secondary query document (Document Store record).

Returns: Document Store variable record.
Return type: kuha_common.document_store.records.Variable

get_secondary_query_filter_for_record()[source]

Get filter which queries the correct record from Document Store.

Returns: filter to use for query.
Return type: dict

build_relative_record_payload(relative_record)[source]

Build payload for relative record.

Parameters: relative_record (kuha_common.document_store.records.Variable) – Relative record instance.

build_record_payload()[source]: Build payload for record.

class kuha_osmh_repo_handler.osmh.records.VariableRecord(variable)[source]

Represents OSMH Variable.

Derived from OSMHRecord.

Parameters: variable (kuha_common.document_store.records.Variable) – Variable from Document Store.
Returns: Instantiated OSMH Variable record
Return type: VariableRecord

query_document: alias of kuha_common.document_store.records.Variable

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns: Variable fields required to build record header.
Return type: list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns: Variable fields required to build record header.
Return type: list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters: identifier (str) – variable identifier.
Returns: filter to use for query.
Return type: dict

build_record_payload()[source]: Build payload for record.

class kuha_osmh_repo_handler.osmh.records.QuestionRecord(question)[source]

Represents OSMH Question.

Derived from OSMHRecord.

Parameters: question (kuha_common.document_store.records.Question) – Question from Document Store.
Returns: Instantiated OSMH Question record
Return type: QuestionRecord

query_document: alias of kuha_common.document_store.records.Question

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns: Question fields required to build record header.
Return type: list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns: Question fields required to build record header.
Return type: list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters: identifier (str) – question identifier.
Returns: filter to use for query.
Return type: dict

build_record_payload()[source]: Build record payload.

class kuha_osmh_repo_handler.osmh.records.StudyGroupRecord(study_group)[source]

Represents OSMH StudyGroup.

Derived from OSMHRecord.

Parameters: study_group (kuha_common.document_store.records.StudyGroup) – StudyGroup from Document Store.
Returns: Instantiated OSMH StudyGroup record
Return type: StudyGroupRecord

query_document: alias of kuha_common.document_store.records.StudyGroup

static fields_for_header()[source]

Get fields to query that are required to build the record header.

Returns: StudyGroup fields required to build record header.
Return type: list

static fields_for_record()[source]

Get fields to query that are required to build the record.

Returns: StudyGroup fields required to build record header.
Return type: list

static query_filter_for_record(identifier)[source]

Get filter which queries the correct record from Document Store.

Parameters: identifier (str) – Study group identifier.
Returns: filter to use for query.
Return type: dict

build_record_payload()[source]: Build record payload.

kuha_osmh_repo_handler.osmh.records.get_osmh_record_for_type(osmh_record_type)[source]

Return the OSMH record class representing osmh_record_type.

Parameters: osmh_record_type (str) – Supported OSMH record type.
Returns: One of the OSMH records defined in this module.
Return type: StudyRecord or VariableRecord or QuestionRecord or StudyGroupRecord