Installation

This chapter describes the installation of each application.

Installing Kuha Document Store

This guide will provide step-by-step instructions in installing Kuha Document Store and MongoDB database. The supported operating system is Ubuntu 16.04, but other modern Linux variants may be used.

In this guide the installation of the database is done on a separate server, which will also be Ubuntu 16.04. However, Document Store and MongoDB may be installed on the same server.

Install MongoDB

Note

These actions should be done on the MongoDB server.

It is recommended to use the latest version of MongoDB which can be obtained from MongoDB’s own repository. Refer to MongoDB manual on how to install MongoDB to Ubuntu 16.04. At the time of writing the installation was done as follows.

  1. Obtain MongoDB public key.
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
  1. Add MongoDB source.
echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
  1. Update indexes and install.
sudo apt-get update && sudo apt-get install -y mongodb-org
  1. Configure MongoDB to accept incoming connections. Use IP of your MongoDB server in <mongodb-ip>.
sudo sed -i 's/  bindIp: 127.0.0.1/  bindIp: <mongodb-ip>/' /etc/mongod.conf
  1. Start MongoDB.
sudo service mongod start

Now MongoDB is running and accepting incoming connections. Note that the MongoDB instance is now accepting incoming connections without authentication. Authentication will be enabled later in this guide. For up-to-date information on configuration and security see MongoDB manual.

The next step is to create administrator credentials and setup databases, collections and database users. This can be done with a script bundled with Kuha Document Store.

First it is required to install the Document Store package.

Install Document Store

Note

These actions should be done on the Document Store server.

  1. Create directory for document store and Python virtualenv.
mkdir kuha2
  1. Clone package to subdirectory.
hg clone -r releases https://bitbucket.org/tietoarkisto/kuha_document_store kuha2/kuha_document_store
  1. Install Python virtual environment.
sudo apt install -y python3-venv
  1. Make installation script executable.
chmod +x ./kuha2/kuha_document_store/scripts/install_kuha_document_store_virtualenv.sh
  1. Install Kuha Document Store to virtual environment.
./kuha2/kuha_document_store/scripts/install_kuha_document_store_virtualenv.sh

Setup MongoDB for Document Store

Document store provides a script which will help setup MongoDB. The script will prompt for administrator credentials, which will be created. Give hostname/IP of your MongoDB server as command line parameter.

The script will create needed collections and database users. It will also setup indexes for the collections to speed up database queries.

Note

You may wish to provide DB credentials for editor and reader. Give parameter --help to see how.

  1. Make the setup script executable.
chmod +x ./kuha2/kuha_document_store/scripts/setup_mongodb.sh
  1. Run the MongoDB setup script. Replace <mongodb-ip> with the IP of your MongoDB server.
./kuha2/kuha_document_store/scripts/setup_mongodb.sh --database-host=<mongodb-ip>

Now the database is ready to be used with Document Store. Care should be taken to secure the MongoDB instance. For Kuha2 the only IP that needs access to the database is Kuha Document Store’s IP. Authentication should also be enabled.

Enable authentication to MongoDB

Note

These actions should be done on the MongoDB server.

  1. Enable authentication.
sudo sed -i 's/#security:/security:\n  authorization: enabled/' /etc/mongod.conf
  1. Restart MongoDB.
sudo service mongod restart

Running the Document Store

Note

This action should be done on the Document Store server.

Start serving Document Store. Replace <mongodb-ip> with the IP of your MongoDB server.

./kuha2/kuha_document_store/scripts/run_kuha_document_store.sh --database-host=<mongodb-ip>

Installing Kuha OSMH Repo Handler

  1. Create directory for OSMH Repo Handler and Python virtualenv.
mkdir kuha2
  1. Clone package to subdirectory.
hg clone -r releases https://bitbucket.org/tietoarkisto/kuha_osmh_repo_handler kuha2/kuha_osmh_repo_handler
  1. Install Python virtual environment.
sudo apt install -y python3-venv
  1. Make install script executable.
chmod +x ./kuha2/kuha_osmh_repo_handler/scripts/install_kuha_osmh_repo_handler_virtualenv.sh
  1. Install Kuha OSMH Repo Handler to virtual environment.
./kuha2/kuha_osmh_repo_handler/scripts/install_kuha_osmh_repo_handler_virtualenv.sh

To run Kuha OSMH Repo Handler you need access to Kuha Document Store. First you will need to make run script executable.

chmod +x ./kuha2/kuha_osmh_repo_handler/scripts/run_kuha_osmh_repo_handler.sh

Run by calling the script. Replace <document-store-url> with the URL to the Document Store.

./kuha2/kuha_osmh_repo_handler/scripts/run_kuha_osmh_repo_handler.sh --document-store-url=<document-store-url>

Installing Kuha OAI-PMH Repo Handler

  1. Create directory for OAI-PMH Repo Handler and Python virtualenv.
mkdir kuha2
  1. Clone package to subdirectory.
hg clone -r releases https://bitbucket.org/tietoarkisto/kuha_oai_pmh_repo_handler kuha2/kuha_oai_pmh_repo_handler
  1. Install Python virtual environment.
sudo apt install -y python3-venv
  1. Make install script executable.
chmod +x ./kuha2/kuha_oai_pmh_repo_handler/scripts/install_kuha_oai_pmh_repo_handler_virtualenv.sh
  1. Install Kuha OAI-PMH Repo Handler to virtual environment.
./kuha2/kuha_oai_pmh_repo_handler/scripts/install_kuha_oai_pmh_repo_handler_virtualenv.sh

To run Kuha OAI-PMH Repo Handler you need access to Kuha Document Store. First make the run script executable.

chmod +x ./kuha2/kuha_oai_pmh_repo_handler/scripts/run_kuha_oai_pmh_repo_handler.sh

Run by calling the script. Replace <document-store-url> with the URL to the Document Store. You also need to specify few configuration values for OAI-PMH: base_url and admin_email.

./kuha2/kuha_oai_pmh_repo_handler/scripts/run_kuha_oai_pmh_repo_handler.sh --document-store-url=<document-store-url> --oai-pmh-base-url=<base_url> --oai-pmh-admin-email=<email>

Installing Kuha Client

  1. Create directory for Kuha Client and Python virtualenv.
mkdir kuha2
  1. Clone package to subdirectory.
hg clone -r releases https://bitbucket.org/tietoarkisto/kuha_client kuha2/kuha_client
  1. Install Python virtual environment.
sudo apt install -y python3-venv
  1. Install Kuha Client to virtual environment
cd kuha2
python3 -m venv kuha_client-env
source ./kuha_client-env/bin/activate
cd kuha_client
pip install -r requirements.txt
python setup.py install

Batch import files to Document Store. Python virtual environment must be active.

python -m kuha_client.kuha_import --document-store-url=<document-store-url> --file-log-path=file_log /path/to/directory

Batch upsert records to Document Store and remove records not in current batch. Python virtual environment must be active.

python -m kuha_client.kuha_upsert --document-store-url=<document-store-url> --remove-absent --file-log-path=file_log /path/to/directory