Raytion Unified Content Store Installation & Configuration

Overview

The Raytion Unified Content Store (UCS) is a standalone application that offers a REST API to push JSON data to.

It is used if the original source system cannot provide a JSON REST interface according to the connector’s API definition. The UCS acts as middleware with a generic JSON API, allowing for flexible sets of metadata and document’s binary content, which the connector will pull and process afterwards.

Data includes new and updated documents with their content and metadata fields, as well as delete requests for previously existing documents. The UCS stores the information in its own database and offers it to the connector on request.

The REST API documentation is provided separately in the api-docs folder in the Unified Content Store’s installation directory as a Swagger UI HTML interface description.

The format used to define a document or delete request in the REST request is described in Content Types.

Installation

The basic installation of the Unified Content Store is straightforward. The connector is shipped with a zip archive named:

raytion-unified-content-store-x.x.x.zip

Unzip this file in an empty directory of the system hosting the Unified Content Store. In the following, this directory will be referred to as $UCSDIR.

Configuration

The Unified Content Store uses a single YAML configuration file for all settings. The configuration file is located at $UCSDIR/conf/application.yml.

Below are the main configuration sections and their parameters.

Complete Configuration Example

http:
  enabled: true
  port: 8042

https:
  enabled: true
  port: 8043
  key-store: conf/keystore.jks
  key-store-type: JKS
  key-store-password: CjfSIFFEfIb+8nLhYsD8BQ==
  key-alias: ucs

basic-auth:
  enabled: true
  realm: Raytion Unified Content Store
  account: raytion
  password: raytion
  encrypted: false

spring.datasource:
  url: jdbc:postgresql://localhost:5432/ucs
  username: ucs
  password: ucs

ucs:
  ncsa-request-log.enabled: false
  service:
    batch-timeout-minutes: 60
    deletion-block-size: 5000
    max-job-size: 200
    retry-after-delay-seconds: 120

logging:
  level:
    root: INFO
    org.jooq: WARN
    org.springframework: ERROR
    org.springframework.boot.web: INFO
    com.raytion.ucs.server: INFO
    com.raytion.ucs.util.TimeMeasurement: DEBUG

HTTP Connection

Name Property Key Description

Name	Property Key	Description
HTTP Connection	`http.enabled`	If set to true, the Unified Content Store’s REST service will listen to requests using unsecured HTTP (default: true).
HTTP Port	`http.port`	Port number for accepting HTTP requests (default: 8042).

HTTP Connection

http.enabled

If set to true, the Unified Content Store’s REST service will listen to requests using unsecured HTTP (default: true).

HTTP Port

http.port

Port number for accepting HTTP requests (default: 8042).

HTTPS Connection

Protecting access to the API with SSL is recommended, but not obligatory.

The connector itself currently does not support connecting to the UCS via HTTPS. Regular HTTP must be enabled for the connector to work. It can still make sense to activate the HTTPS connection for protected communication with other systems.

Name Property Key Description

Name	Property Key	Description
HTTPS Connection	`https.enabled`	If set to true, the Unified Content Store’s REST service will listen to requests using certificate based HTTPS (default: true).
HTTPS Port	`https.port`	Port number for accepting HTTPS requests (default: 8043).
Keystore	`https.key-store`	Path to the Java keystore containing the certificate for this service. It is speci- fied relative to $UCSDIR (default: `conf/keystore.jks`).
Keystore Type	`https.key-store-type`	Type of the server certificate store. PKCS12 is recommended over JKS (default: `JKS`).
Keystore Password	`https.key-store-password`	Encrypted password for the keystore. Use the Password Encryptor Tool for encryption.
Key Alias	`https.key-alias`	Optional alias of the keystore certificate to use for this service. (default: undefined, use first available certificate in keystore)

HTTPS Connection

https.enabled

If set to true, the Unified Content Store’s REST service will listen to requests using certificate based HTTPS (default: true).

HTTPS Port

https.port

Port number for accepting HTTPS requests (default: 8043).

Keystore

https.key-store

Path to the Java keystore containing the certificate for this service. It is speci- fied relative to $UCSDIR (default: conf/keystore.jks).

Keystore Type

https.key-store-type

Type of the server certificate store. PKCS12 is recommended over JKS (default: JKS).

Keystore Password

https.key-store-password

Encrypted password for the keystore. Use the Password Encryptor Tool for encryption.

Key Alias

https.key-alias

Optional alias of the keystore certificate to use for this service. (default: undefined, use first available certificate in keystore)

Database

The Unified Content Store requires a database to store the transferred data. For the basic installation and the first function test, an H2 database can be used. In productive use it is strongly recommended to use a PostgreSQL database. Other enterprise databases are not currently supported.

PostgreSQL Configuration Example

spring.datasource:
  url: jdbc:postgresql://localhost:5432/ucs
  username: ucs
  password: ucs

H2 Database Configuration Example

spring.datasource:
  url: jdbc:h2:./db/ucs
  username: ucs
  password: ucs

The configuration uses the default Spring Boot spring.datasource properties. The minimum required properties to be configured are:

Name Property Key Description

Name	Property Key	Description
JDBC URL	`spring.datasource.url`	The JDBC connection string for the database used.
Username	`spring.datasource.username`	The username to access the database. The user must have the proper rights to create, write and query a new schema in the database.
Password	`spring.datasource.password`	The user password to access the database, unencrypted.

JDBC URL

spring.datasource.url

The JDBC connection string for the database used.

Username

spring.datasource.username

The username to access the database. The user must have the proper rights to create, write and query a new schema in the database.

Password

spring.datasource.password

The user password to access the database, unencrypted.

The Unified Content Store creates a schema named "ucs" the first time it is started. This schema contains all the required database tables.

Service

Name Property Key Description

Name	Property Key	Description
NCSA Request Log	`ucs.ncsa-request-log.enabled`	If set to true, the UCS will write an NCSA type log file for all requests to `logs/yyyy_MM_dd.request.log` (default: `false`).
Batch Timeout	`ucs.service.batch-timeout-minutes`	Maximum duration for running batches before new batches are accepted and the old one is rolled back (default: 60).
Deletion Block Size	`ucs.service.deletion-block-size`	Maximum documents deleted at once during cleanup or batch rollbacks (default: 5000).
Maximum Job Size	`ucs.service.max-job-size`	The maximum number of documents that can be simultaneously pushed to the UCS within one batch (default: 200). If more documents are pushed, the request will be declined with HTTP code 400 - Bad Request.
Retry After Delay	`ucs.service.retry-after-delay-seconds`	Retry interval when dataset access is locked. Sets Retry-After header for HTTP 423 responses (default: 120).

NCSA Request Log

ucs.ncsa-request-log.enabled

If set to true, the UCS will write an NCSA type log file for all requests to logs/yyyy_MM_dd.request.log (default: false).

Batch Timeout

ucs.service.batch-timeout-minutes

Maximum duration for running batches before new batches are accepted and the old one is rolled back (default: 60).

Deletion Block Size

ucs.service.deletion-block-size

Maximum documents deleted at once during cleanup or batch rollbacks (default: 5000).

Maximum Job Size

ucs.service.max-job-size

The maximum number of documents that can be simultaneously pushed to the UCS within one batch (default: 200). If more documents are pushed, the request will be declined with HTTP code 400 - Bad Request.

Retry After Delay

ucs.service.retry-after-delay-seconds

Retry interval when dataset access is locked. Sets Retry-After header for HTTP 423 responses (default: 120).

HTTP Basic Access Authentication

The JSON Connector supports basic access authentication for securing the access to the Content Store’s REST API. Configuration for this feature is done in the basic-auth block of the configuration YAML:

basic-auth:
  enabled: true
  realm: Raytion Unified Content Store
  account: raytion
  password: raytion
  encrypted: false

Protecting access to the API with Basic Authentication is recommended, but not obligatory.

The following configuration parameters are supported:

Name Property Key Description

Name	Property Key	Description
Basic Auth Enabled	`basic-auth.enabled`	If set to true, the Unified Content Store’s REST API is secured using basic access authentication (default: false).
Authentication Realm	`basic-auth.realm`	The realm information returned during the authentication process (default: Raytion Unified Content Store).
Username	`basic-auth.account`	The username used for authentication (required if basic access authentication is enabled).
Password	`basic-auth.password`	The password used for authentication (required if basic access authentication is enabled).
Password Encryption	`basic-auth.encrypted`	If set to true, the password given in the configuration parameter `basic-auth.password` must be encrypted using the Password Encryptor Tool (optional, default: `false`).

Basic Auth Enabled

basic-auth.enabled

If set to true, the Unified Content Store’s REST API is secured using basic access authentication (default: false).

Authentication Realm

basic-auth.realm

The realm information returned during the authentication process (default: Raytion Unified Content Store).

Username

basic-auth.account

The username used for authentication (required if basic access authentication is enabled).

Password

basic-auth.password

The password used for authentication (required if basic access authentication is enabled).

Password Encryption

basic-auth.encrypted

If set to true, the password given in the configuration parameter basic-auth.password must be encrypted using the Password Encryptor Tool (optional, default: false).

Logging

The UCS uses Logback together with SLF4J for logging. The logging levels can be configured in the application.yml directly, under logging.level (see Example Configuration above).

For more fine-grained control of loggers and to modify the defaults, edit the Logback configuration file under $UCSDIR/conf/logback.xml.

Command Line Utility

For testing purposes, the Unified Content Store can be started from the command line. However, this is not recommended for production.

Windows: Start the batch file ucsControl.bat. This will open a new command processor running the Unified Content Store process in the background. Ending the user’s login session will also shut down the Unified Content Store. To use the Content Store as a Windows service, see the Windows Service section below.
Linux: Execute the file ucsControl.sh. The Content Store will be started as a background process detached from the current login session.

The command line starter of the Unified Content Store supports different options:

start: Run the UCS until it gets a stop command (see command line option stop).
stop: Stop a UCS server running in another process on the same machine.

Make sure the port used for the Unified Content Store (default: 8042) is open at the installation host.

Windows Service

For Windows production environments, the Unified Content Store shall be installed as a service.

Install the Windows Service

As a preparation, ensure the JAVA_HOME environment variable is set to the right Java installation. To install the service, start a command line as administrator, navigate to $UCSDIR/service, and run the command:

RaytionUcsService.exe install

Start the Windows service when the installation is finished using

RaytionUcsService.exe start

After performing these steps, a service is created (Id: RaytionUcsService}, display name: Raytion UCS, description: Raytion Unified Content Store) and displayed in the Windows Service application.

Uninstall the Windows Service

To uninstall the service, use:

RaytionUcsService.exe uninstall

REST Endpoint for Status Checks

The Raytion Unified Content Store provides a JSON/REST endpoint for status checks. It is reachable via:

http://ucs.domain.com:8042/rest/v4/status

where the server name is the fully qualified domain name (FQDN) of the server hosting the Unified Content Store.

The following status information is available:

Name Property Key Description

Name	Property Key	Description
Server Status	`server`	Will always be 'ok' as long as the Raytion Unified Content Store is running.
Version	`version`	Internal version number of the Unified Content Store.
Number of Datasets	`numberOfDataSets`	Shows the number of data sets known in the Unified Content Store.
Open Sessions	`openSessions`	Shows the number of total active sessions (batches) across all datasets.

Server Status

server

Will always be 'ok' as long as the Raytion Unified Content Store is running.

Version

version

Internal version number of the Unified Content Store.

Number of Datasets

numberOfDataSets

Shows the number of data sets known in the Unified Content Store.

Open Sessions

openSessions

Shows the number of total active sessions (batches) across all datasets.

In addition, the log files available in the logs directory provide insights of the Unified Content Store status.