Link Search Menu Expand Document Documentation Menu
Documentation

Docker

Docker greatly simplifies the process of configuring and managing your Lucenia clusters. You can pull official images from Docker Hub and quickly deploy a cluster using Docker Compose and any of the sample Docker Compose files included in this guide. Experienced Lucenia users can further customize their deployment by creating a custom Docker Compose file.

Docker containers are portable and will run on any compatible host that supports Docker (such as Linux, MacOS, or Windows). The portability of a Docker container offers flexibility over other installations methods, like Tarball installation, which requires additional configuration after downloading and unpacking.

This guide assumes that you are comfortable working from the Linux command line interface (CLI). You should understand how to input commands, navigate between directories, and edit text files. For help with Docker or Docker Compose, refer to the official documentation on their websites.

Install Docker and Docker Compose

Visit Get Docker for guidance on installing and configuring Docker for your environment. If you are installing Docker Engine using the CLI, then Docker, by default, will not have any constraints on available host resources. Depending on your environment, you may wish to configure resource limits in Docker. See Runtime options with Memory, CPUs, and GPUs for information.

Docker Desktop users should set host memory utilization to a minimum of 4 GB by opening Docker Desktop and selecting Settings → Resources.

Docker Compose is a utility that allows users to launch multiple containers with a single command. You pass a file to Docker Compose when you invoke it. Docker Compose reads those settings and starts the requested containers. Docker Compose is installed automatically with Docker Desktop, but users operating in a command line environment must install Docker Compose manually. You can find information about installing Docker Compose on the official Docker Compose GitHub page.

If you need to install Docker Compose manually and your host supports Python, you can use pip to install the Docker Compose package automatically.

Configure important host settings

Before installing Lucenia using Docker, configure the following settings. These are the most important settings that can affect the performance of your services, but for additional information, see important system settings.

Linux settings

For a Linux environment, run the following commands:

  1. Disable memory paging and swapping performance on the host to improve performance. 
    sudo swapoff -a
  2. Increase the number of memory maps available to Lucenia. 
    # Edit the sysctl config file
sudo vi /etc/sysctl.conf

# Add a line to define the desired value
# or change the value if the key exists,
# and then save your changes.
vm.max_map_count=262144

# Reload the kernel parameters using sysctl
sudo sysctl -p

# Verify that the change was applied by checking the value
cat /proc/sys/vm/max_map_count

Windows settings

For Windows workloads using WSL through Docker Desktop, run the following commands in a terminal to set the vm.max_map_count:

wsl -d docker-desktop
sysctl -w vm.max_map_count=262144

Deploy a Lucenia cluster using Docker Compose

Although it is technically possible to build a Lucenia cluster by creating containers one command at a time, it is far easier to define your environment in a YAML file and let Docker Compose manage the cluster. The following section contains example YAML files that you can use to launch a predefined cluster with Lucenia. These examples are useful for testing and development, but are not suitable for a production environment. If you don’t have prior experience using Docker Compose, you may wish to review the Docker Compose specification for guidance on syntax and formatting before making any changes to the dictionary structures in the examples.

The YAML file that defines the environment is referred to as a Docker Compose file. By default, docker compose commands will first check your current directory for a file that matches any of the following names:

  • docker-compose.yml
  • docker-compose.yaml
  • compose.yml
  • compose.yaml

If none of those files exist in your current directory, the docker compose command fails.

You can specify a custom file location and name when invoking docker compose with the -f flag:

# Use a relative or absolute path to the file.
docker compose -f /path/to/your-file.yml up

If this is your first time launching a Lucenia cluster using Docker Compose, use the following example docker-compose.yml file. Save it in the home directory of your host and name it docker-compose.yml. This file creates a cluster that contains two containers; one running the Lucenia service, and the other running OpenSearch Dashboards. These containers communicate over a bridge network called lucenia-net. Because this file does not explicitly disable the demo security configuration, self-signed TLS certificates are installed and internal users with default names and passwords are created.

Configuring your license

To obtain a trial license, follow the instructions in the Lucenia Trial License Activation guide. After you receive your trial license, save it as trial.crt in the same directory as your docker-compose.yml file.

Setting a custom admin password

A custom admin password is required to set up a demo security configuration. To simplify this example, we set the password in the docker-compose.yml file, and opensearch_dashboards.yml configuration. In a production environment, you should set the password using the Security configuration settings.

Sample docker-compose.yml

name: lucenia
services:
  lucenia-node1:
    image: lucenia/lucenia:0.1.0
    container_name: lucenia-node1
    environment:
      - cluster.name=lucenia-cluster # Name the cluster
      - node.name=lucenia-node1 # Name the node that will run in this container
      - discovery.type=single-node
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "LUCENIA_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
      - network.host=0.0.0.0
      - plugins.license.certificate_filepath=config/trial.crt
      - LUCENIA_INITIAL_ADMIN_PASSWORD=MyStrongPassword123!
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the lucenia user - set to at least 65536
        hard: 65536
    volumes:
      - lucenia-data1:/usr/share/lucenia/data # Creates volume called lucenia-data1 and mounts it to the container
      - ./trial.crt:/usr/share/lucenia/config/trial.crt
    ports:
      - 9200:9200 # REST API
    networks:
      - lucenia-net # All of the containers will join the same Docker bridge network
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.18.0
    container_name: opensearch-dashboards
    ports:
      - 5601:5601 # Map host port 5601 to container port 5601
    expose:
      - "5601" # Expose port 5601 for web access to OpenSearch Dashboards
    environment:
      - 'OPENSEARCH_HOSTS=["https://lucenia-node1:9200"]'
    volumes:
      - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
    networks:
      - lucenia-net

volumes:
  lucenia-data1:

networks:
  lucenia-net:

If you override opensearch_dashboards.yml settings using environment variables in your compose file, use all uppercase letters and replace periods with underscores (for example, for opensearch.hosts, use OPENSEARCH_HOSTS). This behavior is inconsistent with overriding lucenia.yml settings, where the conversion is just a change to the assignment operator (for example, discovery.type: single-node in lucenia.yml is defined as discovery.type=single-node in docker-compose.yml).

Sample opensearch_dashboards.yml

server.port: 5601
server.host: "0.0.0.0"
opensearch.hosts: ["https://lucenia-node1:9200"]
opensearch.ssl.verificationMode: none
opensearch.username: "admin"
opensearch.password: "MyStrongPassword123!"
opensearch.requestHeadersWhitelist: [ authorization,securitytenant ]
opensearch.ignoreVersionMismatch: true

From the home directory of your host (containing docker-compose.yml), create and start the containers in detached mode:

docker compose up -d

Verify that the service containers started correctly:

docker compose ps

If a container failed to start, you can review the service logs:

# If you don't pass a service name, docker compose will show you logs from all of the nodes
docker compose logs <serviceName>

Verify access to OpenSearch Dashboards by connecting to http://localhost:5601 from a browser, authenticating with the username admin and the value of $LUCENIA_INITIAL_ADMIN_PASSWORD. We do not recommend using the default security configuration on hosts that are accessible from the public internet; the security configuration of your deployment should be customized.

Remember that localhost cannot be accessed remotely. If you are deploying these containers to a remote host, then you will need to establish a network connection and replace localhost with the IP or DNS record corresponding to the host.

Stop the running containers in your cluster:

docker compose down

docker compose down will stop the running containers, but it will not remove the Docker volumes that exist on the host. If you don’t care about the contents of these volumes, use the -v option to delete all volumes, for example, docker compose down -v.

Configure Lucenia

Unlike the RPM distribution of Lucenia, which requires a large amount of post-installation configuration, running Lucenia clusters with Docker allows you to define the environment before the containers are even created. This is possible whether you use Docker or Docker Compose.

When you build your Lucenia cluster with Docker Compose you might find it easier to pass custom configuration files from your host to the container, as opposed to enumerating each individual setting in docker-compose.yml. Recall from our sample docker-compose.yml file, we specify volumes to mount as a sub-option to the corresponding service. The following truncated YAML file highlights how to mount a file or directory to the container. Refer to the official Docker documentation on volumes for comprehensive information about volume usage and syntax.

services:
  lucenia-node1:
    volumes:
      - lucenia-data1:/usr/share/lucenia/data
      - ./trial.crt:/usr/share/lucenia/config/trial.crt
      - ./lucenia.yml:/usr/share/lucenia/config/lucenia.yml # Custom Lucenia configuration
  opensearch-dashboards:
    volumes:
      - ./opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml

Configuring basic security settings

Before making your Lucenia cluster available to external hosts, it’s a good idea to review the deployment’s security configuration. Unless disabled by setting DISABLE_SECURITY_PLUGIN=true, a bundled script will apply a default demo security configuration to the nodes in the cluster. Because this configuration is used for demo purposes, the default usernames and passwords are known. For that reason, we recommend that you create your own security configuration files and use volumes to pass these files to the containers. For specific guidance on Lucenia security settings, see Security configuration.

To use your own certificates in your configuration, add all of the necessary certificates to the volumes section of the compose file:

volumes:
  - ./root-ca.pem:/usr/share/lucenia/config/root-ca.pem
  - ./admin.pem:/usr/share/lucenia/config/admin.pem
  - ./admin-key.pem:/usr/share/lucenia/config/admin-key.pem
  - ./node1.pem:/usr/share/lucenia/config/node1.pem
  - ./node1-key.pem:/usr/share/lucenia/config/node1-key.pem

When you add TLS certificates to your Lucenia nodes with Docker Compose volumes, you should also include a custom lucenia.yml file that defines those certificates. For example:

volumes:
  - ./root-ca.pem:/usr/share/lucenia/config/root-ca.pem
  - ./admin.pem:/usr/share/lucenia/config/admin.pem
  - ./admin-key.pem:/usr/share/lucenia/config/admin-key.pem
  - ./node1.pem:/usr/share/lucenia/config/node1.pem
  - ./node1-key.pem:/usr/share/lucenia/config/node1-key.pem
  - ./custom-lucenia.yml:/usr/share/lucenia/config/lucenia.yml

Remember that the certificates you specify in your compose file must be the same as the certificates defined in your custom lucenia.yml file. You should replace the root, admin, and node certificates with your own. For more information see Configure TLS certificates.

plugins.security.ssl.transport.pemcert_filepath: node1.pem
plugins.security.ssl.transport.pemkey_filepath: node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.http.pemcert_filepath: node1.pem
plugins.security.ssl.http.pemkey_filepath: node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.authcz.admin_dn:
  - CN=admin,OU=SSL,O=Test,L=Test,C=DE

After configuring security settings, your custom lucenia.yml file might look something like the following example, which adds TLS certificates and the distinguished name (DN) of the admin certificate, defines a few permissions, and enables verbose audit logging:

plugins.security.ssl.transport.pemcert_filepath: node1.pem
plugins.security.ssl.transport.pemkey_filepath: node1-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: node1.pem
plugins.security.ssl.http.pemkey_filepath: node1-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=A,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA
plugins.security.nodes_dn:
  - 'CN=N,OU=UNIT,O=ORG,L=TORONTO,ST=ONTARIO,C=CA'
plugins.security.audit.type: internal_lucenia
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
cluster.routing.allocation.disk.threshold_enabled: false
opendistro_security.audit.config.disabled_rest_categories: NONE
opendistro_security.audit.config.disabled_transport_categories: NONE

For a full list of settings, see Security.

Use the same process to specify a Backend configuration in /usr/share/lucenia/config/lucenia-security/config.yml as well as new internal users, roles, mappings, action groups, and tenants in their respective YAML files.

After replacing the certificates and creating your own internal users, roles, mappings, action groups, and tenants, use Docker Compose to start the cluster:

docker compose up -d

Working with plugins

To use the Lucenia image with a custom plugin, you must first create a Dockerfile. Review the official Docker documentation for information about creating a Dockerfile.

FROM lucenia/lucenia:0.1.0
RUN /usr/share/lucenia/bin/lucenia-plugin install --batch <pluginId>

Then run the following commands:

# Build an image from a Dockerfile
docker build --tag=lucenia-custom-plugin .
# Start the container from the custom image
docker run -p 9200:9200 -p 9600:9600 -v /usr/share/lucenia/data lucenia-custom-plugin

You can also use a Dockerfile to pass your own certificates for use with the Security implementation:

FROM lucenia/lucenia:0.1.0
COPY --chown=lucenia:lucenia lucenia.yml /usr/share/lucenia/config/
COPY --chown=lucenia:lucenia my-key-file.pem /usr/share/lucenia/config/
COPY --chown=lucenia:lucenia my-certificate-chain.pem /usr/share/lucenia/config/
COPY --chown=lucenia:lucenia my-root-cas.pem /usr/share/lucenia/config/

350 characters left

Have a question? .