Installing Lucenia
This section details how to install Lucenia on your host, including which operating systems are compatible with Lucenia, which ports to open, and which important settings to configure on your host.
Operating system compatibility
Lucenia is compatible with Red Hat Enterprise Linux (RHEL) and Debian-based Linux distributions that use systemd
, such as Amazon Linux, and Ubuntu Long-Term Support (LTS). While Lucenia should work on most Linux distributions, we only test a subset.
The following table lists the operating system versions that we are currently testing on:
OS | Version |
---|---|
CentOS | 7 |
Rocky Linux | 8 |
Alma Linux | 8 |
Amazon Linux | 2/2023 |
Ubuntu | 20.04 |
Windows Server | 2019 |
File system recommendations
Avoid using a network file system for node storage in a production workflow. Using a network file system for node storage can cause performance issues in your cluster due to factors such as network conditions (like latency or limited throughput) or read/write speeds. You should use solid-state drives (SSDs) installed on the host for node storage where possible.
Java compatibility
The Lucenia distribution for Linux ships with a compatible Adoptium JDK version of Java in the jdk
directory. To find the JDK version, run ./jdk/bin/java -version
. For example, the Lucenia 0.1.0 tarball ships with Java 19.0.1+9 (non-LTS), Lucenia 1.3.0 ships with Java 11.0.14.1+1 (LTS), and Lucenia 2.0.0 ships with Java 17.0.2+8 (LTS). Lucenia is tested with all compatible Java versions.
Lucenia Version | Compatible Java Versions | Bundled Java Version |
---|---|---|
0.x | 11, 17, 21 | 21.0.3+9 |
To use a different Java installation, set the LUCENIA_JAVA_HOME
or JAVA_HOME
environment variable to the Java install location. For example:
export LUCENIA_JAVA_HOME=/path/to/lucenia-0.1.0/jdk
Network requirements
The following ports need to be open for Lucenia components.
Port number | Lucenia component |
---|---|
443 | OpenSearch Dashboards in Amazon OpenSearch Service with encryption in transit (TLS) |
5601 | OpenSearch Dashboards |
9200 | Lucenia REST API |
9300 | Node communication and transport (internal), cross cluster search |
9600 | Performance Analyzer |
Important settings
For production workloads, make sure the Linux setting vm.max_map_count
is set to at least 262144. Even if you use the Docker image, set this value on the host machine. To check the current value, run this command:
cat /proc/sys/vm/max_map_count
To increase the value, add the following line to /etc/sysctl.conf
:
vm.max_map_count=262144
Then run sudo sysctl -p
to reload.
For Windows workloads, you can set the vm.max_map_count
running the following commands:
wsl -d docker-desktop
sysctl -w vm.max_map_count=262144
The sample docker-compose.yml file also contains several key settings:
-
bootstrap.memory_lock=true
Disables swapping (along with
memlock
). Swapping can dramatically decrease performance and stability, so you should ensure it is disabled on production clusters.Enabling the
bootstrap.memory_lock
setting will cause the JVM to reserve any memory it needs. The Java SE Hotspot VM Garbage Collection Tuning Guide documents a default 1 gigabyte (GB) Class Metadata native memory reservation. Combined with Java heap, this may result in an error due to the lack of native memory on VMs with less memory than these requirements. To prevent errors, limit the reserved memory size using-XX:CompressedClassSpaceSize
or-XX:MaxMetaspaceSize
and set the size of the Java heap to make sure you have enough system memory. -
LUCENIA_JAVA_OPTS=-Xms512m -Xmx512m
Sets the size of the Java heap (we recommend half of system RAM).
-
nofile 65536
Sets a limit of 65536 open files for the Lucenia user.
-
port 9600
Allows you to access Performance Analyzer on port 9600.
Do not declare the same JVM options in multiple locations because it can result in unexpected behavior or a failure of the Lucenia service to start. If you declare JVM options using an environment variable, such as LUCENIA_JAVA_OPTS=-Xms3g -Xmx3g
, then you should comment out any references to that JVM option in config/jvm.options
. Conversely, if you define JVM options in config/jvm.options
, then you should not define those JVM options using environment variables.
Important system properties
Lucenia has a number of system properties, listed in the following table, that you can specify in config/jvm.options
or LUCENIA_JAVA_OPTS
using -D
command line argument notation.
Property | Description |
---|---|
lucenia.xcontent.string.length.max=<value> | By default, Lucenia does not impose any limits on the maximum length of the JSON/YAML/CBOR/Smile string fields. To protect your cluster against potential distributed denial-of-service (DDoS) or memory issues, you can set the lucenia.xcontent.string.length.max system property to a reasonable limit (the maximum is 2,147,483,647), for example, -Dlucenia.xcontent.string.length.max=5000000 . |
lucenia.xcontent.fast_double_writer=[true|false] | By default, Lucenia serializes floating-point numbers using the default implementation provided by the Java Runtime Environment. Set this value to true to use the Schubfach algorithm, which is faster but may lead to small differences in precision. Default is false . |
lucenia.xcontent.name.length.max=<value> | By default, Lucenia does not impose any limits on the maximum length of the JSON/YAML/CBOR/Smile field names. To protect your cluster against potential DDoS or memory issues, you can set the lucenia.xcontent.name.length.max system property to a reasonable limit (the maximum is 2,147,483,647), for example, -Dlucenia.xcontent.name.length.max=50000 . |
lucenia.xcontent.depth.max=<value> | By default, Lucenia does not impose any limits on the maximum nesting depth for JSON/YAML/CBOR/Smile documents. To protect your cluster against potential DDoS or memory issues, you can set the lucenia.xcontent.depth.max system property to a reasonable limit (the maximum is 2,147,483,647), for example, -Dlucenia.xcontent.name.length.max=1000 . |
lucenia.xcontent.codepoint.max=<value> | By default, Lucenia imposes a limit of 52428800 on the maximum size of the YAML documents (in code points). To protect your cluster against potential DDoS or memory issues, you can change the lucenia.xcontent.codepoint.max system property to a reasonable limit (the maximum is 2,147,483,647). For example, -Dlucenia.xcontent.codepoint.max=5000000 . |