Spatial reprojection ingest processor
Lucenia’s spatial reprojection ingest processor enables automatic transformation of geometries into a target coordinate reference system (CRS) at index time. This simplifies your geospatial pipeline by eliminating the need for external ETL (Extract, Transform, Load) tools or maintaining duplicate indexes for each projection. Whether your data sources use EPSG codes (standard identifiers for coordinate systems defined by the European Petroleum Survey Group), UTM zones, or custom local grids, Lucenia can reproject them directly during ingestion into a consistent CRS for fast and accurate spatial indexing.
By performing reprojection during ingestion, Lucenia ensures that all spatial data is stored in the optimal format for querying - enabling consistent integration with search tools, mapping clients, analytics pipelines, or STAC catalogs without requiring runtime transformations or projection-aware query logic.
Request fields
The following table lists all available request fields.
| Field | Type | Description | Required |
|---|---|---|---|
field | String | The source geometry field to reproject. WKT string or GeoJSON. | Yes |
target_field | String | Field that receives the reprojected geometry. Defaults to field. | No |
ignore_missing | Boolean | If true and field is missing, the processor leaves the document unchanged. Defaults to false. | No |
source_crs | String | Coordinate reference system of the geometry in field. | Yes |
target_crs | String | Coordinate reference system to reproject into. Defaults to EPSG:4326. | No |
tolerance | Double | Accuracy passed to the CRS transform, in coordinate units of that transform. Defaults to 0.01. | No |
shape_type | String | Field mapping type used when parsing field. Defaults to geo_shape. | No |
provider | String | CRS provider name. Defaults to sis. | No |
ensure_indexable | Boolean | After conversion, drop the fewest interior holes that fail the tessellation check a geo_shape field uses. An outer ring that fails the check is written as-is; the index request then reports a tessellation error. Defaults to false. See Indexable output. | No |
Example
The following examples demonstrate how to creat and use an ingest pipeline with a reproject processor to transform geometries from the standard World Geodetic System of 1984 latitude, longitude (decimal degrees) projection to the Mercator X, Y (meters) projection commonly used by Web Mapping applications. This ensures that incoming data is consistently stored in the desired spatial reference system at the time of indexing.
Create an Index Reprojection Processor
The example below creates an ingest processor to reproject incoming document geometry from Mercator into WGS84 projection.
PUT /_ingest/pipeline/reproject_to_wgs84
{
"description" : "reproject spatial geometries from EPSG:3857 X,Y coordinate system to EPSG:4326 lat,lon coordinate system during ingest",
"processors" : [
{
"reproject" : {
"field" : "geometry",
"source_crs" : "EPSG:3857",
"shape_type" : "shape"
}
}
]
}
The input geometry field in the document is treated as a shape and dynamically reprojected from the EPSG:3857 (X, Y) coordinate system to EPSG:4326 (lat, lon), using the same field name. This enables conversion of a user-provided shape query in Easting/Northing (X, Y) to a geo_shape query in (lat, lon), aligning with the coordinate system of the indexed data.
Using the Ingest Reprojection Processor
To reproject geometries at ingest time, specify the pipeline name using the pipeline parameter when indexing documents. For example, the following request ingests data provided in the EPSG:3857 (X, Y) coordinate system and uses the ingest pipeline to reproject it into EPSG:4326 (latitude, longitude) before storing it in the index.
PUT /wgs84_data/_doc?pipeline=reproject_to_wgs84
{
"geometry": {
"type": "envelope",
"coordinates": [ [1812358.0, 6139791.0], [1812369.2, 6139780.1] ]
}
}
Now the user can query the geometry field in the wgs84_data index using the native WGS84 coordinate system.
GET /wgs84_data/_search
{
"query" : {
"geo_shape" : {
"geometry" : {
"shape" : {
"type" : "envelope",
"coordinates": [ [16.2806889, 48.1975963], [16.2807895, 48.1975310] ]
},
"relation" : "intersects"
}
}
}
}
An example result is provided below:
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0,
"hits": [
{
"_index": "wgs84_data",
"_id": "roads_1_0",
"_score": 0,
"_source": {
"filename": "roads_1",
"feature_id": 0,
"geometry": "LINESTRING (16.2806999 48.1975878,16.2806995 48.1976023,16.2806986 48.1976328,16.2807182 48.1984326)",
"properties": {
"osm_id": 149673,
"fclass": "tertiary",
"code": "5115",
"maxspeed": 30,
"length": 92,
"edge_id": 1,
"oneway": "B",
"speed": 27,
"layer": 0,
"ref": null,
"ete": 13,
"name": "WaidhausenstraÃe",
"bridge": "F",
"lastchange": "2022-01-23T01:36:31Z",
"tunnel": "F"
}
}
}
]
}
Indexable output
Set ensure_indexable to true when the reprojected geometry will be stored in a geo_shape field. After conversion, the processor runs the same tessellation check the indexer uses:
- Polygons that pass the check are left unchanged.
- Polygons that fail the check keep the outer ring and drop the fewest interior holes that make the shape pass. The stored geometry is that result; dropped holes are not listed as extra fields.
- If the outer ring fails the check, the processor still writes the converted rings. The index request then reports a tessellation error for that document.
Default false writes every converted ring as-is. Set true on vectorize and image_segment pipelines that write a geo_shape field.
reproject reads one geometry field. vectorize writes an array of regions, so wrap reproject in foreach. Set source_crs to the raster CRS (the crs value on each region, for example EPSG:5070):
{
"foreach": {
"field": "segments",
"processor": {
"reproject": {
"field": "_ingest._value.mask_xy_shape",
"target_field": "_ingest._value.mask",
"source_crs": "EPSG:5070",
"target_crs": "EPSG:4326",
"shape_type": "shape",
"ensure_indexable": true
}
}
}
}
Map segments as nested and segments.mask as geo_shape. _simulate runs this processor (including hole dropping). A remaining outer-ring failure appears when the document is indexed.
Registering a Custom Reprojection Provider
Developers can also implement their own reprojection logic by implementing the CRSProviderInterface and registering it via the Java Service Provider Interface (SPI) system through their own search plugin.
Below is an example class for implementing a custom projection provider:
public class MyReprojectionProvider implements CRSProviderInterface<MyCRS, MyTransform> {
private static final String NAME = "my_crs_provider";
@Override
public String getName() {
return NAME;
}
@Override
public MyTransform getTransform(final MyCRS fromCRS, final MyCRS toCRS, Object... extraArgs) throws Exception {
MyBBox bbox = (extraArgs != null && extraArgs.length > 0 && extraArgs[0] instanceof MyBBox)
? (MyBBox) extraArgs[0]
: null;
return MyCRS.findMathOperation(fromCRS, toCRS, bbox).getMathTransform();
}
@Override
public MyCRS getCRS(final String crsString) throws Exception {
return AccessController.doPrivileged((PrivilegedAction<MyCRS>) () -> {
try {
return MyCRS.forCode(crsString);
} catch (FactoryException e) {
throw new RuntimeException(e);
}
});
}
@Override
public CRSHandler<MyCRS> createCRSHandler(
final String fromCRS,
final String toCRS,
final GeometryProcessorFieldType shapeFieldType
) throws Exception {
MyCRS toReferenceSystem = toCRS != null ? getCRS(toCRS) : MyHandler.DEFAULT_TO_CRS;
return new MyHandler(getCRS(fromCRS), toReferenceSystem, getTransform(getCRS(fromCRS), toReferenceSystem), shapeFieldType);
}
static class MyHandler implements CRSHandler<MyCRS> {
static final MyCRS DEFAULT_TO_CRS;
private final MyTransform transform;
private final MyPoint reusableFrom;
private final MyPoint reusableTo;
static {
try {
DEFAULT_TO_CRS = MyCRS.forCode(DEFAULT_TARGET_CRS_EPSG_CODE);
} catch (FactoryException e) {
throw new RuntimeException(e);
}
}
MyHandler(
final MyCRS fromCRS,
final MyCRS toCRS,
final MyTransform transform,
final GeometryProcessorFieldType shapeFieldType
) {
this.transform = transform;
this.reusableFrom = new MyPoint(fromCRS);
this.reusableTo = new MyPoint(toCRS);
}
@Override
public MyCRS getFromCRS() {
return reusableFrom.getCoordinateReferenceSystem();
}
@Override
public MyCRS getToCRS() {
return reusableTo.getCoordinateReferenceSystem();
}
@Override
public void reproject(final double[] from, double[] to, final double tolerance) throws Exception {
// Custom reprojection logic
}
}
}
To activate the provider with Java SPI, add the following line to ./resources/META-INF/services/io.skylite.geographic.referencing.CRSProviderInterface in the generated jar file in your plugin zip: com.example.geographic.referencing.MyReprojectionProvider
You will now be able to use your custom provider by providing it's registered NAME in the provider parameter of the search processor:
PUT /_ingest/pipeline/reproject_to_wgs84
{
"description" : "reproject spatial geometries from EPSG:3857 X,Y coordinate system to EPSG:4326 lat,lon coordinate system during ingest",
"processors" : [
{
"reproject" : {
"field" : "geometry",
"source_crs" : "EPSG:3857",
"shape_type" : "shape",
"provider" : "my_crs_provider"
}
}
]
}
Lucenia will discover and register your custom provider automatically on startup.
Configuring the Apache SIS Provider
The default Apache SIS provider uses an embedded Derby database to store EPSG coordinate reference system definitions. When running Lucenia with the SecurityManager enabled (the default), you must configure Derby to use a safe directory and grant appropriate permissions.
Step 1: Configure JVM Options
Add the following to $LUCENIA_HOME/config/jvm.options:
# --------------------------------------------------------------------
# Apache SIS / Derby embedded database sandbox isolation
# --------------------------------------------------------------------
# Force Derby to use a safe directory so it doesn't try to read
# derby.properties from the JVM working directory (blocked by SecurityManager)
# Note: macOS uses /private/tmp/lucenia/derby
-Dderby.system.home=/tmp/lucenia/derby
# Redirect Derby's error logs to standard error instead of derby.log
-Dderby.stream.error.method=java.lang.System.err
# Use a global security policy to grant write permission to the Derby directory
-Djava.security.policy=config/global.policy
Step 2: Create a Global Security Policy
Create or update the $LUCENIA_HOME/config/global.policy file with the following permissions:
grant {
// Required by the Derby embedded database
permission javax.management.MBeanServerPermission "createMBeanServer";
permission javax.management.MBeanServerPermission "newMBeanServer";
permission javax.management.MBeanPermission "org.apache.derby.*", "*";
permission javax.management.MBeanTrustPermission "register";
// File permissions for the Derby database directory
// Use /private/tmp/lucenia/derby on macOS
permission java.io.FilePermission "/private/tmp/lucenia/derby", "read,write,delete";
permission java.io.FilePermission "/private/tmp/lucenia/derby/-", "read,write,delete";
permission java.io.FilePermission "/private/tmp/lucenia", "read";
};
On macOS, the /tmp directory is a symlink to /private/tmp, so your policy file should reference /private/tmp/lucenia/derby to ensure the permissions apply correctly.
Summary
Lucenia’s reprojection processors allow seamless integration of data and queries across coordinate systems without requiring reindexing or external tools. Whether your users operate in Web Mercator, UTM zones, or Mars-based CRS, Lucenia handles the transformation transparently using robust, high-performance ingest pipelines.