Couchbase Elasticsearch Connector
The Couchbase Elasticsearch Connector replicates your documents from Couchbase Server to Elasticsearch in near real time. The connector uses the high-performance Database Change Protocol (DCP) to receive notifications when documents change in Couchbase.
|
Note
|
If you’re looking for the Elasticsearch Plug-in flavor of the connector, that’s in a different branch. |
This product is neither affiliated with nor endorsed by Elastic. Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries.
Building the connector from source
The connector distribution may be built from source with the command:
./gradlew build
The distribution archive will be generated under build/distributions.
During development, it might be more convenient to run:
./gradlew installDist
which creates build/install/couchbase-elasticsearch-connector as a $CBES_HOME directory.
Running the integration tests
A local Docker installation is required for these tests. To quickly test using only the latest Couchbase and Elasticsearch:
./gradlew integrationTest
To test against all supported versions of Couchbase and Elasticsearch:
./gradlew exhaustiveTest
IntelliJ IDEA setup
Because the project uses annotation processors, some fiddly setup is required when importing the project into IntelliJ IDEA.
Building a Docker image
Use Dockerfile to build a Docker image from source using Gradle.
The version should be set in build.gradle before running.
docker build -t imagename:tag .
Use Dockerfile.download to build a Docker image from released binaries hosted at packages.couchbase.com.
docker build -f Dockerfile.download -t imagename:tag --build-arg VERSION=<version>
where <version> is the latest tag from the connector’s GitHub repo.
Running a Docker image
The built docker image can be configured using volume mounts.
The /opt/couchbase-elasticsearch-connector/config directory should contain the configuration files, and the /opt/couchbase-elasticsearch-connector/secrets directory should contain the secrets.
Find example configuration files in the src/dist directory.
Be sure to rename example-connector.toml to default-connector.toml.
docker run -p 31415:31415 -v ./config:/opt/couchbase-elasticsearch-connector/config -v ./secrets:/opt/couchbase-elasticsearch-connector/secrets -e CBES_GROUPNAME=groupname image:tag
It is also valid to pass environment variables in via the Docker command line, which can then be used to substitute values in default-connector.toml.
Port 31415 can be accessed via HTTP to get metrics.
Running in Kubernetes
The connector can run in Kubernetes.
See the examples in the examples/kubernetes directory, and the documentation for more details.