Please use the form below to provide your feedback. Because your feedback is valuable to us, the information you submit in this form is recorded in our issue tracking system (JIRA), which is publicly available. You can track the status of your feedback using the ticket number displayed in the dialog once you submit the form.

A newer version of this documentation is available.

View Latest

Release Notes

    +
    Release notes, installation instructions, and download archive for the Couchbase Elasticsearch Connector.

    Installation

    Scroll down to the version you want, then click the "Download" link to get the full connector distribution. Refer to Getting Started for detailed installation instructions.

    Version 4.2.15 (2022-01-18)

    Download

    This release upgrades Log4j again (sigh).

    Regarding CVE-2021-44832: The connector’s out-of-the-box logging configuration does not use Log4j’s JDBC appender. You may still wish to upgrade to avoid false positives from vulnerability scanners.

    Enhancements

    • CBES-241: Upgrade Log4j from 2.17.0 to 2.17.1.

    Version 4.2.14 (2021-12-21)

    Download

    This release upgrades Log4j from 2.15.0 to 2.17.0.

    This prevents vulnerability scanners from flagging Log4j 2.15.0 as a potential security risk.

    All versions of the connector are immune to CVE-2021-45046 and CVE-2021-45105 because the connector does not use the Thread Context Map / Mapped Diagnostic Context (MDC) feature of Log4j.

    Enhancements

    • CBES-232: Upgrade Log4j from 2.15.0 to 2.17.0.

    Version 4.2.13 (2021-12-10)

    Download

    This release fixes a high severity vulnerability related to Log4j 2. All users should upgrade the connector to 4.2.13 (or 4.3.3 and later) as soon as possible.

    Bug Fixes

    • CBES-230: Upgrade Log4j 2 from 2.14.1 to 2.15.0 to address CVE-2021-44228.

    Version 4.2.12 (2021-07-20)

    Download

    This release upgrades various dependencies to the latest versions and improves how permanent indexing failures are logged.

    Enhancements

    • CBES-213: When logging permanent indexing failures, the connector now sanitizes the contents of the failure message to ensure sensitive information is redacted.

    • CBES-209: Upgraded the Couchbase Java SDK from 3.1.3 to 3.2.0.

    • CBES-217: Upgraded the DCP client from 0.34.0 to 0.35.0.

    • CBES-211: Upgraded various dependencies to the latest versions.

    Version 4.2.11 (2021-05-18)

    Download

    This release makes it easier for custom tools to parse config files that have environment variable placeholders.

    Enhancements

    • CBES-206: Integer and boolean config properties can now be specified as strings. For example: 7 and '7' are now both valid for an integer property, and true and 'true' are now both valid for a boolean property. This lets you use environment variable placeholders for integer and boolean properties without invalidating the TOML syntax — just enclose them in quotes, like: '${SOME_INTEGER}'. The connector resolves placeholders before parsing the TOML, so it doesn’t care about the invalid syntax; this change just makes it easier for other tools to parse the connector config file.

    Version 4.2.10 (2021-04-20)

    Download

    This release brings minor improvements to monitoring and configurability.

    Enhancements

    • CBES-202: The connector now waits until it has connected to both Couchbase and Elasticsearch before starting the HTTP server for the /metrics endpoint. This makes it more useful as a "readiness" probe.

    • CBES-204: Added a new [couchbase.env] config section for tuning Couchbase Java SDK settings.

    • CBES-203: Upgraded the Couchbase Java SDK from 3.1.2 to 3.1.3.

    • CBES-205: Upgraded the DCP client from 0.33.0 to 0.34.0.

    Version 4.2.9 (2021-03-16)

    Download

    This release improves diagnostic logging and fixes an issue with scopes & collections.

    Enhancements

    • CBES-192: When the new logDocumentLifecycle config property is set to true in the logging section, the connector writes detailed log entries as each document flows through the connector.

    • CBES-198: Log redaction is now configurable via the new redactionLevel config property in the logging section.

    • CBES-199: Upgraded the DCP client from 0.32.0 to 0.33.0.

    Bug Fixes

    • CBES-193: If two documents in different collections had the same ID, and both were processed by the connector in the same batch, one would be incorrectly flagged as a duplicate and dropped, even if it had a different destination index. With this fix, documents are never considered duplicates if they have different destination indexes. This issue only affected users experimenting with the Scopes & Collections feature.

    Version 4.2.8 (2021-02-16)

    Download

    This release adds uncommitted support for client certificate authentication (mTLS), adds hostname verification for secure DCP connections, and improves the stability of the connector.

    Enhancements

    • CBES-183: When secure connections are enabled, it is now possible to authenticate with Couchbase and/or Elasticsearch using an X.509 certificate instead of a username & password. See the Client Certificates documentation for details. (This feature is added as "uncommitted", meaning it may change without notice.)

    • CBES-187: Errors during an early phase of connector startup were written to the console instead of being logged. Now these errors will appear in the log as well.

    • CBES-189: Upgraded the Couchbase Java SDK from 3.1.0 to 3.1.2.

    • CBES-188: Upgraded the DCP client from 0.31.0 to 0.32.0.

    • JDCP-188: Previously, TLS hostname verification was done only for the Couchbase Java client connection; now the DCP client connection is verified as well.

    Bug Fixes

    • JDCP-183: If an invalid stream offset is detected, the connector will now fail fast instead of potentially corrupting the saved checkpoint.

    • JDCP-184: Resolved an issue that could cause a flow control deadlock when streaming from a subset of a bucket’s collections or scopes.

    Version 4.2.7 (2021-01-19)

    Download

    For those running the connector in Autonomous Operations mode, this release improves compatibility with recent Consul versions.

    Enhancements

    • CBES-185: The connector no longer fails to start when using Consul version 1.8.4 and later. The list of tested and supported Consul versions now includes Consul 1.9.1.

    Version 4.2.6 (2020-12-15)

    Download

    This release brings improvements to monitoring and adds an option for disabling TLS certificate hostname verification.

    Enhancements

    • CBES-184: The connector now exposes Prometheus metrics at /metrics/prometheus. Prometheus metrics are "Uncommitted API" and subject to change between releases without notice. The original Dropwizard JSON metrics are still available at /metrics, and can now be accessed at the alternative path /metrics/dropwizard.

    • CBES-181: Added a new config property, hostnameVerification (default: true). Couchbase TLS certificate hostname verification was silently enabled by default in version 4.2.3. If this caused problems for your deployment, and you are unable to issue certificates matching the Couchbase server nodes, you can now disable hostname verification by setting this new config property to false.

    • CBES-182: Upgraded the Couchbase Java SDK from 3.0.9 to 3.1.0.

    Version 4.2.5 (2020-11-17)

    Download

    This release fixes an issue with configuring secure connections to Elasticsearch using custom ports. It also adds a new metric for monitoring the replication backlog.

    Enhancements

    • CBES-121: Added new cbes.backlog metric which estimates the number of Couchbase document changes yet to be processed. This is a general indication of how well the connector is keeping up with changes in Couchbase. Note that the count only includes changes in the Couchbase partitions handled by the connector instance reporting the metric. Unlike cbes.backfill, this value is dynamic; it goes up when changes happen in Couchbase, and goes down as the changes are processed by the connector.

    • CBES-178: Upgraded the DCP client from 0.30.0 to 0.31.0.

    Bug Fixes

    • CBES-179: Fixed an issue that prevented the connector from establishing a secure connection to Elasticsearch if a custom port was specified.

    Version 4.2.4 (2020-10-20)

    Download

    This release improves compatibility with Couchbase Cloud, and fixes a few minor issues.

    Enhancements

    • CBES-170: Bootstrap performance is improved when specifying custom ports.

    • CBES-175: Upgraded the Couchbase Java SDK from 3.0.6 to 3.0.9. The connector no longer logs spurious warnings about being unable to fetch collections manifests.

    • CBES-177: Upgraded the DCP client from 0.28.0 to 0.30.0.

    Bug Fixes

    • CBES-173: Fixed a regression in version 4.2.3 that broke alternate address resolution. The connector now handles DNS SRV and alternate addresses correctly, and can connect to Couchbase Cloud or other network environments that use alternate addresses.

    • CBES-172: Removed duplicate command line scripts from the ZIP archive. You can now unzip the archive without being prompted about overwriting the duplicate files.

    Version 4.2.3 (2020-07-21)

    Download

    This release adds "uncommitted" support for Collections and Scopes, a new feature planned for Couchbase Server 7.0.

    Enhancements

    • CBES-163: Type definitions now have a matchOnQualifiedKey property that lets a rule match against the qualified document name, which includes the scope and collection. This enables type definition rules that write to an Elasticsearch index whose name matches the Couchbase collection name.

    • CBES-164: The [couchbase] config section now has optional scope and collection properties that limit the replication to a single scope or to a set of collections.

    • CBES-165: The [couchbase] config section now has optional metadataCollection property that controls which collection is used to store metadata like replication checkpoints.

    Known Issues

    • CBES-170: If you specify a custom port for a Couchbase host, it can take a long while to connect, and the connector will log lots of warnings about being unable to connect to the KV service.

    The workaround is to bootstrap using a KV port (default 11210) instead of a manager port (default 8091), and to explicitly tag the port as belonging to the KV service, like this:

    hosts = ['example.com:12345=kv']

    Version 4.2.2 (2020-05-19)

    Download

    The connector now behaves better in environments where DNS entries are highly dynamic. Instead of caching resolved hostnames, it now resolves hostnames prior to every connection attempt.

    This release also improves decompression performance, activates Netty native transports, and fixes a bug that prevented the 'couchbase.network' config setting from being honored.

    The compatibility matrix is updated to add support for Elasticsearch 7.6 and 7.7. Elasticsearch versions prior to 6.6 are dropped due to end of life, with the exception of 5.6.16.

    Bug Fixes

    • CBES-155 Couchbase client ignores 'couchbase.network' config setting

    Enhancements

    • JDCP-163 Force DNS lookups on reconnect

    • JDCP-156 Enable Netty native transports by default

    • JDCP-82 Decompress with org.iq80.snappy instead of Netty

    • CBES-158 Upgrade Couchbase SDK from 2.7.11 to 2.7.15

    • CBES-154 Upgrade DCP client from 0.25.0 to 0.28.0

    • JDCP-146 Upgrade Netty from 4.0.56 to 4.1.48

    Version 4.2.1 (2020-01-21)

    Download

    This maintenance release addresses an issue with shutdown hooks that could prevent the connector from terminating in some circumstances.

    Also fixed in this release, bulk request timeouts longer than 30 seconds are now honored instead of being reduced to 30 seconds.

    Metrics from the Couchbase DCP client are now included in the metrics report, along with gauges for CPU load.

    Bug Fixes

    • CBES-147 Stuck shutdown hook can prevent/delay JVM termination

    • CBES-149 Bulk request timeout is capped at 30 seconds

    Enhancements

    • CBES-143 Report DCP metrics

    • CBES-148 Report CPU usage metrics

    • CBES-150 Upgrade Couchbase client to 2.7.11 and DCP client to 0.25.0

    Version 4.2.0 (2019-10-15)

    Download

    Hot on the heels of 4.1, we’re releasing 4.2 with support for connecting directly to an Amazon Elasticsearch Service instance. There’s a new [elasticsearch.aws] config section for specifying the AWS region of the service. Amazon credentials are obtained from the Default Credential Provider Chain.

    Also new in 4.2, the cbes-consul command now takes an optional --consul-config argument which points to a separate config file where you can specify a Consul ACL token.

    On the version compatibility front, we’ve added support for Elasticsearch 7.4 and removed support for Elasticsearch 5.4 (which reached EOL on 2018-11-04).

    Enhancements

    • CBES-129 Support direct connections to Amazon Elasticsearch Service

    • CBES-140 Support ACL Token Authentication when communicating with Consul

    • CBES-141 Extend support coverage to Elasticsearch 7.4

    Version 4.1.0 (2019-09-05)

    Download

    We are excited to unveil the new Autonomous Operations (AO) mode with major improvements to the availability and manageability of the connector. When the connector is deployed in AO mode, worker processes use your HashiCorp Consul cluster to communicate with each other and automatically distribute the replication workload. You can add or remove worker processes at any time without having to manually stop and reconfigure all of the workers. Any worker that fails a health check is automatically removed, and its workload is redistributed among remaining workers.

    The new cbes-consul command line tool is used to start a worker in AO mode. It also provides streamlined checkpoint management and the ability to reconfigure or pause/resume all of the workers in an AO group at once.

    Also new in this release is support for multi-network configuration. This feature allows the connector to talk to Couchbase Server nodes that have been configured to advertise alternate network addresses for connecting to the node from outside a container/cloud networking environment. The new network property in the [couchbase] section of the configuration gives you control over network selection (although the default value of auto is appropriate for most cases).

    Finally, the range of supported Elasticsearch versions is extended to include 7.1, 7.2. and 7.3.

    Enhancements

    • CBES-65 Autonomous Operations Mode with Consul

    • CBES-135 Expose multi-network config options

    • CBES-138 Upgrade to Couchbase client 2.7.9 and DCP client 0.24.0

    Version 4.0.2 (2019-05-21)

    Download

    This maintenance release fixes a bug that prevented some versions of Couchbase Server from rebalancing when the connector is running.

    It also adds compatibility with the official Docker images for Elasticsearch 6.7.x and 7.0.x, and is the first version tested against OpenJDK 8 and OpenJDK 11.

    Enhancements

    • CBES-122 Add support for OpenJDK

    • CBES-123 Support Elasticsearch 6.7 & 7.0 docker images

    • CBES-125 Suppress "types removal" warnings from Elasticsearch 7.0

    Bug Fixes

    • CBES-128 Couchbase Server fails to rebalance if Elasticsearch connector is running

    Version 4.0.1 (2019-04-15)

    Download

    This maintenance release improves the stability of the connector and adds new configuration options.

    Enhancements

    • CBES-90 Ability to use environment variables inside config

    • CBES-107 Misleading error message when can’t connect to Elasticsearch

    • CBES-110 Need document routing to support join

    • CBES-114 Allow saving checkpoints in a different bucket

    Bug Fixes

    • CBES-117 Connector exits on values that fail to parse

    Version 4.0.0 (2018-10-12)

    Download

    New in this version

    • The connector is now a standalone process instead of an Elasticsearch plug-in.

    • Compatible with Elasticsearch versions 5 and 6.

    • Support for secure connections to Couchbase and Elasticsearch.

    • Tools for managing replication checkpoints.

    • A "rejection log" for documents Elasticsearch permanently refuses to index.

    • Configurable document structure (omit metadata if you don’t need it).

    • The connector now listens for document changes using the high performance Couchbase Database Change Protocol (DCP).

    Things to be aware of

    This is a major version upgrade. Because the plug-in and the standalone connector are so different, there is no online upgrade process. See the Migration documentation for details.

    • Parent-child relationships are no longer supported, as this feature was removed in ES 6.

    • Routing documents to specific Elasticsearch shards is not implemented. Please let us know if this feature is still relevant for your deployment.

    Plug-in Version 3.0.2 (2018-09-18)

    Maintenance release for improved compatibility and stability.

    Bug Fixes

    • CBES-82: NoClassDefFoundError under Java 9.

    • CBES-83: VersionConflictEngineException in storeUUID if document already exists. (Causes XDCR replication to be removed.)

    Plug-in Version 3.0.1 (2018-02-18)

    The birch development line has been retired. The cypress line now supports all 5.x versions of Elasticsearch.

    RegexParentSelector now reads the parent ID format from the correct config property (couchbase.parentSelector.documentTypesParentFormat.<type>). If you previously put the parent ID format in the documentTypeParentFields property as a workaround, please update your configuration.

    The connector can now delete child documents and other documents that have custom routing. If you’re using RegexParentSelector then this feature has no additional overhead. Otherwise, for each document whose routing cannot be derived from the document ID, the connector creates a separate signpost document to record the routing. Be aware that child documents created by previous versions of the connector are not eligible for deletion, since they don’t have signposts (you’ll see "missing signpost" warnings in the log when the connector tries to delete those documents).

    The signposts have a document type of couchbaseSignpost. The mapping for this type must store the contents of the meta field. This is already the case if you’re using the default mapping template included in the plug-in distribution.

    Bug Fixes

    • CBES-49: RegexParentSelector reads parent ID format from wrong config property.

    • CBES-50: RoutingMissingException when deleting child documents.

    Plug-in Version 3.0.0 (2017-11-22)

    With this release, the Couchbase plug-in for Elasticsearch moves to a new versioning and branch management strategy that allows for simultaneous releases across ES versions (see [Version Compatibility]). All the versions are expected to work and are supported. However, versions which have received more testing and are officially supported are:

    • 3.0.0-cypress-es5.6.4

    • 3.0.0-birch-es5.2.2

    • 3.0.0-alder-es2.4.0

    Don’t be alarmed by the major version bump; upgrading from version 2.x of the plug-in should be seamless, and is recommended for all users. The code has just been relabeled for easier maintenance.

    New Feature

    • Adds the couchbase.pipeline config for specifying the ingestion pipeline.

    Enhancements

    • Improves logging for indexing errors.

    • Uses Dropwizard Metrics to collect and log richer stats.

    Bug Fixes

    • CBES-48: Connection counter leak could cause spurious TooManyConcurrentConnections errors.

    • #153: ClassCastException error when the TTL is of type Long instead of Integer.

    Known issues

    The Elasticsearch Plug-in does not support IPv6. So to use the plug-in, the Couchbase Server and Elasticsearch clusters will need to run on instances which are addressable with IPv4.

    Plug-in Version 2.2 (2017-02)

    This release note applies to the 2.2 version of the Elasticsearch Transport Plug-in (February 2017). It adds a number of bug fixes. See Installation and Configuration for versioning and compatibility information.

    Plug-in Version 2.1.1 (2015-09)

    This release note applies to the 2.1.1 version of the Elasticsearch Transport Plug-in (September 2015). It adds compatibility with newer Elasticsearch versions up to 1.7.x, multiple new features, and quite a few bug fixes, including several that solve issues found in 2.1.0. In particular, this release fixes a long-standing bug with an incorrect concurrent bulk request counter, which could eventually cause the plug-in to stop accepting requests from Couchbase Server altogether.

    Some configuration option-names changed.

    Elasticsearch plug-in version Couchbase versions Elasticsearch versions

    2.1.1

    2.5.x - 4.x

    1.3.0 - 1.7.x

    Plug-in Version 2.0 (2014-10)

    This release note is for the Elasticsearch plug-in release 2.0 GA (October 2014). Elasticsearch plug-in version 2.0 is compatible with:

    • Elasticsearch 1.3.0.

    • Couchbase Server 3.0

    • Couchbase Server 2.5.x (backward compatible)

    The new feature(s) available in Elasticsearch Plug-in v2.0:

    • Support more than one document type in Elasticsearch. (MB-12284)

    The following are known issues:

    • The att_reason value for non-JSON documents changed from non-JSON mode to invalid_json. If a Couchbase cluster has a lot of deletes, the Elasticsearch log could fill up with a lot of messages. (CBES-31)

    Plug-in Version 1.3.0 (2014-04)

    This release note is for the Elasticsearch plug-in release 1.3.0 GA (April 2014). This release is compatible only with Elasticsearch 1.0.1.

    This release is compatible with Couchbase Server 2.5.x, and it is backward compatible with earlier 2.x releases.

    • Support for new XDCR checkpoint protocol. (CBES-26)

    • Fixed failure handling due to bounded queue with Elasticsearch 1.x. (CBES-27)

    Plug-in Version 1.2.0 (2013-10)

    This release note is for the Elasticsearch plug-in release 1.2.0 GA (October 2013). This release adds compatibility with Elasticsearch 0.90.5.

    This release is compatible with Couchbase Server 2.2, and it is backward compatible with earlier 2.x releases.

    Plug-in Version 1.1.0 (2013-08)

    This release note is for the Elasticsearch plug-in release 1.1.0 GA (August 2013). This release adds compatibility with Elasticsearch 0.90.2.

    Plug-in Version 1.0.0 (2013-02)

    This release note is for the Elasticsearch plug-in release 1.0.0 GA (February 2013). This is the first general availability (GA) release. It contains the following enhancements and bug fixes:

    • Now compatible with version 0.20.2 of Elasticsearch.

    • Now supports document expiration using Elasticsearch TTL.

    • Now supports XDCR conflict resolution to reduce bandwidth usage in some cases.

    • Fixed Couchbase index template to allow searching on the document metadata.

    • Fixed data corruption under high load. (CBES-11)

    • Fixed recognition of non-JSON documents. (CBES-11)

    • Improved log information when indexing stub documents.

    Plug-in Version 1.0.0 Beta (2013-02)

    This is the beta release of the Couchbase plug-in for Elasticsearch 1.0.0 Beta (February 2013).

    Older Releases

    Although no longer supported, documentation for older releases continues to be available in our docs archive.