Release Notes

KAFKAC-250: Sink You can now configure Couchbase Java SDK Settings in the connector config file. This includes KV timeout durations, Response Time Observability settings, and lots more.

KAFKAC-251: Source Upgraded DCP client from 0.33.0 to 0.34.0.

KAFKAC-211: Source A rollback in Couchbase no longer causes the connector to fail.

KAFKAC-234: The Couchbase root CA certificate can now be read directly from a PEM file; it’s no longer necessary to add it to a Java keystore first. The new couchbase.trust.certificate.path config property points to the PEM file.

KAFKAC-242: Source When the new couchbase.log.document.lifecycle config property is set to true, the connector writes detailed log entries as each document flows through the connector.

KAFKAC-245: Sink Upgraded Couchbase client from 3.1.2 to 3.1.3.

KAFKAC-246: Source Upgraded DCP client from 0.32.0 to 0.33.0. OBSERVE_SEQNO events are now logged at TRACE level instead of DEBUG.

KAFKAC-241: When secure connections are enabled, it is now possible to authenticate with Couchbase using an X.509 certificate instead of a username & password. See couchbase.client.certificate.path and couchbase.client.certificate.password for details. (This feature is added as "uncommitted", meaning it may change without notice.)

KAFKAC-238: Sink Upgraded Couchbase client from 3.0.9 to 3.1.2.

KAFKAC-239: Source Upgraded DCP client from 0.31.0 to 0.32.0. Previously, TLS hostname verification was done only for the Couchbase Java client connection; now the DCP client connection is verified as well. If for some reason you need to disable TLS hostname verification, this is now possible by setting the couchbase.enable.hostname.verification config property to false.

JDCP-183: If an invalid stream offset is detected, the connector will now fail fast instead of potentially corrupting the saved checkpoint.

JDCP-184: Resolved an issue that could cause a flow control deadlock when streaming from a subset of a bucket’s collections or scopes.

KAFKAC-152: Documentation - Added a "Monitoring" page which refers users to the Kafka Connect framework documentation for monitoring connectors via JMX.

KAFKAC-232: Source - Metrics specific to the Couchbase source connector are now exposed via JMX under the kafka.connect.couchbase domain.

KAFKAC-110: Source - Added a gauge that reports the Couchbase connection status.

KAFKAC-231: Source - Upgrade DCP client from 0.30.0 to 0.31.0

KAFKAC-227: Source - Added config property couchbase.xattrs (boolean, defaults to false). If set to true, a custom Filter or SourceHandler may inspect a document’s extended attributes by calling DocumentEvent.xattrs().

KAFKAC-226: Renamed the ZIP archive from couchbaseinc-kafka-connect-couchbase to couchbase-kafka-connect-couchbase (removed the "inc" from "couchbaseinc"). This made it easier to publish the connector on Confluent Hub.

KAFKAC-228: Upgraded the Couchbase Java SDK from 3.0.6 to 3.0.9. Bootstrap performance is improved when specifying custom ports, and the connector no longer logs spurious warnings about being unable to fetch collections manifests.

KAFKAC-229: Upgraded the DCP client from 0.29.0 to 0.30.0, adding support for XATTRs.

KAFKAC-225: Fixed a regression in version 4.0.0 that broke alternate address resolution. The connector now handles DNS SRV and alternate addresses correctly, and can connect to Couchbase Cloud or other network environments that use alternate addresses.

KAFKAC-192: Renamed the connector config properties to follow the standard Kafka naming convention ("lowercase.with.dots.between.words"). See the Migration Guide for a comprehensive list of changes, and a shell script that can help upgrade your 3.x config files to use the new property names.

KAFKAC-157: The connector is now packaged as a Confluent Hub component. Confluent Platform users can easily install the connector using the confluent-hub install command. The Quickstart Guide has been updated to show how Apache Kafka users can install the connector.

KAFKAC-167: You can now specify custom ports in the list of Couchbase seed nodes. If you specify a port, it must be the port of the Key/Value service (which defaults to 11210 for unencrypted connections).

KAFKAC-207: Sink - You can now specify an "enhanced durability" requirement with the new couchbase.durability config property. Enhanced durability requires Couchbase Server 6.5 or later.

KAFKAC-197: Sink - Added config property couchbase.topic.to.collection which maps Kafka topics to Couchbase collections. Added config property couchbase.default.collection which is used when a message’s topic is not present in the map.

KAFKAC-209: Source - The connector now evenly distributes the workload among all tasks, instead of trying to minimize the total number of Couchbase connections.

KAFKAC-177: Source - The example config files now use RawJsonSourceHandler and publish Kafka messages whose contents are the same as the Couchbase documents.

KAFKAC-212: Source - If you don’t care about the content of the Couchbase document, you can set the new couchbase.no.value config property to true, and the connector will omit the document content from Kafka messages.

KAFKAC-194: Source - A custom SourceHandler can now set headers on the Kafka record.

KAFKAC-223: Source - The connector is now more responsive to "pause" requests.

KAFKAC-220: Source - Custom Filter and SourceHandler components can now access connector configuration properties. These interfaces now have an init(Map<String, String>) method. The connector config is passed to this method when the component is created.

KAFKAC-222: The custom-extensions example project now includes a CustomFilter class that demonstrates how to read properties from the connector config. This example filter accepts or rejects documents based on a field of the document. The target field and the list of acceptable values are both configurable.

KAFKAC-196: Source - Added couchbase.scope and couchbase.collection config properties that let you stream from a specific scope or set of collections.

KAFKAC-195: Source - Modified the couchbase.topic config property to be a format string that supports ${bucket}, ${scope}, and ${collection} placeholders. This makes it easy to publish to different Kafka topics depending on the Couchbase document’s parent collection. The default value is ${bucket}.${scope}.${collection}.

KAFKAC-171: The couchbase.password config property (previously called connection.password) no longer defaults to an empty string.

KAFKAC-175: APIs deprecated in version 3.x have been removed.

Upgraded Kafka Connect API from 1.0.2 to 2.5.0.

Upgraded Couchbase client from 2.7.13 to 3.0.6.

Upgraded DCP client from 0.26.0 to 0.29.0.

KAFKAC-169: Sink - If two Kafka messages with the same key arrive in rapid succession, it’s no longer theoretically possible for them to be written to Couchbase in the wrong order.

KAFKAC-220: Source - Custom Filter and SourceHandler components can now access connector configuration properties. These interfaces now have an init(Map<String, String>) method. The connector config is passed to this method when the component is created.

KAFKAC-222: The custom-extensions example project now includes a CustomFilter class that demonstrates how to read properties from the connector config. This example filter accepts or rejects documents based on a field of the document. The target field and the list of acceptable values are both configurable.

KAFKAC-197: Sink - Added config property couchbase.topic.to.collection which maps Kafka topics to Couchbase collections. Added config property couchbase.default.collection which is used when a message’s topic is not present in the map.

KAFKAC-207: Sink - You can now specify an "enhanced durability" requirement with the new couchbase.durability config property. Enhanced durability requires Couchbase Server 6.5 or later.

KAFKAC-206: Source - Config property couchbase.connector.name.in.offsets now defaults to false again. This property doesn’t do anything useful, and should only be set to true if you previously had compat.connector_name_in_offsets set to true.

KAFKAC-177: Source - The example config files now use RawJsonSourceHandler and publish Kafka messages whose contents are the same as the Couchbase documents.

KAFKAC-209: Source - The connector now evenly distributes the workload among all tasks, instead of trying to minimize the total number of Couchbase connections.

KAFKAC-212: Source - If you don’t care about the content of the Couchbase document, you can set the new couchbase.no.value config property to true, and the connector will omit the document content from Kafka messages.

KAFKAC-205: Removed the unused couchbase.force.ipv4 config property.

KAFKAC-169: Sink - If two Kafka messages with the same key arrive in rapid succession, it’s no longer theoretically possible for them to be written to Couchbase in the wrong order.

KAFKAC-214: Sink - The Couchbase Java SDK has been updated from 3.0.5 to 3.0.6. As a result, setting couchbase.document.expiration to longer than 30 days now works correctly instead of causing immediate expiration. (This was a regression in 4.0.0-dp.1.)

KAFKAC-203: Source - The 3.x → 4.0 migration script now properly converts the old couchbase.flow_control_buffer property to the new name: couchbase.flow.control.buffer.size.

KAFKAC-204: Source - Fixed the invalid value for couchbase.bootstrap.timeout in the quickstart-couchbase-source.json example config file.

KAFKAC-182: Upgraded Kafka Connect API from 1.0.2 to 2.5.0.

KAFKAC-188: Upgraded Couchbase client from 2.7.13 to 3.0.5.

KAFKAC-189: Upgraded DCP client from 0.26.0 to 0.28.0.

KAFKAC-192: Renamed the connector config properties to follow the standard Kafka naming convention ("lowercase.with.dots.between.words"). See the Migration Guide for a comprehensive list of changes, and a shell script that can help upgrade your 3.x config files to use the new property names.

KAFKAC-196: Source: Added couchbase.scope and couchbase.collection config properties that let you stream from a specific scope or set of collections.

KAFKAC-195: Source: Modified the couchbase.topic config property to be a format string that supports ${bucket}, ${scope}, and ${collection} placeholders. This makes it easy to publish to different Kafka topics depending on the Couchbase document’s parent collection. The default value is ${bucket}.${scope}.${collection}.

KAFKAC-194: Source: A custom SourceHandler can now set headers on the Kafka record.

KAFKAC-157: The connector is now packaged as a Confluent Hub component. Confluent Platform users can easily install the connector using the confluent-hub install command. The Quickstart Guide has been updated to show how Apache Kafka users can install the connector.

KAFKAC-167: You can now specify custom ports in the list of Couchbase seed nodes. If you specify a port, it must be the port of the Key/Value service (which defaults to 11210 for unencrypted connections).

KAFKAC-171: The couchbase.password config property (previously called connection.password) no longer defaults to an empty string.

KAFKAC-173: The couchbase.connector.name.in.offsets config property (previously called compat.connector_name_in_offsets) now defaults to true.

KAFKAC-175: APIs deprecated in version 3.x have been removed.