Please use the form below to provide your feedback. Because your feedback is valuable to us, the information you submit in this form is recorded in our issue tracking system (JIRA), which is publicly available. You can track the status of your feedback using the ticket number displayed in the dialog once you submit the form.

A newer version of this documentation is available.

View Latest

Source Configuration Options

    +

    Reference of the source connector options.

    Database

    connection.cluster_address

    Couchbase Cluster addresses to listen (use comma to specify several).

    • Type: list

    • Importance: high

    couchbase.network

    The network selection strategy for connecting to a Couchbase Server cluster that advertises alternate addresses. A Couchbase node running inside a container environment (like Docker or Kubernetes) might be configured to advertise both its address within the container environment (known as its "default" address) as well as an "external" address for use by clients connecting from outside the environment. Setting the couchbase.network config property to default or external forces the selection of the respective addresses. Setting the value to auto tells the connector to select whichever network contains the addresses specified in the connection.cluster_address config property; this heuristic works well in most environments and is the recommended mode.

    • Possible values:

      • auto

      • default

      • external

    • Since: 3.4.5 (uncommitted)

    • Type: string

    • Importance: low

    • Default: "auto"

    connection.bucket

    Couchbase bucket name.

    • Type: string

    • Importance: high

    connection.username

    Couchbase username for Couchbase Server 5.0 and later.

    • Type: string

    • Importance: high

    • Default: the bucket name specified by connection.bucket

    connection.password

    Couchbase password. For Couchbase Server versions prior to 5.0 this is the bucket password. For versions 5.0 and later, this is the password associated with connection.username.

    Two options are available for protecting the password. If you set the environment variable KAFKA_COUCHBASE_PASSWORD, its value will override the password in the connector configuration. Alternatively, you can take advantage of the Kafka Connect framework’s support for externalized secrets.

    • Environment Variable: KAFKA_COUCHBASE_PASSWORD

    • Type: password

    • Importance: low

    • Default: ""

    connection.timeout.ms

    Connection timeout in milliseconds.

    • Type: long

    • Importance: low

    connection.ssl.enabled

    Use SSL to connect to Couchbase. This feature only available in Couchbase Enterprise.

    • Type: boolean

    • Importance: low

    • Default: false

    • Dependents: connection.ssl.keystore.location, connection.ssl.keystore.password

    connection.ssl.keystore.location

    The location of the key store file.

    • Type: string

    • Importance: low

    connection.ssl.keystore.password

    The password of the private key in the key store file.

    Two options are available for protecting the password. If you set the environment variable KAFKA_COUCHBASE_SSL_KEYSTORE_PASSWORD, its value will override the password in the connector configuration. Alternatively, you can take advantage of the Kafka Connect framework’s support for externalized secrets.

    • Environment Variable: KAFKA_COUCHBASE_SSL_KEYSTORE_PASSWORD

    • Type: password

    • Importance: low

    • Default: ""

    Connector

    topic.name

    Name of the Kafka topic to publish data to.

    • Type: string

    • Importance: high

    use_snapshots (Deprecated)

    If true, it will only commit into Kafka when full snapshot from Couchbase received.

    • Deprecated. Prior to version 3.4.6, enabling this feature can cause OutOfMemoryError during initial backfill, since snapshot contents are buffered in memory and the first snapshot contains the entire contents of the database. In version 3.4.6 and later, enabling this feature has no effect. This configuration property will be removed entirely in a future release.

    • Type: boolean

    • Importance: low

    • Default: false

    dcp.message.converter.class

    The class name of the message converter to use.

    • Type: string

    • Importance: low

    • Default: "com.couchbase.connect.kafka.handler.source.DefaultSchemaSourceHandler"

    event.filter.class

    The class name of the event filter to use.

    • Type: string

    • Importance: low

    • Default: "com.couchbase.connect.kafka.filter.AllPassFilter"

    couchbase.stream_from

    The point in Couchbase history to start streaming from.

    • Possible values:

      • SAVED_OFFSET_OR_BEGINNING - Restart from saved state, or if none, restart from oldest available mutation in Couchbase

      • SAVED_OFFSET_OR_NOW - Restart from saved state, or if none, restart from current Couchbase state

      • BEGINNING - Restart from oldest available mutation in Couchbase (ignore any potential saved state)

      • NOW - Restart from current Couchbase state (ignore any potential saved state)

    • Since: 3.2.2

    • Type: string

    • Importance: low

    • Default: "SAVED_OFFSET_OR_BEGINNING"

    couchbase.log_redaction

    Optionally tag sensitive values in the log output for later redaction.

    • Possible values:

      • NONE - No redaction is performed.

      • PARTIAL - Only user data is redacted, system and metadata are not.

      • FULL - User, System and Metadata are all redacted.

    • Since: 3.2.3

    • Type: string

    • Importance: low

    • Default: "NONE"

    couchbase.compression

    To reduce bandwidth usage, Couchbase Server 5.5 and later can send documents to the connector in compressed form. (Messages are always published to the Kafka topic in uncompressed form, regardless of this setting.) If the requested mode is not supported by your version of Couchbase Server, compression will be disabled.

    • Possible values:

      • ENABLED - (default) Couchbase Server decides whether to use compression on a per-document basis, depending on whether the compressed form of the document is readily available. Select this mode to prioritize Couchbase Server performance and reduced bandwidth usage (recommended). Requires Couchbase Server 5.5 or later.

      • DISABLED - No compression. Select this mode to prioritize reduced CPU load for the Kafka connector.

      • FORCED - Compression is used for every document, unless compressed size is greater than uncompressed size. Select this mode to prioritize bandwidth usage reduction above all else. Requires Couchbase Server 5.5 or later.

    • Since: 3.3.0

    • Type: string

    • Importance: low

    • Default: "ENABLED"

    couchbase.forceIPv4

    In a network environment that supports both IPv4 and IPv6, setting this property to true will force the use of IPv4 when resolving Couchbase Server hostnames.

    • Since: 3.3.0

    • Type: boolean

    • Importance: low

    • Default: false

    couchbase.persistence_polling_interval

    The frequency at which the connector will poll Couchbase Server to determine which database changes are unlikely to be rolled back. A value of 0 disables polling and causes changes to be published to the Kafka topic as soon as they are received.

    If polling is disabled, when rollbacks occur you are more likely to end up with events in the Kafka topic that do not match the actual database state, because they are from an "alternate timeline" in Couchbase Server’s history.

    The longer the polling interval, the larger the flow control buffer required in order to maintain steady throughput, since events are buffered until persistence is confirmed.

    If present, the value must be 0 or an integer followed by a time unit: (ms = milliseconds, s = seconds)

    If the source is an ephemeral bucket (which never persists documents) this value must be set to 0 to disable the persistence check.

    • Since: 3.4.0

    • Type: string

    • Importance: low

    • Default: "100ms"

    couchbase.flow_control_buffer

    The amount of heap space to reserve for the flow control buffer. This is the amount of data Couchbase Server will push to the connector before awaiting acknowledgement that the data has been published to the Kafka topic.

    If present, the value must be an integer followed by a storage size unit: (b = bytes, k = kilobytes, m = megabytes, g = gigabytes)

    This value must be smaller than the connect worker’s heap size. To allocate more memory to the connect worker process, set the KAFKA_HEAP_OPTS environment variable before starting the worker. For example: export KAFKA_HEAP_OPTS=-Xmx1024M

    • Since: 3.4.0

    • Type: string

    • Importance: low

    • Default: "128m"

    Parent topic: Kafka Connector

    Previous topic: Quickstart

    Next topic: Sink Configuration Options