Please use the form below to provide your feedback. Because your feedback is valuable to us, the information you submit in this form is recorded in our issue tracking system (JIRA), which is publicly available. You can track the status of your feedback using the ticket number displayed in the dialog once you submit the form.

Release Notes

    +
    Release notes for the Spark Connector.

    Couchbase Spark Connector 3.2.1 GA (May 2022)

    Version 3.2.1 is the second version to support Spark 3.2.x and is built on top of the Couchbase Scala SDK 1.3.x.

    Features

    • Updated the underlying SDK to 1.3.0, which (along with general enhancements and fixes) bundles the Capella certificate, so it doesn’t have to be added manually.

    • SPARKC-133: Added support for spark structured streaming. For more details, see the documentation section for streaming.

    Couchbase Spark Connector 3.2.0 GA (January 2022)

    Version 3.1.0 is the first version to support Spark 3.2.x and is built on top of the Couchbase Scala SDK 1.2.x.

    Features

    • Support for Apache Spark 3.2.x

    • SPARKC-146: Added support for scope-level Query and Analytics RDDs (through Keyspace)

    • SPARKC-148: Added support for aggregate pushdown for Query DataFrames.

    • SPARKC-149: Added support for aggregate pushdown for Analytics DataFrames.

    Bug fixes and stability

    • SPARKC-143: Allow passing timeouts to KV, Query and Analytics DataFrame as option.

    • SPARKC-151: Fix ScanConsistency not being applied for Analytics and Query DataFrames.

    • SPARKC-144, SPARKC-145: Move Jackson JSON handling into the connector. This solves issues in the databricks notebook environment.

    • SPARKC-153: Properly escape fields for Analytics and Query DataFrames.

    Couchbase Spark Connector 3.1.0 GA (September 2021)

    Version 3.1.0 is the first version to support Spark 3.1.x and is built on top of the Couchbase Scala SDK 1.1.x.

    This release contains identical features to the Spark 3.0.0 connector, the only difference being compiled against Spark 3.1.x. Please refer to the 3.0 migration guide for changes and new features over 2.4.x.

    Highlights

    • Support for Apache Spark 3.1.x

    Couchbase Spark Connector 3.0.0 GA (September 2021)

    Version 3.0.0 is the first version to support Spark 3.0.x and is built on top of the Couchbase Scala SDK 1.1.x.

    Please note that this release does not have separate release notes, rather refer to the migration guide page for more information.

    Highlights

    • Support for Apache Spark 3.0.x

    • Built on top of the new Scala SDK 1.2.x

    • Support for Couchbase Server 7.0 and later

    Couchbase Spark Connector 2.4.1 GA (November 2020)

    Version 2.4.1 brings support for Couchbase Cloud as well as:

    Features

    • SPARKC-110: Allow passthrough of timestampFormat and dateFormat for inferring schema

    • Updated the Java SDK to 2.7.18

    • Allows enabling DNS SRV via a configuration property

    Bug fixes and stability

    • SPARKC-104: Stopping and restarting of Spark-Couchbase-Streaming-job throws ClassCastException

    • Properly propagate the expiry into the DefaultSource

    Couchbase Spark Connector 2.4.0 GA (July 2019)

    Version 2.4.0 brings support for Spark 2.4.0, and is compiled exclusively with Scala 2.12.

    Couchbase Spark Connector 2.3.0 GA (May 2019)

    Version 2.3.0 brings support for Spark 2.3.0 along with:

    Features

    • SPARKC-93: Support for Apache Spark 2.3.0

    • SPARKC-89: Support Analytics

    • SPARKC-88: Allow N1QL queries to run on Spark node(s) co-located with query service

    • SPARKC-96: Provide more fault-tolerant batch mutations. saveToCouchbase now takes a maxConcurrent parameter, giving the application control over the size of batches that will be written, from each executor.

    Bug fixes and stability

    • SPARCK-85: Raise N1QL errors as exceptions rather than logging them.

    • SPARKC-82: When running a N1QLQuery, if multiple buckets have been specified, then the bucket to use must now be explicitly chosen. E.g. sc.couchbaseQuery(query, bucketName = "default"). This is safer than choosing an arbitrary bucket.

    • SPARKC-95: Fix to get streaming source working with Spark 2.3

    Couchbase Spark Connector 2.2.0 GA (September 2017)

    Version 2.2.0 is the first stable release of the 2.2.x series. It brings support for Spark 2.2 and the following enhancements and bugfixes:

    Spark Core

    • Support for Apache Spark 2.2.0

    • SPARKC-80: Support for Couchbase Server 5.0 and Role-Based Access Control

    • SPARKC-77: Global and per-operation timeout configuration is now possible

    • SPARKC-44: Support for Subdocument Mutations has been added.

    • SPARKC-79: Support for easier SSL/TLS configuration via spark config.

    Spark SQL

    • SPARKC-77: per-operation timeout configuration is now possible

    Spark Streaming

    No changes for Spark Streaming have been made in this release.

    Older Releases

    Although no longer supported, documentation for older releases continues to be available in our docs archive.