Couchbase Distributed ACID Transactions for Java SDK Release Notes

TXNJ-339, TXNJ-370, TXNJ-388, TXNJ-400, TXNJ-404: Continuing improvements to the beta-level support for Query.

TXNJ-399: Include the version of the transactions library in the logging.

TXNJ-347: Solves a potential rare data race issue where a transaction that expires immediately after beginning the commit phase, which could in theory have its staged changes overwritten by another transaction.

TXNJ-413: After Transactions::close was called, cleanup threads could continue to be created.

TXNJ-382, TXNJ-402: If the cleanup window was set to an aggressive value (less than 10 seconds), then cleanup could both happen more quickly than the configured cleanup window and. in some cases, skip some ATRs due to backpressure.

TXNJ-405: If T2 is blocked writing to a document because it is involved in T1, but T1’s ATR has been deleted, then permit T2 to continue.

TXNJ-341, TXNJ-393, TXNJ-394, TXNJ-363, TXNJ-378, TXNJ-388: Further improvements to beta support for N1QL queries in transactions.

TXNJ-248, TXNJ-372, TXNJ-355: Logging improvements and fixes.

TXNJ-293: Very short cleanup window times are now handled correctly.

TXNJ-379: Improved efficiency of lost cleanup thread pool.

TXNJ-385: Fixed OpenTelemetry spans for cleanup.

TXNJ-391: Deferred transactions are no longer placed on cleanup queue.

TXNJ-396: The SDK’s configured JsonSerializer and Transcoder are no longer used for any internal JSON operations.

TXNJ-262: Support for OpenTelemetry and OpenTracing, at Developer Preview level, providing transparency and visibility of both transactional operation and performance. See the documentation for further details.

TXNJ-375: Reducing the amount of metadata writes required for each transaction attempt, from four to three. This should give a significant improvement in performance, particularly in latency on smaller transactions, and throughput generally.

TXNJ-358: Allow all valid combinations of multiple serial Key-Value operations on the same document.

TXNJ-362: Future-proofing the protocol to handle currently unknown transaction states.

TXNJ-275, TXNJ-371, TXNJ-373: The Java SDK’s configured JsonSerializer is no longer used for internal JSON serialization tasks.

TXNJ-301: Previously cleanup of 'lost' transactions was only happening on buckets that the application had opened. Now, it will happen on all buckets in the cluster, as intended.

TXNJ-345: Transaction B is allowed to overwrite staged changes from Transaction A in some situations. B was previously allowed to insert a document over a staged replace or remove from A, this has now been patched: B should not be able to insert a doc that already exists.

TXNJ-356: Fixed ambiguity resolution handling during ATR commit.

TXNJ-351: Lost transactions involves polling of ATR documents, and this is now correctly rate-limited so that the time taken for a cleanup run should be much closer to the configured cleanup window.

TXNJ-353: Documents with staged data may now have that data overwritten if their Active Transaction Records (ATRs) have been deleted, providing some protection against accidental user modification of the ATRs. (A reminder that ATRs should not be modified by the user, they are managed automatically by the transactions libraries.)

TXNJ-313, TXNJ-334, TXNJ-354: Further refinements for query support. This support remains at beta-level while Couchbase Server 7.0 is in beta.

TXNJ-304: Expose parameters for all TransactionQueryOptions.

TXNJ-306: Fix reactive query streaming.

TXNJ-298: Refactor sending to same query node into Java SDK.

TXNJ-303: Refactor handling of query’s transaction queue being full into SDK.

TXNJ-311: Fail deferred transactions in query mode.

TXNJ-314: Add expiry and done checks in all query stages.

TXNJ-316: Fail gracefully if query used on a pre-7.0 node.

TXNJ-321: Query in next beta release will return more detailed errors, handle them

TXNJ-323: Timeout on COMMIT should raise TransactionCommitAmbiguous.

TXNJ-324: Do not issue a query if the transaction is about to expire.

TXNJ-328: Fatal query errors should cause immediate rollback.

TXNJ-329: getOptional of non-existent doc should not fail transaction.

TXNJ-330: Insert of existing doc should raise ignorable DocExistsException.

TXNJ-309: Transaction operations issued after a rollback/commit are now rejected.

TXNJ-327: Remove dependency on Flux.retryWhen, to provide compatibility with Java client 3.1.0.

TXNJ-264: The durability level used for the transaction is now stored with the transaction’s metadata. This means that compatible cleanup clients can use the same durability level to cleanup the transaction.

TXNJ-335: Fetching a document’s ATR entry now correctly uses that ATR’s collection.

TXNJ-337: Consistently applied log redaction.

TXNJ-318: TransactionResult::attempts() is now deprecated and should no longer be used, as this information is not available if queries are used inside a transaction.

TXNJ-302: Initial beta-level support for queries inside transactions.

TXNJ-280: Initial beta-level support for custom metadata collections.

TXNJ-290: Transaction metadata documents now have binary bodies (rather than empty JSON) to improve compatibility with query.

TXNJ-270: Cleanup of a doc is always using the default collection.

TXNJ-271: Cleanup should continue as success if it finds the ATR entry has been removed.

TXNJ-276: Trying to remove clients from client record will overflow the sub-doc 16 limit.

TXNJ-282: ATR ids should be redacted in logs.

TXNJ-283: ATRs are using the collection of the first mutated document, rather than the default collection of that document’s bucket.

TXNJ-269: Cleanup does not need to fetch the ATR.

TXNJ-272: Cleanup unnecessarily fetches the document body.

TXNJ-273: MAV reads do not need to fetch the ATR, they only need to fetch an entry.

TXNJ-274: Client record can use $vbucket.HLC rather than performing a dummy update.

TXNJ-278: Increase client record safety margin from 2 to 20 seconds.

TXNJ-261: The PerTransactionConfig durability setting was not being used. It should now be passed through from the builder to the build.

TXNJ-259: In the very unlikely case of eternally repeated errors while setting ATR to ROLLED_BACK, the logic would repeat indefinitely and not check transaction expiry. The logic has been tweaked to ensure that this cannot happen.

This release requires Couchbase Server 6.6, it cannot run on 6.5.

Previous releases are not fully forward-compatible with this release. This release must not be run concurrently with a previous release.

Upgrade the cluster fully from Couchbase Server 6.5 to 6.6. This can be a live or offline upgrade.

Fully bring down the application running the previous release. Make sure no such application instances are running.

Bring up the application running the new release.

Make sure to never bring up an application running the previous release.

TXNJ-125: Our transactions stages changes to a document alongside that document. In previous releases and Couchbase Server 6.5, this required creating an empty document to store inserts. Whilst these were not visible pre-commit to many parts of the Couchbase Data Platform, they were visible in some places — including the UI. Couchbase Server 6.6 brings new functionality that allows these inserts to be fully committed, providing full Read Committed isolation for all mutations.

TXNJ-246: Write-write conflict has been tuned to backoff-and-retry much less aggressively, which will result in much less 'churn' and overhead.

TXNJ-86: T1 previously had to backoff and retry until either T2 completed, or cleanup discovered it (with default settings, that can take up to a minute). Now, T1 can proceed to overwrite the document if it finds T2 has expired or completed.

TXNJ-217: Performance improvement — T1 now only fetches the T2-specific data from T2’s Active Transaction Record (ATR), rather than the full document.

TXNJ-208: If cleanup is still unable to cleanup a transaction after two hours, it will begin logging that failure at WARN level. If there is not a reasonable explanation for cleanup continuing to fail (such as a downed node that has not been failed-over), then users are encouraged to send these logs to Couchbase support.

TXNJ-13: Performance improvement — cleanup has been tuned to remove unnecessary writes.

TXNJ-108: Cancel cleanup if a pending transaction is discovered to now be committed. This should not occur, and is simply a safety-check.

TXNJ-229: Cleanup already checked that documents are in the expected state before cleaning them up, and now also uses CAS to verify that this is still the case when it gets to the cleanup point.

TXNJ-240: Cleanup of documents is now done with durable writes.

TXNJ-232: The expiry field inside an attempt now reflects time elapsed inside the transaction.

TXNJ-227: A runtime check is now performed to ensure the minimum required java-client version is used. This will cause the application to fast-fail at the point of creating the Transactions object, rather than failing with a less clear error when creating transactions.

TXNJ-219: The TransactionConfigBuilder documentation stated that underlying Key-Value reads and writes would use kvTimeout and kvDurableTimeout from the configured environment if one is not specified. That was not being done, and now is.

TXNJ-237: A Scala bug with Class.getSimpleName meant the library could not be called from Scala.

TXNJ-250: MAV reads should return post-transaction content on COMPLETED, not just COMMITTED.

TXNJ-249: The documentation advises applications to not catch exceptions inside the lambda (and if they have to, to propagate them). This advice remains, but if it is not followed, now subsequent operations (including commit) will fail. This means we no longer rely on the application propagating errors to ensure that transactions work correctly.

TXNJ-231: Rollback previously assumed that documents had not changed between them being staged and rolled back. That should be the case, but now CAS is used to confirm it.

TXNJ-184: If doc is found to be removed while committing it, create it.

TXNJ-187: Errors raised while setting ATR state to Committed are now regarded as pre-commit errors, e.g. they can cause the transaction to fail or retry.

TXNJ-188, TXNJ-192: The TransactionResult.attempts() field now always correctly reflects all attempts made.

TXNJ-196: Pre-commit expiry now always triggers rollback.

TXNJ-197: Expiry during app-rollback now enters expiry-overtime-mode and has one more attempt at completing rollback.

TXNJ-202, TXNJ-200: FAIL_ATR_FULL now handled consistently throughout protocol.

TXNJ-210: If anything goes wrong during app-rollback, return success. Rollback will be completed by the asynchronous cleanup process, and unrolled-back metadata is handled by the protocol anyway.

TXNJ-211: If rollback fails, the original error that caused the rollback is the one raised as the getCause() of any final exception raised to the application.

TXNJ-218: Inserting the same document in the same transaction is now caught as an application bug, and will fail the transaction.

TXNJ-189: Adds a transactionId field throughout the metadata, to assist with debugging.

TXNJ-190: Write elided stacktraces in the logs to improve readability.

TXNJ-191, TXNJ-199: Performance improvement so that transactions are only rolled back and/or cleaned up if they reached the point of creating an ATR entry (e.g. if there is anything to cleanup).

TXNJ-204: All context methods, such as ctx.insert and ctx.replace, will now consistently only raise ErrorWrapper exceptions. Applications should not be catching any exceptions from these methods so this is not expected to be impacting.

TXNJ-215: Adjust exponential backoff retry parameter from 250msecs down to 100msecs.

TXNJ-119: Calling commit multiple times leads to an error mentioning "rollback"; changed to separate error messages for commit and rollback.

TXNJ-126: Expire a transaction immediately if it expires while deferred.

TXNJ-149: Improve ambiguous error handling during commit.

TXNJ-163, TXNJ-156: Ensure that java-client errors are correctly handled after its changes to error handling.

TXNJ-165: Fix reactor.core.Exceptions$OverflowException seen during cleanup. Would only affect users setting a cleanup window of 1 second or less.

TXNJ-173: Register for changes to bucket configurations, so if the application opens a bucket after creating Transactions object, background cleanup will be started on those new buckets.

TXNJ-69: Provide some future proofing for client-record. Implementation detail, allows for future extensibility of the background cleanup process.

TXNJ-111, TXNJ-110: Sends new events to the java-client event-bus with details from the background cleanup process.

TXNJ-120: Once Transactions object is closed, prevent futher operations on it as a sanity check.

TXNJ-151: Add Java client metadata to help track failures. Implementation detail, adds some additional info to exceptions to help debugging.

TXNJ-152: When transaction expires, attempt to roll it back (or complete commit). Completes the work in TXNJ-124 (in 1.0.0-beta.3), to cover all cases.

TXNJ-155: Remove OpenTracing dependency. An implementation detail as it has already been removed from the API, this completes the work and removes the dependency on the jar. Once OpenTelemetry (OpenTracing’s direct replacement) is stable, support for it will be added.

TXNJ-162: Performance improvements related to String parsing. Internal benchmarking sees a huge reduction in memory allocations and GC pressure with this change.

TXNJ-167: Rebuild on java-client 3.0.0 GA.

TXNJ-170: Perform ATR cleanup in serial, which prevents errors being sent to Hooks.onErrorDropped (and by default stdout) when shutting down transactions. Along with java-client fix JVMCBC-781, users should no longer see Hooks.onErrorDropped errors.

TXNJ-171: Rename MAJORITY_AND_PERSIST_ON_MASTER to MAJORITY_AND_PERSIST_ON_ACTIVE, in accordance with new naming conventions.

TXNJ-169: Simplify cleanup window logic to remove divide-by-2; previously, specifying TransactionConfigBuilder.cleanupWindow(120 seconds) would lead to ATRs getting checked every 60 seconds (in the general case): now to reduce confusion they will be checked every 120 seconds.

TXNJ-164: Remove TransactionConfigBuilder.logDirectlyCleanup. As all cleanup trace is now sent directly to java-client’s event-bus, it can be configured from there and this parameter is useless.

TXNJ-127: Fixes an ArrayIndexOutOfBoundsException error when using the reactive API to perform concurrency inside a txn.

TXNJ-145: ATR names are changed, to hash to their expected vbucket. It is recommended to flush (remove all documents from) the bucket to remove ATR documents from previous releases.

TXNJ-136: Fixes an erroneous IllegalDocumentState alert in edge-cases involving the DurabilityAmbiguous error.

TXNJ-124: An aborting transaction will now attempt rollback before completing raising TransactionExpired failure, rather than leaving cleanup solely to cleanup process.

TXNJ-131: Minor logging improvement.

JVMCBC-728: This fix in the underlying Couchbase java-client prevented conflict detection from working correctly in all situations with durability enabled (which it is, by default). All users are strongly recommended to upgrade as soon as possible, for this fix.

TXNJ-121: Align with the Couchbase Java client, by renaming the get method to getOptional, getOrError to get, and TransactionJsonDocument to TransactionGetResult.

TXNJ-40: Retry operations on many transient errors, rather than retrying the full transaction.

TXNJ-85: (Experimental) Supported deferred commit for transactions. Please see the documentation for details. This feature should be regarded as alpha, as its API and functionality may change in the future; please feel free to try it out and provide feedback.

TXNJ-112: (Experimental) Allow the number of ATRs to be configured. This can potentially be used to improve transactions throughput, but is currently experimental and should be used only with Couchbase’s guidance.

TXNJ-107: Make txn metadata documents more consistently named by prefacing them with "_txn:"

TXNJ-41: Add fields required for backup

TXNJ-52, TXNJ-53, TXNJ-56, TXNJ-71, TXNJ-72, TXNJ-75, TXNJ-76, TXNJ-83: More readable and useful logging

TXNJ-47: Improved handling for expiry — the transaction will now try to enter Aborted state

TXNJ-50: Intermittent reactive asserts about blocking on a thread from the "parallel" scheduler

TXNJ-55: Retry transient errors while rolling back ATR entry, rather than retrying transaction

TXNJ-57: Add log redaction for document keys

TXNJ-59: Fix an issue with the lost cleanup thread aborting early

TXNJ-64: Commit documents in the order they were staged

TXNJ-79, TXNJ-81: Improved handling of transient server errors while trying to create ATR entries

TXNJ-90: Improved handling of conflicts when multiple applications start at once and try to create the client record

TXNJ-96: Improved handling of transient errors removing entries from the client record

TXNJ-92: OpenTracing removed from API — will be re-added when OpenTelemetry is ready