Compression
In response to increasing volumes of data being sent over the wire, Couchbase Data Platform provides data compression between the SDK and Couchbase Server.
Overview
Documents may already be stored compressed with Snappy on the server. New documents may be passed from client applications to the server in compressed form, saving around 40% in bandwidth (depending on the document content and size), and also transmission time. These operations take place automatically, after the SDK negotiates the capability with the server, and do not require any changes on the client side.
For SDKs with Snappy compression enabled, documents will be automatically compressed if the Couchbase Server is not set to Off for Compression see see below.
Instructions to disable compression can be found at the bottom of the page.
Limits
The document must be below 20MiB in size in both compressed and uncompressed form. Compression is only available in the Enterprise Edition of Couchbase Data Platform.
|
This size limit is enforced by Couchbase Server (6.5 and earlier); in practice it will affect very few users, as most JSON documents are considerably smaller. A compressed doument of just under 20MB, which is greater than 20,971,520 bytes (20 MiB) when uncompressed, will be rejected by the server as follows:
Therefore, where necessary, enforce document size limits in your application on uncompressed documents. |
Operating Modes
The three modes in which compression can be used, "off", "passive" and "active", are configured per bucket in the configuration settings on the cluster.
| OFF | PASSIVE | ACTIVE | |
|---|---|---|---|
Negotiating SNAPPY |
ignore |
acknowledge |
acknowledge |
Sending compressed data when SNAPPY feature enabled |
- |
accept |
accept |
Sending compressed data when SNAPPY feature disabled |
reject as invalid |
reject as invalid |
reject as invalid |
Receiving data which were previously compressed on the server |
server inflates and sends plain data |
server sends compressed data |
server sends compressed data |
Receiving data which were not previously compressed on the server |
server sends plain data |
server sends plain data |
server might send compressed data (the compression is done in the background on the server) |
Acceptance Guarantee
The server may change compression settings for the bucket at any time, but it is guaranteed that once the socket negotiates compression, the server will never reject compressed data, even if the bucket setting has been changed.
Minimum Size
While the tiniest of documents will not be reduced in size by compressing, there is another category of slightly larger documents in whose case the time overhead of compressing and decompressing outweighs the slight advantage of marginally reduced transmission time from client to server or back.
To safeguard against the case of several thousand documents stealing CPU time to barely discernable advantage, a threshold for minimum doument size to compress is set in the SDK, with a sensible default value - that value can be seen for your chosen SDK in its API documentation (32 bytes), and you can override this to disable compression of smaller ones:
ClusterEnvironment env = ClusterEnvironment
.builder()
// start compressing at 1024 bytes
.compressionConfig(CompressionConfig.minSize(1024))
.build();
// #end::compression_1[];
}
public static void main(String[] args) throws Exception {
CompressionExample obj = new CompressionExample();
obj.init();
obj.compression_1();
System.out.println("Done.");
}
}The CompressionConfig also allows you to disable compression completely or tune the ratio of when a compressed document (compared to the size of the uncompressed) should be sent over the wire.