Managing Advanced Settings
XDCR advanced settings can be used to configure replications.
HTTP methods and URI
POST /settings/replications/ POST /settings/replications/<settings_URI> GET /settings/replications/ GET /settings/replications/<settings_URI>
Description
Used with the POST
method, the URIs respectively change global settings for all replications; and for a specific replication, which is referenced by its settings_URI
.
The settings_URI
comprises the id for the replication, and can be retrieved by means of the GET /pools/default/tasks
method and URI: see Getting Cluster Tasks.
The global settings are the default values used if settings are not specified during the creation of a particular replication. If settings are specified for a particular replication, the specified settings overwrite the global settings.
If the global settings are themselves changed, existing replications are not affected: only replications created after the change made to the global settings receive the updated global settings.
Used with the GET
method, the URIs respectively retrieve global settings for all replications; and for a specific replication, which is referenced by its settings_URI
.
Each command requires the Full Admin, Cluster Admin, ReadOnly Admin, External User Security Admin, Local User Security Admin, Backup Full Admin, or XDCR Admin role.
Curl Syntax
curl -u <username>:<password> -X POST http://<ip-address-or-domain-name>:8091/settings/replications [ -d <xdcr-advanced-setting>=<value> ]* curl -u <username>:<password> -X POST \ http://<ip-address-or-domain-name>:8091/settings/replications/<settings_URI> [ -d <xdcr-advanced-setting>=<value> ]* curl -u <username>:<password> -X GET \ http://<ip-address-or-domain-name>:8091/settings/replications curl -u <username>:<password> -X GET \ http://<ip-address-or-domain-name>:8091/settings/replications/<settings_URI>
Each instance of the POST method allows one or more instances of the xdcr-advanced-setting
flag to be specified, with an appropriate value
.
All flags are listed below, in the section List of Advanced Settings.
Responses
If successful, GET
and POST
commands return 200 OK
.
Commands for establishing or retrieving either global settings or the settings for a specific replication return an object containing the settings.
Failure to authenticate returns 401 Unauthorized
.
A URI featuring an incorrect hostname or port returns a failure to connect
message.
An incorrect settings_URI
returns 400 Bad Request
, and an object such as {"errors":{"_":"requested resource not found"}}
.
An attempt to set a global-only setting for a specific replication returns 400 Bad Request
, and an object indicating which submitted parameters were incorrect.
For example, the attempted use of POST /settings/replications/<settings_URI>
to establish values for goGC
and goMaxProcs
returns the following:
{ "gogc": "Invalid settings key", "gomaxprocs": "Invalid settings key" }
A replication-specific parameter (such as pauseRequested
) is ignored, if incorrectly specified in an attempt to modify global settings.
Examples
The following examples show how settings can be established and retrieved.
Get Settings for All Replications
The following example retrieves settings for all replications. Note that the output is piped to the jq command, to facilitate readability:
curl -u Administrator:password -X GET http://localhost:8091/settings/replications | jq '.'
If successful, the call returns an object similar to the following:
{ "checkpointInterval": 600, "ckptSvcCacheEnabled": true, "compressionType": "Auto", "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "goGC": 100, "goMaxProcs": 4, "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, "retryOnRemoteAuthErr": true, "retryOnRemoteAuthErrMaxWaitSec": 3600, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, "workerBatchSize": 500 }
Get Settings for a Specific Replication
To reference a specific replication, the replication’s settings_URI
must first be retrieved.
Use the following call, which is explained in detail in Getting Cluster Tasks.
curl -v -X GET http://localhost:8091/pools/default/tasks -u Administrator:password | jq '.'
If successful, the call returns an array of ongoing cluster tasks, similar to the following:
[ { "statusId": "20caaa733e53b3ab3a370ce89de21cc0", "type": "rebalance", "subtype": "rebalance", "status": "notRunning", "statusIsStale": false, "masterRequestTimedOut": false, "lastReportURI": "/logs/rebalanceReport?reportID=57461c2db70751343a307fb91b8b242a" }, { "cancelURI": "/controller/cancelXDCR/0ad6c74d0f9fee5e447688933df6b281%2Ftravel-sample%2Fts2", "settingsURI": "/settings/replications/0ad6c74d0f9fee5e447688933df6b281%2Ftravel-sample%2Fts2", "status": "running", "replicationType": "xmem", "continuous": true, "filterBypassExpiry": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", "id": "0ad6c74d0f9fee5e447688933df6b281/travel-sample/ts2", "pauseRequested": false, "source": "travel-sample", "target": "/remoteClusters/0ad6c74d0f9fee5e447688933df6b281/buckets/ts2", "type": "xdcr", "recommendedRefreshPeriod": 10, "changesLeft": 0, "docsChecked": 84792, "docsWritten": 0, "maxVBReps": null, "errors": [ ] }, { "cancelURI": "/controller/cancelXDCR/2b5dcd1b0101a9d52f31a802d8c4231e%2Ftravel-sample%2Fts", "settingsURI": "/settings/replications/2b5dcd1b0101a9d52f31a802d8c4231e%2Ftravel-sample%2Fts", "status": "running", "replicationType": "xmem", "continuous": true, "filterBypassExpiry": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", "id": "2b5dcd1b0101a9d52f31a802d8c4231e/travel-sample/ts", "pauseRequested": false, "source": "travel-sample", "target": "/remoteClusters/2b5dcd1b0101a9d52f31a802d8c4231e/buckets/ts", "type": "xdcr", "recommendedRefreshPeriod": 10, "changesLeft": 0, "docsChecked": 84792, "docsWritten": 0, "maxVBReps": null, "errors": [ ] } ]
A settings_URI
is provided for each of the ongoing replications, within the second member of the array.
The settings_URI
can be used to retrieve information on the corresponding replication.
For example, enter the following:
curl -v -X GET http://localhost:8091/settings/replications/2b5dcd1b0101a9d52f31a802d8c4231e%2Ftravel-sample%2Fts \ -u Administrator:password | jq '.'
If the call is successful, an object containing the settings for the specified replication is returned:
{ "checkpointInterval": 600, "ckptSvcCacheEnabled": true, "colMappingRules": {}, "collectionsExplicitMapping": false, "collectionsMigrationMode": false, "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, "retryOnRemoteAuthErr": true, "retryOnRemoteAuthErrMaxWaitSec": 3600, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, "type": "xmem", "workerBatchSize": 500 }
Change a Setting for a Specific Replication
The following example modifies the value of checkpointInterval
, for a specific replication:
curl -v X POST \ http://localhost:8091/settings/replications/2b5dcd1b0101a9d52f31a802d8c4231e%2Ftravel-sample%2Fts -d checkpointInterval=700 \ -u Administrator:password | jq '.'
If successful, the call returns an object containing all current settings for the replication, including what has been changed:
{ "checkpointInterval": 700, "ckptSvcCacheEnabled": true, "colMappingRules": {}, "collectionsExplicitMapping": false, "collectionsMigrationMode": false, "collectionsMirroringMode": false, "collectionsOSOMode": true, "compressionType": "Auto", "desiredLatency": 50, "docBatchSizeKb": 2048, "failureRestartInterval": 10, "filterBypassExpiry": false, "filterBypassUncommittedTxn": false, "filterDeletion": false, "filterExpiration": false, "filterExpression": "", "hlvPruningWindowSec": 259200, "jsFunctionTimeoutMs": 20000, "logLevel": "Info", "mergeFunctionMapping": {}, "networkUsageLimit": 0, "optimisticReplicationThreshold": 256, "pauseRequested": false, "preReplicateVBMasterCheck": true, "priority": "High", "replicateCkptIntervalMin": 20, "retryOnRemoteAuthErr": true, "retryOnRemoteAuthErrMaxWaitSec": 3600, "sourceNozzlePerNode": 2, "statsInterval": 1000, "targetNozzlePerNode": 2, "type": "xmem", "workerBatchSize": 500 }
List of Advanced Settings
The advanced settings for XDCR are as follows. The subset of advanced settings that can be configured by means of Couchbase Web Console is provided at XDCR Advanced Settings.
Parameter | Value | Description |
---|---|---|
|
Integer (60 to 14400) |
Default: 600 The interval for checkpointing in seconds. This setting can be established and retrieved either for an individual replication or globally. |
|
JSON Object |
Collection-related rules according to which explicit mapping or migration is defined for the replication. See Rules for Explicit Mappings and Rules for Migration. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
Boolean (true or false) |
Whether the replication uses explicit mapping: see Rules for Explicit Mappings. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
Boolean (true or false) |
Whether the replication uses migration mode: see Migration. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
Boolean (true or false) |
Whether the replication optimizes performance; by streaming, from a source bucket, mutations that could be out of order, in terms of sequence-number. Default is true. |
|
String |
Default: This setting can be established and retrieved either for an individual replication or globally. |
|
Integer |
Specifies the amount of time in milliseconds within which a high-priority replication’s currently remaining changes should be reduced in number to zero, by ongoing transmission of the changes from source to target bucket.
If Couchbase Server estimates that the time required is greater than that specified by This setting applies only to high-priority replications, or to medium-priority replications that achieve high-priority status (see XDCR Priority). The default value is 50. The lower the value, the higher (potentially) the transmission rate, and the greater the load on the target cluster. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (10 to 10000) |
Default: 2048. The size of a batch in kilobytes. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (1 to 300) |
Default: 10. The number of seconds to wait after a failure before restarting replication. This setting can be established and retrieved either for an individual replication or globally. |
|
Boolean (true or false) |
Default: false. Whether a document’s TTL should be replicated with the document or not. A value of true means that the TTL is removed from the document. This setting can be established and retrieved either for an individual replication or globally. |
|
Boolean (true or false) |
Default: false. When false, the setting specifies that if a document contains uncommitted transactional metadata (xattributes), XDCR removes them, before replicating the document. The original document in the source bucket is unaffected. When true, the setting specifies that if the document contains uncommitted transactional metadata, XDCR does not replicate the document. |
|
Boolean (true or false) |
Default: false. Whether mutations corresponding to the deletion of documents on the source cluster should be either filtered out of the replication to the target cluster, or allowed to remain in. A value of true means that the mutation is filtered out, ensuring that it is not replicated to the target cluster; while a value of false means that the mutation is not filtered out, ensuring that it is replicated to the target cluster. This setting can be established and retrieved either for an individual replication or globally. |
|
Boolean (true or false) |
Default: false. Whether mutations corresponding to the expiration of documents on the source cluster should be either filtered out of the replication to the target cluster, or allowed to remain in. A value of true means that the mutation is filtered out, ensuring that it is not replicated to the target cluster; while a value of false (the default) means that the mutation is not filtered out, ensuring that it is replicated to the target cluster. This setting can be established and retrieved either for an individual replication or globally. |
|
String (a filter expression) |
Default: null. A filter expression to be matched against the ids, field-names, values, and extended attributes of documents in the source bucket. Each document that produces a successful match is replicated; other documents are not replicated. For supported expressions, see XDCR Advanced Filtering Reference. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
Integer (0 to 100) or String ("off") |
Default: 100. The initial garbage collection target percentage for the replication. A garbage collection is triggered when the ratio of freshly allocated data to live data remaining after the previous collection reaches this percentage. A value of "off" disables the garbage collector entirely. This setting is only returned when global settings are retrieved; and can only be set when global settings are set. It cannot be set when an individual replication is created, or when its settings are modified. |
|
Integer |
Default: 4. The maximum number of threads used per node, to support XDCR. A greater number of threads increases parallelism, and may thereby produce enhanced XDCR performance. This setting is only returned when global settings are retrieved; and can only be set when global settings are set. It cannot be set when an individual replication is created, or when its settings are modified. |
|
Integer |
Used in custom conflict-resolution. The integer specifies the duration, in seconds, of the time-window that starts when a remote cluster has modified the document that is currently being replicated. The remote cluster’s update is recorded in a Hybrid Logical Vector (HLV). If the specified time-window expires without further modification being made by the remote cluster, information about the remote cluster’s update is removed from the HLV. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer |
Used in custom conflict-resolution. Specifies the number of milliseconds that must elapse before the timeout of a JavaScript conflict-resolution function. This setting can be established and retrieved either for an individual replication or globally. |
|
String |
Default: This setting can be established and retrieved either for an individual replication or globally. |
|
JSON Object |
Each of the key-value pairs in the JSON object should map a collection-specifier (in the form |
|
Integer |
Default: 0. Specifies the upper limit for network usage during replication, for the entire cluster, in MB per second. The default is 0, meaning no limit is applied. For information, see the Table of XDCR Advanced Settings. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (0 to (20*1024*1024)) |
Default: 256. Documents with sizes less than this threshold (in bytes) will be replicated optimistically. This setting can be established and retrieved either for an individual replication or globally. |
|
Boolean (true or false) |
Default: false. Indicates whether the replication has been issued a pause request. This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
High, Medium, or Low |
Default: High. Resource-allocation for the replication. For information, see XDCR Priority. This setting can be established and retrieved either for an individual replication or globally. |
|
Boolean (true or false) |
Whether to retry connection for a replication that has failed due to a problem in authenticating with the target cluster. The default is true. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer |
The maximum number of seconds to wait before retrying a connection that failed due to a problem in authenticating with the target cluster.
This only takes effect when This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (1 to 100) |
The number of nozzles that can be used for this replication per source cluster node.
This together with This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (200 to 600000) |
Default: 1000. The interval (in milliseconds) for statistics updates. This setting can be established and retrieved either for an individual replication or globally. |
|
Integer (1 to 100) |
The number of outgoing nozzles per target node.
This together with This setting can be established and retrieved either for an individual replication or globally. |
|
String |
The replication type, which must be This setting can only be established for and retrieved from an individual replication: it cannot be established or retrieved as part of global settings. |
|
Integer (500 to 10000) |
Default: 500. The number of mutations in a batch. This setting can be established and retrieved either for an individual replication or globally. |
See Also
The REST call for creating a replication and specifying parameters is described in Creating a Replication. An overview of XDCR is provided in Cross Data Center Replication (XDCR). The subset of advanced settings that can be configured by means of Couchbase Web Console is provided at XDCR Advanced Settings.