Manage Backup and Restore

The data on a Couchbase-Server cluster can be backed up, restored, and archived by means of either of the following:

The technology that underlies the Backup Service includes that provided by cbbackupmgr: note, however, that cbbackupmgr and the Backup Service differ, in terms of the features they offer (as described below, in The Backup Service and cbbackupmgr).

Detailed information on cbbackupmgr is provided in cbbackupmgr. An overview of the Backup Service is provided in Backup Service.

For backup, restore, and other related tasks to be scheduled and performed, the Backup Service must be running on an assigned node. The service (as is the case with all other Couchbase services) can be assigned either when a node is initially provisioned as a one-node cluster (as described in Create a Cluster), or when a node is added to an existing cluster (as described in Add a Node and Rebalance). Provided that at least one node runs the Backup Service, data for the entire cluster can be backed up, restored, and archived. Locations to be used for saving data must be accessible to all cluster-nodes that are running the Backup Service. Note also that Couchbase Server must have read and write access to the location. On Linux, therefore, for a filesystem location, use the chgrp command to set the group ID of the folder to couchbase; unless a non-root installation has been performed, in which case set the group ID either to the username of the current user, or to a group of which the current user is a member — see Non-Root Install and Upgrade, for more information.

To access the Backup Service UI, proceed as follows:

The Backup Service allows backups (and merges) to be scheduled, as tasks. This section describes how task-definition and scheduling can be accomplished. Note that for any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. Therefore, the administrator-defined interval between tasks should always be sufficient to allow each task to run to completion. If a new task is scheduled to start while a previously started task is still running, the new task cannot run. For information, see Avoiding Task Overlap.

To schedule one or more backups, proceed as follows:

This concludes the process for creating repository and plan. The Backup screen now appears as follows:

The newly created repository, hourlyBackupRepo, is thus displayed with its associated plan, HourlyBackupPlan, with the affected bucket (travel-sample) and the next scheduled backup displayed. Data Service and Index Service data for travel-sample will now be backed up to the specified location on the specified schedule.

A repository whose plan is being executed (with data thereby backed up repeatedly, on schedule) is referred to as an active repository.

By means of the Backup Service, an immediate backup can be run: this eliminates the need to wait for a scheduled backup to run at an appointed time. To run an immediate backup, access the Backup screen, and left-click on the row for an already-defined, active repository. For example:

This causes the row to expand vertically, as follows:

A number of buttons now appear, arranged horizontally across the bottom of the row, permitting a variety of actions. To perform an immediate backup, left-click on the Run Backup button:

This displays the Trigger Backup dialog, which appears as follows:

The immediate backup to be performed will be incremental by default. To perform a full backup, check the Perform a full backup checkbox.

Left-click on the Backup button, at the lower right of the dialog. The dialog disappears, and a notification is displayed at the lower left of the console:

This duly indicates that an immediate backup has been triggered.

Using Couchbase Web Console, the history of backups to a specified repository can be reviewed. Left-click on the row of a repository, to expand it vertically. Then, left-click on the Inspect Backups button:

This displays the Backup facility’s Repository screen, which appears as follows:

The screen provides two possible views, which are Inspect Backups and Task History : these can be selected by means of the buttons at the upper right:

The Inspect Backups view is selected by default. (Note the left-clicking the Task History button displays the Tasks History view: this is the same display as that accessed by means of the Task History button, from the expanded row on the *Repositories view of the Backup screen; and is described in Inspect Tasks, below.)

The main, lower panel of the Backups view provides the ID of the repository (in this case, 83f3b752-78e6-49f8-a527-2844c30fbc75) and its size (here, 235.551MiB); and also provides a vertically arranged list of all backups that have occurred, with the earliest at the top. Each backup has its own row; with its start-time, type (full or incremental), and size. To inspect a particular backup in detail, left-click on the control at the left-hand side of the row:

This causes the row to expand vertically:

The displayed data includes the UUID for the source cluster. Also specified are the numbers of Eventing Functions written for the Eventing Service, and the number of Full Text Search Aliases for the Search Service (here, the numbers are both zero).

Each bucket that has been backed up (in this case, the travel-sample bucket alone), appears on its assigned row in a table that specifies its size; along with the number of items, mutations, and tombstones that have been included in the backup. The row also lists the numbers of backed up indexes for the Index, Search, and Analytics Services; and the number of backed up Views. Additionally, in a searchable sub-panel, each scope that the bucket contains is individually listed (these here being the default and inventory scopes, and four tenant_agent scopes); with the number of mutations and tombstones listed, per scope.

To inspect the individual collections within a displayed scope, left-click on the row for the scope. The row expands vertically, as follows:

Thus, left-clicking on the row for the inventory scope has displayed the individual collections within the scope; and thereby shows the mutations and tombstones for each collection. Collections can be searched for, based on strings entered into the filter collections field, which is located to the upper right of the collections panel.

The upper panel of the Data screen provides interactive fields labelled Key and Search Path. These can be used to search for a specific document within the repository. Optionally, the subset of backups within the repository can be specified, by means of the Start and End fields. For example, by accessing the control at the left-hand side of the Start field, a pull-down menu is displayed: this lists backups any one of which can be used as the starting point for the search:

For example, type a known document key into the Key field — such as airline_10. Then, enter the bucket name into the Search Path field: note that this requires explicit specification of both scope and collection; unless default scope and collection have been used, in which case, explicit specification of the defaults is optional — for example, travel-sample._default._default.

When a search is expressed to include all backups of the bucket for the inventory scope and airline collection, the panels appear as follows:

To run the search, left-click on the Examine button. The Examine screen is now displayed:

Note that the controls adjacent to the Diff button, near the top of the screen, allow different backups to be selected, so that the differences between the document-versions they contain can be individually examined:

The specified document is thereby shown, in the left and right-hand panels of the main display, in versions that respectively correspond to the backups selected. When a field has changed, the earlier version appears shaded red, the later shaded green.

By default, a Side-by-Side Diff view of the specified document is shown. To display an Inline Diff view, access the control at the upper right of the screen:

The Inline Diff view is now provided:

To inspect currently defined tasks, do either of the following:

The Tasks screen appears as follows:

The Get Tasks button allows specific tasks to be identified by search, and displayed. The earliest date for the task can be specified in the Since Day field; and the name of the task in the Task Name field, so as to narrow the search. A limit on the number of tasks displayed can be specified as an integer, between 1 and 100 inclusive, in the Limit field.

The Refresh Tasks button causes the main list of tasks, in the lower panel, to be refreshed. The list has an Offset figure displayed at its head: this indicates the position in the list of the first displayed task; and changes when the prev batch and next batch controls, at the right-hand side, are left-clicked on. The task list is presented as a table, which shows, for each task that has been executed, the Task name, Task type (such as Backup or Merge), status (such as done or running), the Elapsed time for the task, the number of Items and size of data that was Transferred by the task, and the Start and End times for the task.

To inspect a particular task in detail, left-click on the row for the task. For example:

The selected row is expanded vertically, as follows:

The details of the task are thus displayed as a JSON document. The details include counts of items, vBuckets, and bytes received from the operation. The node_runs subdocument provides information specific to each node in the cluster.

A merge allows multiple backups to be combined as one; with deduplication occurring.

Merges are supported for filesystem-based repositories: however, merges are not supported for cloud-based repositories. If a merge is scheduled for a cloud-based repository, the Backup Service skips the task. An immediate merge cannot be triggered for a cloud-based repository.

Merges can be scheduled as tasks, to be applied to backed up data within a defined repository. This section describes how task-definition and scheduling for merges can be accomplished. Note that for any given repository, the Backup Service performs one task at a time; with each task maintaining a lock on the repository. Therefore, the administrator-defined interval between tasks should always be sufficient to allow each task to run to completion. If a new task is scheduled to start while a previously started task is still running, the new task cannot run. For information, see Avoiding Task Overlap.

Proceed as follows, noting that the initial steps (for adding a repository, creating a new plan, and adding a backup task to the plan) are the same as those described in Schedule Backups, above.

The defined backups and merges will now occur, on the specified schedule. This can eventally be seen by left-clicking on the row for the new repository, and then left-clicking on Inspect Backups.

By accessing a vertically expanded repository-display in the Repositories view of the Backup screen, an immediate merge can be manually triggered. The repository does not need to have scheduled merges in its plan; but must already contain multiple backups, so that some or all of these can be merged.

Note that merges are supported for filesystem-based repositories only: they are not supported for cloud-based repositories.

Proceed as follows:

One or more backups can be restored to the cluster; which means that the data in the backups is copied back into the buckets from which it was originally backed up, or into other buckets. Proceed as follows:

Subsequent to the operation, its results can be checked; by means of the Buckets screen of Couchbase Web Console, which might now appear as follows:

A new bucket, named ts, has thus been created. Its item-count indicates that it contains only a subset of the documents contained in travel-sample, in accordance with the filtering specified for the restore operation.

The Backup Service allows scheduled backups to be paused, indefinitely. This may be useful when a cluster is undergoing maintenance, or when buckets are otherwise not available. Once a pause is executed by the administrator, no scheduled task for the repository is executed. Then, when a resume is subsequently executed by the administrator, the task-schedule is resumed at the earliest, planned opportunity: thus, if backups are hourly, and a pause is executed at 9:50 am, if a resume occurs at 9:58 am, the next backup occurs at 10:00 am; but if the resume occurs at 10:02 am, then the next backup occurs at 11:00 am.

To pause a backup, access the Repositories view of the Backup screen, and left-click on the row for the repository to be paused. This expands the row vertically, and displays the Pause button. Left-click on this, to pause backups:

The button now changes into a Resume button. Left-click on this whenever backups are to be resumed:

When a repository should no longer receive fresh backups, the repository can be archived. This means that it remains available, with all its data; but no longer receives modifications. To archive a repository, proceed as follows;

Repositories can be deleted. However, no repository can be deleted until it has been archived, as described above, in Archive Repositories.

Therefore, to delete a repository, and the backups it contains, proceed as follows:

A repository not currently managed by the Backup Service can be imported into the service, by means of the Backup UI. For example, a repository that was archived and subsequently deleted can be imported back into the service, provided that its on-disk files were not explicitly deleted (see Delete Repositories, above, for information on deleting repositories while preserving on-disk files).

To import a repository, proceed as follows:

All plans created for the Backup Service can be reviewed, by left-clicking on the Plans tab, on the upper, horizontal navigation bar of the Backup screen:

This displays the Backup screen’s Plans view:

The plan displays, in the name column, the name of every current plan; under the instances column, an integer that represents the number of repositories maintained by means of the plan identified on the current row; and under services, the Couchbase Services whose data is handled by the plan.

To inspect a plan in detail, left-click on its row, to make the row expand vertically. For example:

The displayed details indicate that HourlyBackupPlan contains two tasks: the HourlyBackupTask performs a task of type BACKUP every hour; while the FourthHourMergeTask performs a task of type MERGE every 4 hours, merging all backups from the current day to two days ago..

At the right-hand side of each row, a garbage-can icon is displayed. To delete a plan, left-click on the icon on the row of the plan to be deleted: the plan is deleted immediately, and no longer appears on the Plans screen. (Note that a plan cannot be deleted until all repositories using it are archived or themselves deleted.)

Backup-Service scheduling options are provided per task, on the Select Plan dialog; which is provided as part of the sequence for repository-definition, after the administrator has left-clicked on the ADD REPOSITORY tab, at the upper right of the Repositories view of the Backup screen. See Schedule Backups, for details.

The task-definition panel provided for each task appears by default as follows:

The default, Weekly Calendar option refers to use of the two, interactive, days-of-the-week selectors that appear under Full Backups and Normal Backups (a normal backup being an incremental backup). The day-by-day frequency of each kind of backup can thus be determined by left-clicking on the appropriate days. For example, the following selection would indicate that a full backup should occur on Saturday, every week; and an incremental build should occur every week on each of the following days: Monday, Wednesday, and Friday.

A Weekly Calendar schedule means that one backup happens daily. The time of the daily backup can be specified by means of the Time panel. No merge can be scheduled, nor can backups be scheduled more frequently.

Alternatively scheduled merges and backups can be configured by accessing the control at the left-hand side of the Period field:

The pull-down menu thus displayed contains three kinds of scheduling option. One is the default, Weekly Calendar. Another is by means of time-units: which are Minutes, Hours, Days, and Weeks. If a unit is specified, an appropriate integer must be entered into the Frequency field, to indicate the number of time-units that must elapse between repetitions of the task. If Minutes are specified, only the minutes portion of the time entered into the Time field is used — as the starting point, every hour, for the task sequence; with the task being repeated throughout the hour as many times as specified in the Frequency field. If Hours are specified, the task is first performed at the time specified in the Time field, and then repeated as specified in the Frequency field. If Days or Weeks are specified, the task is performed as scheduled, at the time specified in the Time field. (A full example of using Minutes as time-units is provided above, in Schedule Backups.)

A third option is by means of days: such as Monday, Tuesday, and so on. If a day is selected, an appropriate integer must be entered into the Frequency field, to indicate the number of instances of the day’s occurrence that should elapse before the task is performed. For instance, if Monday and 3 are specified, the task is performed on the first Monday, then two Mondays elapse, and then the task is performed again on the fourth Monday; and so on. The time at which the task is commenced is that specified in the Time field.

Cloud storage can be used for backups. However, cloud storage cannot be used for merges. If a merge is scheduled for cloud storage, the Backup Service skips the task. An immediate merge cannot be triggered for cloud storage.

During the input-sequence for repository-creation, storage for the repository can be specified. See Schedule Backups, for an example of using the local filesystem. Alternatively, cloud storage can be selected, from the Storage Locations field of the Select Plan dialog. When the selection is made, the dialog expands vertically, as follows:

The fields are as follows:

Under Advanced cloud options, the Endpoint field accepts an optional endpoint, used to connect to AWS; and overriding the endpoint used by the cloud-provider.

If checked, the S3 Force path style checkbox ensures that the AWS path style used is the earlier (rather than the more recent, which is not supported by all S3 compatible object stores).

The Backup Service can be configured and used by means of the REST API. For reference pages on each supported endpoint, see Backup Service API.