cbft-bleve for Scorch Indexes

The cbft-bleve tool supports commands that return information on Scorch index partitions.

Using cbft-bleve on Scorch indexes.

The cbft-bleve tool can be used to retrieve information on a Scorch index partition in either of the following ways:

By means of the subset of top level command-values that can also be used on Moss index partitions.
Specifically by means of the scorch top level command-value, which entails additional syntax, and can be used on Scorch index partitions only.

These options are described below. Examples are for MacOS.

Using Top Level Command-Values

The top-level Syntax for cbft-bleve permits the following to be specified as values for the principal command, and to be used alike on Moss index partitions (on all Couchbase-Server supported platforms) and on Scorch index partitions (on all platforms except CentOS and Ubuntu): check, count, dictionary, fields, mapping, query, and registry.

For a description of the syntax entailed by each value, and instances of command-output, see the examples provided for Moss index partitions, in cbft-bleve for Moss Indexes.

Using the `scorch` Command-Value

The scorch command-value specifies that the target index-partition is a Scorch index. The basic syntax is as follows:

cbft-bleve scorch [command] [index-store-path] [snapshot] [flags]

The available commands are listed in the sections below. A command must be specified. Note that a command summary can be printed to standard output, by entering cbft-bleve scorch --help (or -h).

The index-store-path, which is mandatory, is the filesystem path of the store subdirectory, located under the directory for the Scorch index partition. See Specifying Index Partitions.

The snapshot is a reference to a particular snapshot, within a Scorch-index partition, on which information is to be retrieved. See immediately below.

The flags argument can always be --help, which prints help text to standard output. Additional values are indicated as appropriate, below.

Understanding Snapshots

Each Scorch index contains a number of snapshots, each of which represents the state of all indexed data at a particular point in time: this supports high-performance rollbacks of the index. By default, three snapshots are simultaneously available: each is referred to by a unique integer known as the snapshot epoch. To change the number of available snapshots, include the numSnapshotsToKeep key to the store object within the index-definition, specifying an appropriate integer as the value. For example, the following establishes 4 snapshots to be made simultaneously available:

   "store": {
     "kvStoreName": "scorchStore",
     "indexType": "scorch",
     "numSnapshotsToKeep": 4
   }

For more information, see Index-Creation with the REST API.

Some commands used by cbft-bleve scorch refer to and return information on a specific snapshot epoch. Information on currently available snapshot epochs and their unique identifying integers is returned by means of the snapshot command.

snapshot

The snapshot command prints a list of snapshot epochs that currently exist for a specified Scorch index partition. The syntax is as follows:

cbft-bleve scorch snapshot [index-store-path] [flags]

Example: Print Snapshot Epochs

The following example prints the snapshot epochs for the specified index partition:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve scorch snapshot \
/Users/<username>/Library/Application\ Support/Couchbase\
/var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/

If successful, the command prints the following:

snapshot epoch: 37
snapshot epoch: 36
snapshot epoch: 1

ascii

The ascii command prints an ascii representation of the segments in a snapshot. Each segment is a batch of items from the snapshot. The syntax is as follows:

cbft-bleve scorch ascii [index-store-path] [snapshot]

The snapshot argument, which is required, must be the unique integer associated with a snapshot epoch for the specified index, as returned by the snapshot command.

Example: Print an ASCII Representation

The following example prints an ascii representation of the segments in the snapshot specified for an index partition.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve scorch ascii \
/Users/<username>/Library/Application\ Support/Couchbase\
/var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ \
37

If successful, the command returns the following:

37    37:  5433 / 5433 -   .........................

deleted

The deleted command prints a bitmap that represents items that have been deleted from each segment of a specified snapshot, during the course of the segment’s existence. The syntax is as follows:

cbft-bleve scorch deleted [index-store-path] [snapshot] [flags]

The snapshot argument, which is required, must be the unique integer associated with a snapshot epoch for the specified index partition, as returned by the snapshot command.

Example: Print the Deleted Bitmap

The following example prints the bitmap of deleted items for the segments in snapshot 67:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve scorch deleted \
/Users/<username>/Library/Application\ Support/Couchbase\
/var/lib/couchbase/data/@fts/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ \
67

If successful, the command returns a list that contains deleted items, if any exist:

0 <nil>
1 <nil>
2 <nil>

info

The info command prints information on a specified index partition. The syntax is as follows:

cbft-bleve scorch info [index-store-path] [flags]

Example: Print Info on an Index Partition

The following example prints information on the specified Scorch index partition.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve scorch info \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/

If successful, the command returns the following:

doc count: 5433

internal

The internal command prints the internal kv pairs within a snapshot — these constitute meta data for the snapshot. The syntax is as follows:

cbft-bleve scorch internal [index-store-path] [snapshot] [flags]

The snapshot, which is mandatory, must be the unique integer associated with a snapshot epoch for the specified index, as returned by the snapshot command.

The flags argument can be -a or --ascii; which indicates that the values are to be printed in ascii.

Example: Print KV Pairs

The following example provides an ascii print-out of the kv pairs for the specified index-snapshot:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/\
cbft-bleve scorch internal \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/ 37 -a

If successful, the command provides the following output:

42 ?
43 ?
_mapping {"types":{"product":{"enabled":true,"dynamic":true,"properties":{"price":{"enabled":true,"dynamic":false,"fields":[{"name":"price","type":"number","index":true,"include_term_vectors":true,"include_in_all":true}]}}}},"default_mapping":{"enabled":false,"dynamic":true},"type_field":"_type","default_type":"_default","default_analyzer":"standard","default_datetime_parser":"dateTimeOptional","default_field":"_all","store_dynamic":false,"index_dynamic":true,"docvalues_dynamic":true,"analysis":{}}
o:39 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":501,"failOverLog":[[149680438380220,0]]}
36 ?
41 ?
38 ?
o:33 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":484,"failOverLog":[[81822404584738,0]]}
o:36 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":492,"failOverLog":[[174103028649261,0]]}
o:37 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":486,"failOverLog":[[110924524680780,0]]}
o:41 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":497,"failOverLog":[[155103402616817,0]]}
o:43 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":507,"failOverLog":[[217538306806458,0]]}
33 ?
37 ?
o:42 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":492,"failOverLog":[[47136605887494,0]]}
34 ?
o:35 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":485,"failOverLog":[[181174121062964,0]]}
40 ?
o:34 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":499,"failOverLog":[[128188523546156,0]]}
o:38 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":486,"failOverLog":[[161601347095991,0]]}
o:40 {"seqStart":0,"seqEnd":0,"snapStart":0,"snapEnd":504,"failOverLog":[[38787600199365,0]]}
35 ?
39 ?