cbft-bleve for Zap Files

The cbft-bleve tool supports commands that return information on Zap files.

Each Scorch-index partition is represented by a Zap file, which is located in the store sub-directory for the partition (see Specifying Index Partitions). Information can be retrieved from the Zap file, by means of the zap command. The basic syntax is as follows:

cbft-bleve zap [command] [flags]

The possible values of command are listed in the sections below. The only possible value of flags is --help (or -h), which prints help text to standard output.

All examples provided below are for MacOS.

dict

The dict command prints the term dictionary for a given indexed field-name within a specified Zap file. The term dictionary constitutes a list of the values for the field, along with the total incidence of each. The syntax is as follows:

cbft-bleve zap dict [zap-file-path] [field] [flags]

The field argument is mandatory. See Creating Indexes, for information on indexing — in particular, the section Specifying Fields.

Example: Print the Term Dictionary

The following command prints the term dictionary for the field`country`, which is thus assumed to have been indexed, during creation of the index to which the Zap file corresponds:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap dict \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/000000000113.zap country

If successful, the command returns output as follows:

dictionary for field starts at 60155 (eafb)
vellum length: 79
raw vellum data:
 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10 82 ca cb c5 c7 00 10 90 c4 d2 d7 cb c8 00 10 86 c2 c1 c5 c1 00 10 92 c2 c1 c8 cb d0 ea 2b ea aa e9 73 e9 01 08 0f 17 75 73 6b 66 12 04 04 00 00 00 00 00 00 00 3e 00 00 00 00 00 00 00
dictionary:
 france - 59763 (e973) posting byteSize: 18 cardinality: 1
 kingdom - 59818 (e9aa) posting byteSize: 20 cardinality: 2
 states - 59947 (ea2b) posting byteSize: 32 cardinality: 8
 united - 60112 (ead0) posting byteSize: 36 cardinality: 10
Total terms in dictionary : 4 1hit count: 0

The output thus includes the values found for each instance of the term country; providing the total incidence of each.

docvalues

The docvalues command returns document values from a Zap file, in order of field, and by number. The syntax is as follows:

cbft-bleve zap docvalue [zap-file-path] [field] [docNum] [flags]

The field and docNum arguments are optional, and represent the name of an indexed field and a document-number, respectively. If 'field' alone is specified, document values for the whole Zap file are returned. If docNum is additionally specified, document values for that document alone are returned.

Example: Return Document Values

The following example returns document values from a Zap file, with the field-name country specified. No document number is specified.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap docvalue \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_fdcb47f176931043_13aa53f3.pindex/store/000000000113.zap country

If successful, the command returns the following:

Total Doc Values Size on Disk: 0.000159 MB
FieldID: 2 'country' docvalue at 60235 (eb4b) numChunks 2  diskSize 0.000159 MB
Number of docvalue chunks: 2

The following example attempts to return document values, again with field-name country; but with the document-number 214.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap docvalue \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_139f2619566ac4d4_acbbef99.pindex/store/0000000000ba.zap country \
214

If the command is not successful, due to the specified document not containing an instance of the specified field, the following output is displayed:

Total Doc Values Size on Disk: 0.000159 MB
No field values found for localDocNum: 214
Try docNums present in chunk: 86, 99, 180, 319, 406, 581, 653, 955,

Thus, the output lists alternative document-numbers, on which the command can be run. For example, 86 can now be tried:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap docvalue \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_139f2619566ac4d4_acbbef99.pindex/store/0000000000ba.zap country \
86

On success, the following output is displayed:

Total Doc Values Size on Disk: 0.000159 MB
 kingdom  united

Thus, the value of the term country in document number 86 is displayed as kingdom united.

stored

The stored command prints the raw stored data bytes for a specified document number. The syntax is as follows:

cbft-bleve zap stored [zap-file-path] [docNum] [flags]

The required docNum argument is the number of a document.

Example: Return Data Bytes for a Document

The following command returns the data bytes for document 2000, as listed in a specified Zap file.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap stored \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_139f2619566ac4d4_acbbef99.pindex/store/0000000000ba.zap 2000

If successful, the command returns the following:

Stored field starts at 30827 (0x786b)
Meta Len: 1
Data Len: 15
Raw meta: 0e
Raw data (len 15): 6c 61 6e 64 6d 61 72 6b 5f 32 35 33 31 39 00
Raw _id (len 14): 6c 61 6e 64 6d 61 72 6b 5f 32 35 33 31 39
Raw fields (len 1): 00
Uncompressed fields (len 0):

explore

The explore command explores the Zap file by field, term, or document number. The syntax is as follows:

cbft-bleve zap explore [path] [field] [term] [docNum] [flags]

Example: Explore by Field and Term

The following command explores the specified Zap file by the field country field, and the term france:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap explore \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_139f2619566ac4d4_acbbef99.pindex/store/0000000000ba.zap \
country france

If successful, the command returns output as follows:

dictionary for field starts at 60155 (eafb)
vellum length: 79
raw vellum data:
 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10 82 ca cb c5 c7 00 10 90 c4 d2 d7 cb c8 00 10 86 c2 c1 c5 c1 00 10 92 c2 c1 c8 cb d0 ea 2b ea aa e9 73 e9 01 08 0f 17 75 73 6b 66 12 04 04 00 00 00 00 00 00 00 3e 00 00 00 00 00 00 00
FST val is 59763 (e973)
Posting List Length: 18
Posting List: {406}
Freq details at: 59745 (e961)
freq chunk: 0, len 6, start at 59748 (e964) end 59754 (e96a)
freq chunk: 1, len 6, start at 59754 (e96a) end 59760 (e970)
Loc details at: 59754 (e96a)
there are 2 loc chunks
loc chunk: 0, len 6(6), start at 59757 (e96d) end 59763 (e973)
loc chunk: 1, len 6(6), start at 59763 (e973) end 59769 (e979)

fields

The fields command prints out the indexed fields that are contained within a specified Zap file. The syntax is as follows:

cbft-bleve zap fields [zap-file-path] [flags]

Example: Print the Fields in a File

The following example prints out the fields in the specified Zap file. The field country is assumed to have been indexed for airline documents, within the travel-sample bucket.

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin\
/cbft-bleve zap fields \
/Users/<username>/Library/Application\ Support/Couchbase/var/lib/couchbase/data/@fts\
/scorchIndex_139f2619566ac4d4_acbbef99.pindex/store/0000000000ba.zap

If successful, the command prints out the following:

field 0 '_id' starts at 44746 (aeca)
field 1 '_all' starts at 59665 (e911)
field 2 'country' starts at 60155 (eafb)

The indexed fields and their positions within the index are displayed.

The footer command prints out the footer for the specified Zap file. The syntax is as follows:

cbft-bleve zap footer [zap-file-path] [flags]

The following example prints out the footer of the specified Zap file:

/Applications/Couchbase\ Server.app/Contents/Resources/couchbase-core/bin/\
cbft-bleve zap footer \
/Users/<username>/Library/Application\ Support/\
Couchbase/var/lib/couchbase/data/@fts/\
demoIndex_1125629e41599ad8_1c9ffdcc.pindex/store/000000000026.zap