Install and Start Using the Python SDK with Couchbase Server

The Couchbase Python SDK allows Python applications to access a Couchbase cluster. It offers a traditional synchronous API as well as integration with twisted, gevent, and asyncio. It depends on the C SDK, libcouchbase, (included automatically) which it uses for performance and reliability.

The Couchbase Python SDK 3.x is a complete rewrite of the API, reducing the number of overloads to present a simplified surface area, and adding support for Couchbase Server features like Collections and Scopes (available from Couchbase Server 7.0).

The 3.x Python SDK introduces comprehensive PEP-484 style type annotations.

Requirements

Couchbase Python SDK bundles libcouchbase automatically, so no need to install it separately. You may need CMake to install, although the installer will attempt to download it from PyPI automatically.

The Python SDK 3.x requires Python 3, with Python 3.5, Python 3.6, Python 3.7, Python 3.8 and Python 3.9 supported. See the Compatibility section for details.

The Couchbase Python SDK will drop support for Python 3.5 and Python 3.6 with the 4.0.0 release, tentatively April 2022.

Currently the Python Client source distribution requires the OpenSSL headers and libraries that the Python client itself was built against to be installed prior to the client itself for TLS support to be provided. Additionally the installer relies on PEP517 which older versions of PIP do not support. If you experience issues installing it is advised to upgrade your PIP/setuptools installation as follows:

$ python3 -m pip install --upgrade pip setuptools wheel

Linux

Best practice is to use a Python virtual environment such as venv or pyenv to manage multible versions of Python. See pyenv docs for details.

Debian and Ubuntu
RHEL and CentOS

During first-time setup:

$ sudo apt-get install git-all python3-dev python3-pip python3-setuptools cmake build-essential

For TLS/SSL support (recommended):

Connecting to Couchbase Capella requires TLS support.

$ sudo apt-get install libssl-dev

During first-time setup:

$ sudo yum install gcc gcc-c++ git python3-devel python3-pip cmake

You may need to update your installed version of CMake. For example, by following the steps here.

For TLS/SSL support (recommended):

Connecting to Couchbase Capella requires TLS support.

$ sudo yum install openssl-devel

RHEL/CentOS distributions may not provide the python3-pip package in the base repositories. It may be found in the EPEL repository.

Mac OS

Best practice is to use a Python virtual environment such as venv or pyenv to manage multible versions of Python, but in cases where this is not practicable follow the brew steps below, and also modify your $PATH as shown.

There can be a problem when using the Python (3.8.2) that ships with Xcode on Catalina. It is advised to install Python via pyenv (see the Python SDK Github README for further details on pyenv installation), Homebrew, or python.org

To install the library on Mac OS, first install Homebrew.

Later versions of Mac OS can break the python3 homebrew installer. Simple mitigating steps may be found here.

The following example uses the Python supplied by the Homebrew package manager and not the vendor-supplied Python which ships with Mac OS. Once Homebrew is configured:

Get a list of the latest packages

$ brew update

Install compatible Python 3

$ brew install python3

For ZSH (MacOS 10.15 Catalina and newer)

$ echo 'export PATH="/usr/local/bin:"$PATH' >> ~/.zshrc

For BASH (MacOS 10.14 Mojave and older)

$ echo 'export PATH="/usr/local/bin:"$PATH' >> ~/.bash_profile

For TLS/SSL support - recommended

Best practice is to use a secure connection. However, since some scenarios don’t require a secure connection, TLS support is recommended but not required.

Connecting to Couchbase Capella requires TLS support.

$ brew install openssl

Microsoft Windows

Download and install Python from python.org. Best practice is to use a Python virtual environment such as venv or pyenv.

Checkout the pyenv-win project to manage multiple versions of Python.

Wheels are available on Windows for Python 3.7, 3.8 and 3.9.

Installing on Linux

First, make sure the requirements have been installed.

Install the latest Python SDK:

$ python3 -m pip install couchbase

Installation on Mac OS

First, make sure the requirements have been installed.

Install the latest Python SDK:

$ sudo -H python3 -m pip install couchbase

Installing on Microsoft Windows

First, make sure the requirements have been installed.

Commands assume user is working within a virtual environment.

Install the latest Python SDK (if using Python 3.7, 3.8 or 3.9):

python -m pip install couchbase

The standard Python distributions for Windows include OpenSSL DLLs, as PIP and the inbuilt ssl module require it for correct operation. The binary wheels for Windows are packaged as a binary wheel built against the relevant version OpenSSL (which is fixed per Windows version of Python).

If you require a version without OpenSSL support, or that doesn’t have a suitable binary wheel on PyPi, follow the build instructions on the GitHub repo.

Installing with Anaconda/Miniconda

To use the SDK within the Anaconda/Miniconda platform, make sure the prerequisites for the desired Operating System are met:

Linux
Mac OS
Windows

In the Anaconda Prompt, create a new environment:

conda create -n test_env python=3.9

Activate the environment:

conda activate test_env

Install the SDK:

python -m pip install couchbase

If using Windows, and no wheel is available, see the alternate installlation methods on the Github README. The same process should work within the Anaconda/Miniconda platform.

Hello Couchbase

At this point we want to transition from the terminal to your code editor of choice.

Let’s now create an empty file named cb-test.py and walk through adding code step-by-step:

Connect to a cluster, bucket, and default collection;
Add and retrieve a new document;
Look up (SQL-type query) the new document by attribute value.

Prerequisites

As well as the Python SDK (see above), and a running instance of Couchbase Server, you will need to load up the Travel Sample Bucket using either the Web interface or the command line.

Connection

The basic imports and connection string that you’ll need are given below — for more background information, refer to the Managing Connections page.

# needed for any cluster connection
from couchbase.cluster import Cluster, ClusterOptions
from couchbase.auth import PasswordAuthenticator

# needed to support SQL++ (N1QL) query
from couchbase.cluster import QueryOptions

# get a reference to our cluster
cluster = Cluster('couchbase://localhost', ClusterOptions(
  PasswordAuthenticator('Administrator', 'password')))

Couchbase uses Role Based Access Control (RBAC) to control access to resources. For the sake of this example, we are connecting to Couchbase using the Full Admin role created during the installation of our Couchbase Server. The connection is being made to a single Couchbase node, running locally.

Couchbase RBAC is fully described in the section Authorization. An authenticator, containing username and password, should be defined, and then passed to the cluster. Following successful authentication, the bucket can be opened:

# get a reference to our bucket
cb = cluster.bucket('travel-sample')

We are working with the travel-sample data bucket. If you are not, substitute travel-sample with your bucket-name.

If you are connecting to Couchbase Capella rather than a local Couchbase Server, then also refer to the Cloud section, below.

See Managing Connections for more connection options and details about the connection string.

cb_coll = cb.scope("inventory").collection("airline")

The 3.2 SDK is ready for the introduction of Collections in the 7.0 release of the Couchbase Data Platform. The latest release, Couchbase Server 7.0, brings Collections, allowing Documents to be grouped by purpose or theme, according to specified Scope.

This feature was previously available as a developer preview from Couchbase Server 6.6. When connecting to a 6.6 cluster or earlier, we must use the DefaultCollection, which covers the whole Bucket.

# get a reference to the default collection, required for older Couchbase server versions
cb_coll_default = cb.default_collection()

Document Addition and Retrieval

Let’s create a dictionary object in our application that we can add to our travel-sample bucket that conforms to the structure of a document of type airline.

airline = {
  "type": "airline",
  "id": 8091,
  "callsign": "CBS",
  "iata": None,
  "icao": None,
  "name": "Couchbase Airways",
}

Document operations, such as storing and retrieving documents, can be done using simple methods on the Collection class such as Collection.get and Collection.upsert. Simply pass the key (and value, if applicable) to the relevant methods.

The following function will upsert() a document and print the returned CAS value:

def upsert_document(doc): 
  print("\nUpsert CAS: ")
  try:
    # key will equal: "airline_8091"
    key = doc["type"] + "_" + str(doc["id"])
    result = cb_coll.upsert(key, doc)
    print(result.cas)
  except Exception as e:
    print(e)

Now, we can simply call the upsert_document function passing in our airline document:

upsert_document(airline)

Now let’s retrieve that document using a key-value operation. The following function runs a get() for a document key and either logs out the result or error in our console:

# get document function
def get_airline_by_key(key):
  print("\nGet Result: ")
  try:
    result = cb_coll.get(key)
    print(result.content_as[str])
  except Exception as e:
    print(e)

Key-value Operations are described in detail on the KV Operations page.

Now, we can simply call the get_airline_by_key function passing in our valid document key airline_8091:

get_airline_by_key("airline_8091")

SQL++ Lookup

Couchbase N1QL queries are performed by invoking the Cluster.query() method. The following function executes a lookup for documents of type='airline' by a provided callsign:

# query for new document by callsign
def lookup_by_callsign(cs):
  print("\nLookup Result: ")
  try:
    sql_query = 'SELECT VALUE name FROM `travel-sample`.inventory.airline WHERE callsign = $1'
    row_iter = cluster.query(
      sql_query,
      QueryOptions(positional_parameters=[cs]))
    for row in row_iter: print(row)
  except Exception as e:
    print(e)

We call the lookup_by_callsign function passing in our callsign CBS:

lookup_by_callsign("CBS")

Execute!

Now we can run our code using the following command:

$ python3 cb-test.py

The results you should expect are as follows:

Upsert CAS:
1598469741559152640

Get Result:
{'type': 'airline', 'id': 8091, 'callsign': 'CBS', 'iata': None, 'icao': None, 'name': 'Couchbase Airways'}

Lookup Result:
Couchbase Airways

Full Example

If you want to copy and paste to run the full example, here it is:

Local Couchbase Server
Couchbase Capella Sample

# needed for any cluster connection
from couchbase.cluster import Cluster, ClusterOptions
from couchbase.auth import PasswordAuthenticator

# needed to support SQL++ (N1QL) query
from couchbase.cluster import QueryOptions

# get a reference to our cluster
cluster = Cluster('couchbase://localhost', ClusterOptions(
  PasswordAuthenticator('Administrator', 'password')))

# get a reference to our bucket
cb = cluster.bucket('travel-sample')

cb_coll = cb.scope("inventory").collection("airline")

def upsert_document(doc): 
  print("\nUpsert CAS: ")
  try:
    # key will equal: "airline_8091"
    key = doc["type"] + "_" + str(doc["id"])
    result = cb_coll.upsert(key, doc)
    print(result.cas)
  except Exception as e:
    print(e)

# get document function
def get_airline_by_key(key):
  print("\nGet Result: ")
  try:
    result = cb_coll.get(key)
    print(result.content_as[str])
  except Exception as e:
    print(e)

# query for new document by callsign
def lookup_by_callsign(cs):
  print("\nLookup Result: ")
  try:
    sql_query = 'SELECT VALUE name FROM `travel-sample`.inventory.airline WHERE callsign = $1'
    row_iter = cluster.query(
      sql_query,
      QueryOptions(positional_parameters=[cs]))
    for row in row_iter: print(row)
  except Exception as e:
    print(e)

airline = {
  "type": "airline",
  "id": 8091,
  "callsign": "CBS",
  "iata": None,
  "icao": None,
  "name": "Couchbase Airways",
}

upsert_document(airline)

get_airline_by_key("airline_8091")

lookup_by_callsign("CBS")

If you are connecting to Couchbase Capella, be sure to get the correct endpoint as well as user, password, certificate and couchbasecloudbucket — and see the Cloud section, below.

from couchbase.cluster import Cluster, ClusterOptions
from couchbase.cluster import PasswordAuthenticator
from couchbase.cluster import QueryOptions

# Update this to your cluster
endpoint = "--your-instance--.dp.cloud.couchbase.com"
username = "username"
password = "password"
bucket_name = "bucketname"
cert_path = "path/to/certificate"
# User Input ends here.

# Initialize the Connection
cluster = Cluster("couchbases://{}".format(endpoint), ClusterOptions(
    PasswordAuthenticator(username, password, cert_path=cert_path)))
cb = cluster.bucket(bucket_name)
cb_coll = cb.default_collection()

# Create a N1QL Primary Index (but ignore if it exists)
cluster.query_indexes().create_primary_index(
    bucket_name, ignore_if_exists=True)

# Store a Document
cb_coll.upsert("u:king_arthur", {
               "name": "Arthur", "email": "kingarthur@couchbase.com", "interests": ["Holy Grail", "African Swallows"]})

# Load the Document and print it
print(cb_coll.get("u:king_arthur").content_as[str])

# Perform a N1QL Query
row_iter = cluster.query("SELECT cbc.* FROM {} cbc WHERE $1 IN cbc.interests".format(
    bucket_name), QueryOptions(positional_parameters=["African Swallows"]))

# Print each found Row
for row in row_iter.rows():
    print("Found row: {}".format(row))

Cloud Connections

For developing on Capella, if you are not working from the same Availability Zone as your Capella instance, refer to the following:

Notes on Constrained Network Environments.
Network Requirements.
If you have a consumer-grade router which has problems with DNS-SRV records review our Troubleshooting Guide.

Additional Resources

The API reference is generated for each release and the latest can be found here.

Older API references are linked from their respective sections in the Individual Release Notes. Most of the API documentation can also be accessed via pydoc.

The Migrating from SDK2 to 3 page highlights the main differences to be aware of when migrating your code.

Couchbase welcomes community contributions to the Python SDK. The Python SDK source code is available on GitHub.

PyPy support

Because the Python SDK is written primarily in C using the CPython API, the official SDK will not work on PyPy.

Please create a Jira ticket if you require a PyPy-compatible version of the SDK.