opensource.google.com

Menu

Cloud Spanner Emulator

Thursday, April 30, 2020

After Cloud Spanner’s launch in 2017, there has been huge customer adoption across several different industries and verticals. With this growth, we have built a large community of application developers using Cloud Spanner. To make the service even more accessible and open to the broader developer community, we are introducing an offline emulator for the Cloud Spanner service. The Cloud Spanner emulator is intended to reduce application development cost and improve developer productivity for the customers.

The Cloud Spanner emulator provides application developers with the full set of APIs, including the breadth of SQL and DDL features that could be run locally for prototyping, development and testing. This open source emulator will provide application developers with the transparency and agility to customize the tool for their application use.

This blog introduces the Cloud Spanner emulator and will guide you through installation and use of the emulator with the existing Cloud Spanner CLI and client libraries.

What is Cloud Spanner Emulator?

The emulator provides a local, in-memory, high-fidelity emulator of the Cloud Spanner service. You can use the emulator to prototype, develop and hermetically test your application locally and in your integration test environments.

Because the emulator stores data in-memory, it will not persist data across runs. The emulator is intended to help you use Cloud Spanner for local development and testing (not for production deployments); However, once your application is working with the emulator, you can proceed to end-to-end testing of your application by simply changing the Cloud Spanner endpoint configuration.

Supported Features

The Cloud Spanner emulator exposes the complete set of Cloud Spanner APIs including instances, databases, SQL, DML, DDL, sessions, and transaction semantics. Support for querying schema metadata for a database is available via Information Schema. Both gRPC and REST APIs are supported and can be used with the existing client libraries, OSS JDBC driver as well as the Cloud SDK. The emulator is supported natively on Linux, and requires Docker on MacOS and Windows platforms. To ease the development and testing of an application, IDEs like IntelliJ and Eclipse can be configured to directly communicate with the Cloud Spanner emulator endpoint.

The emulator is not built for production scale and performance, and therefore should not be used for load testing or production traffic. Application developers can use the emulator for iterative development, and to implement and run unit and integration tests.

A detailed list of features and limitations is provided on Cloud Spanner emulator README. The emulator is currently (as of April 2020) in beta release and will be continuously enhanced for feature and API parity with Cloud Spanner service.

Using the Cloud Spanner Emulator

This section describes using the existing Cloud Spanner CLI and client libraries to interact with the emulator.

Before You Start

Starting the emulator locally

The emulator can be started using Docker or using the Cloud SDK CLI on Linux, MacOS and Windows. In either case, MacOS and Windows would require an installation of docker.

Docker

$ docker pull gcr.io/cloud-spanner-emulator/emulator
$ docker run -p 9010:9010 -p 9020:9020 gcr.io/cloud-spanner-emulator/emulator

Note: The first port is the gRPC port and the second port is the REST port.

Cloud SDK CLI

$ gcloud components update beta
$ gcloud beta emulators spanner start
Other alternatives to start the emulator, including pre-built linux binaries, are listed here.

Setup Cloud Spanner Project & Instance

Configure Cloud Spanner endpoint, project and disable authentication:
$ gcloud config configurations create emulator
$ gcloud config set auth/disable_credentials true
$ gcloud config set project test-project
$ gcloud config set api_endpoint_overrides/spanner http://localhost:9020/
Note:
To switch back to the default config:
`$ gcloud config configurations activate default`
To switch back to the emulator config:
`$ gcloud config configurations activate emulator`

Verify gCloud is working with the Cloud Spanner Emulator.
$ gcloud spanner instance-configs list

NAME               DISPLAY_NAME
emulator-config    Emulator Instance Config
Create a Cloud Spanner Instance
$ gcloud spanner instances create test-instance --config=emulator-config --description="Test Instance" --nodes=1

Using Cloud Spanner Client Libraries

With the beta lunch, the latest versions of Java, Go and C++ Cloud Spanner client libraries are supported to interact with the emulator. Use the Getting Started guides to try the emulator.

Prerequisite: Setup Cloud Spanner Project and Instance from step above.

This is an example of running the Java client library with the emulator:
# Configure emulator endpoint
$ export SPANNER_EMULATOR_HOST="localhost:9010"

# Cloning java sample of client library.
$ git clone https://github.com/GoogleCloudPlatform/java-docs-samples && cd java-docs-samples/spanner/cloud-client

$ mvn package

# Create database
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    createdatabase test-instance example-db


# Write
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    write test-instance example-db


# Query
$ java -jar target/spanner-google-cloud-samples-jar-with-dependencies.jar \
    query test-instance example-db
Follow the rest of the sample for Java client library using the Getting Started Guide.

Using the Cloud SDK CLI

Prerequisite: Setup Cloud Spanner Project and Instance from step above.

Configure emulator endpoint

$ gcloud config configurations activate emulator
Create a database

$ gcloud spanner databases create test-database --instance test-instance --ddl "CREATE TABLE TestTable (Key INT64, Value STRING(MAX)) PRIMARY KEY (Key)"
Write into database
$ gcloud spanner rows insert --table=TestTable --database=test-database --instance=test-instance --data=Key=1,Value=TestValue1
Read from database
$ gcloud spanner databases execute-sql test-database --instance test-instance --sql "select * from TestTable"

Using the open source command-line tool spanner-cli

Prerequisite: Setup Cloud Spanner Project, Instance and Database from step above.

Follow examples for an interactive prompt to Cloud Spanner database with spanner-cli.
# Configure emulator endpoint
$ export SPANNER_EMULATOR_HOST="localhost:9010"

$ go get github.com/cloudspannerecosystem/spanner-cli
$ go run github.com/cloudspannerecosystem/spanner-cli -p test-project -i test-instance -d test-database

spanner> INSERT INTO TestTable (Key, Value) VALUES (2, "TestValue2"), (3, "TestValue3");
Query OK, 2 rows affected

spanner> SELECT * FROM TestTable ORDER BY Key ASC;

+-----+----------------+
| Key | Value |
+-----+----------------+
| 2 | TestValue2 |
| 3 | TestValue3 |
+-----+----------------+
2 rows in set

spanner> exit;

Conclusion

Cloud Spanner emulator reduces application development cost and improves developer productivity for Cloud Spanner customers. We plan to continue building and supporting customer requested features and you can follow Cloud Spanner emulator on GitHub for more updates.

By Sneha Shah, Google Open Source
.