VGA

Dockerize your integration tests

NOTE

Originally published at dev.to on Jan 4, 2019

Most of our applications have to talk to a database, a HTTP API, a message broker, a SMTP server, etc… And it’s quite complicated to set up a real test environment with those components.

In some cases, we can simply mock those components or have an in-memory one during test execution. For example, H2 ou HSQLDB are in-memory databases well-known for being used during integration tests. However, they are not the one used in production environment and our tests can seem unrepresentative. Today, it’s possible to use all the power of Docker and set up a connected test environment easily thanks to Testcontainers.

testcontainers logo

Testcontainers

Testcontainers allows us to easily manipulate Docker containers during test execution. It uses the Docker client docker-java to communicate with Docker daemon. It works with most operating systems and environments and despite the best-efforts support for Windows, I use it on a daily basis with Docker Toolbox. You can find the matrix compatibility here.

When you create a container, Testcontainers will try to connect to the Docker daemon by using DOCKER_HOST, DOCKER_TLS_VERIFY and DOCKER_CERT_PATH variables. These environment variables can be easily overridden in the JVM for example.

Create a container

Containers are represented with the object GenericContainer. It’s possible to create a container from an image, a Dockerfile or from a Dockerfile created on the fly. In addition, it’s possible to create containers from a Docker Compose file. For instance, this is an Elasticsearch server created from the image docker.elastic.co/elasticsearch/elasticsearch:6.1.1.

GenericContainer container = new GenericContainer("docker.elastic.co/elasticsearch/elasticsearch:6.1.1")
   .withEnv("discovery.type", "single-node")
   .withExposedPorts(9200)
   .waitingFor(
      Wait
       .forHttp("/_cat/health?v&pretty")
       .forStatusCode(200)
   );

We can see that it’s fairly easy to provide environment variables to the container with the method withEnv. In this case, it’s the variable discovery.type with value single-node. Next, we make sure that our container is up by making an HTTP call on /_cat/health API and having a 200 code response. There are also other strategies to assert a container is running:

  • Wait.forLogMessage waits for a log message,
  • Wait.forListeningPort waits for a listening port
  • Wait.forHealthcheck enables to use HEALTHCHECK feature from docker.

To finalize the container configuration, our container is exposing internal port 9200 and this is explicitly set with the method withExposedPorts. It means that Testcontainers will map this container’s port to a random port. It’s possible to retrieve mapped port with the method getMappedPortotherwise we can define port bindings with the method setPortBindings. Here, we expose the port 9200 from our container to port 9200:

container.setPortBindings(Arrays.asList("9200:9200"));

That’s it! Our Elasticsearch server is ready to be used. To start it up, we simply have to execute the start method:

container.start();

At startup, Testcontainers will run a bunch of checks like the docker version or the connection to the registered Docker Registry. This can be blocking if you are working behind a company proxy, so it’s possible to disable those checks by creating the file testcontainers.properties in the tests resources directory with this content:

check.disable=true

At last, we can stop our container with the method stop.

container.stop();

This will stop the container and also remove the attached volume. This is great because it prevents having dangling volumes.

##During Tests One great strength of Testcontainers is its integration with JUnit framework. In fact, GenericContainer objects are JUnit rules. It means that their lifecycle is directly bound to the test lifecycle. Thereby, by using the @Rule or @ClassRule JUnit annotations, our containers will be initialized before the test start-up and stopped at the end of the tests execution.

@ClassRule
public static GenericContainer redis = new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);

Nevertheless, it means that Testcontainers will come with a JUnit 4 dependency and it can be annoying if your tests run with JUnit 5. Indeed, JUnit has replaced the Rule concept with Extension. Since the version 1.10.0 released on November 2018, Testcontainers supports now JUnit 5 and it’s possible to use extensions with the help of @Testcontainersand @Container annotations from the dedicated library junit-jupiter:

<dependency>
 <groupId>testcontainers</groupId>
 <artifactId>junit-jupiter</artifactId>
 <version>1.10.2</version>
</dependency>

##Preconfigured container Like Docker, Testcontainers ecosystem is very rich. You can find preconfigured containers like MySql, PostgreSQL, Oracle database, Kafka, Neo4j, Elasticsearch, etc.

@Rule
public KafkaContainer kafka = new KafkaContainer();

You can browse the list directly from maven repository.

#A concrete case Let’s see a concrete example of using Testscontainers with the Spring PetClinic application. It’s a demonstrating project based on several Spring components like Spring Boot, Spring MVC and Spring JPA. This application aims at managing a pet clinic with pets, pet owners and vets.

application petclinic

The controller layer exposes HTTP endpoints to create and read entities. Then, the persistence layer communicates with a relational database. The application can be configured to communicate with a HSQLDB or a MySql database.

The persistence layer is tested with integration tests and those uses an in-memory HSQL database while the persistence layer itself uses a MySql database.

##Requirements First, we have to install Docker on the machine which is going to execute tests. Then, we need to add the Testcontainers dependency to the project. In this case, we simply add the following to the pom.xml file:

<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>testcontainers</artifactId>
  <version>1.10.2</version>
  <scope>test</scope>
</dependency>

Database configuration

The default database configuration is done in the application.properties file.

database=hsqldb
spring.datasource.schema=classpath*:db/${database}/schema.sql
spring.datasource.data=classpath*:db/${database}/data.sql

As we can see, this is an in-memory HSQLDB database initialized with a schema from the schema.sql file. Then, the database is populated with the data.sql file. This is the default project configuration. We need to create application-test.properties file to configure a connection to a MySql database.

spring.datasource.url=jdbc:mysql://localhost/petclinic
spring.datasource.username=petclinic
spring.datasource.password=petclinic
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

Next, let’s take the test class ClinicServiceTests.java. This class contains all integration tests for the persistence layer. First of all, we need to change Spring test configuration to ensure that the tests will use our database connection.

The @TestPropertySource annotation enables to load our file application-test.properties and @AutoConfigureTestDatabase with the NONE value prevents Spring from creating an embedded database.

MySql container

Let’s create a MySql database that matches requirements from our tests. In this instance, we use the ability from Testcontainers to create a Docker image from a Dockerfile created on the fly. As a first step, we have pulled a MySql official image from Docker Hub:

Now, we have to create our database and the connection’s user. This is done by using environment variables from the Docker image.

Next, we have to create a database schema and populate the database. From the image documentation, the directory /docker-entrypoint-initdb.d is scanned at startup and all files with .sh, .sql et .sql.gz extension are executed. So, we just have to put our files schema.sql and data.sql in this directory.

By using withClasspathResourceMapping, the files schema.sql and data.sql are put on the classpath into the container as a volume. Then, we can access it into our Dockerfile construction. One last thing, we have to expose the default MySql port: 3306.

Unfortunately, we can’t directly set port bindings with the method setPortBindings. We have to customize the container on creation with the method withCreateContainerCmdModifier. Finally, we are waiting for the listening port to ensure that our container is up. Voilà! With few lines of code, we have easily set a MySql database for our tests without having to manage the container lifecycle. The @ClassRule annotation makes our container starting once for all the tests. You might be wondering: have we extend the test execution time? In fact, it only takes 907 ms with a Docker container against 860 ms with a HSQLDB in-memory database. The source code shown in this section is available on github.

A big thanks to Sonyth, Sebastien, Laurent, Louis and Nicolas for their time and proofreading.