CP testcontainers

Testcontainers modules for Confluent Platform components.

Example

The following snippet will start an Apache Kafka and a Schema Registry container on the same network

final var factory = new CPTestContainerFactory();

final KafkaContainer kafka = factory.createKafka();
final SchemaRegistryContainer schemaRegistry = factory.createSchemaRegistry(kafka);
schemaRegistry.start(); //will implicitly start kafka container

The following snippet can then be used to configure a producer to use these:

properties.put("bootstrap.servers",   kafka.getBootstrapServers());
//...
properties.put("schema.registry.url", schemaRegistry.getBaseUrl());

The following snippet starts a single node Kafka Connect cluster with two connectors from Confluent Hub installed:

final var connect = factory.createCustomConnector(
        Set.of(
            "confluentinc/kafka-connect-s3:latest", 
            "confluentinc/kafka-connect-datagen:0.4.0"
        ), kafka);
connect.start();

The following snippet configures a CP-server container and a schema registry container with RBAC enabled:

final var factory = new CPTestContainerFactory(network);
final var ldap = factory.createLdap();

final var confluentServer = factory.createConfluentServer().enableRbac();

final var sr = factory.createSchemaRegistry(confluentServer).enableRbac();
sr.start()

See the intTest source set for examples on how to set up containers.

Supported components

So far the following components are supported:

• Confluent Server
• Confluent Schema Registry
• ksqlDB
• Kafka Connect
• Confluent Replicator running on Connect
• Confluent REST Proxy.

Moreover, we can create KafkaContainer instances which have RBAC and MDS enabled. So far, the following components can be configured to use RBAC:

• Confluent Server
• Confluent Schema Registry
• Kafka Connect (without secret registry)

Packages

Project package are hosted on jitpack for the time being. Add the following to your gradle file:

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

dependencies {
    implementation 'com.github.christophschubert:cp-testcontainers:Tag'
}

See the jitpack page of the project for information on how to use the package with mvn or sbt.

Noteworthy demos

• LocalStackIntTest shows how to setup S3 sink connector with S3 installation based on localstack.

Troubleshooting

No non-overlapping IPv4 addresses

Status 404: {"message":"could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network"}

Solution: use docker network prune to delete