Confluent's Python client for Apache Kafka

confluent-kafka-python is based on librdkafka v1.1.0, see the librdkafka v1.1.0 release notes for a complete list of changes, enhancements, fixes and upgrade considerations.

• In-memory SSL certificates (PEM, DER, PKCS#12) support (by @noahdav at Microsoft)
• Use Windows Root/CA SSL Certificate Store (by @noahdav at Microsoft)
• ssl.endpoint.identification.algorithm=https (off by default) to validate the broker hostname matches the certificate. Requires OpenSSL >= 1.0.2(included with Wheel installations))
• Improved GSSAPI/Kerberos ticket refresh
• Confluent monitoring interceptor package bumped to v0.11.1 (#634)

Upgrade considerations

• Windows SSL users will no longer need to specify a CA certificate file/directory (ssl.ca.location), librdkafka will load the CA certs by default from the Windows Root Certificate Store.
• SSL peer (broker) certificate verification is now enabled by default (disable with enable.ssl.certificate.verification=false)
• %{broker.name} is no longer supported in sasl.kerberos.kinit.cmd since kinit refresh is no longer executed per broker, but per client instance.

SSL

New configuration properties:

• ssl.key.pem - client's private key as a string in PEM format
• ssl.certificate.pem - client's public key as a string in PEM format
• enable.ssl.certificate.verification - enable(default)/disable OpenSSL's builtin broker certificate verification.
• enable.ssl.endpoint.identification.algorithm - to verify the broker's hostname with its certificate (disabled by default).
• Add new rd_kafka_conf_set_ssl_cert() to pass PKCS#12, DER or PEM certs in (binary) memory form to the configuration object.
• The private key data is now securely cleared from memory after last use.

Enhancements

• Bump message.timeout.ms max value from 15 minutes to 24 days (@sarkanyi, workaround for #2015)

Fixes

• SASL GSSAPI/Kerberos: Don't run kinit refresh for each broker, just per client instance.
• SASL GSSAPI/Kerberos: Changed sasl.kerberos.kinit.cmd to first attempt ticket refresh, then acquire.
• SASL: Proper locking on broker name acquisition.
• Consumer: max.poll.interval.ms now correctly handles blocking poll calls, allowing a longer poll timeout than the max poll interval.
• configure: Fix libzstd static lib detection
• PyTest pinned to latest version supporting python 2 (#634)