xk6-kafka v2.1.0

A feature release that completes the v2 roadmap milestone for Protobuf: full SCHEMA_TYPE_PROTOBUF serde support (Schema Registry and standalone), plus Microsoft Entra (OAuth) authentication for Azure Event Hubs. Fully backward compatible with v2.0.0 scripts.

Highlights

Protobuf Schema Registry serdes (#389, #398)

Protobuf is now fully implemented, finishing the item promised in the v2.0.0 roadmap. SchemaRegistry.serialize() / deserialize() work with SCHEMA_TYPE_PROTOBUF against both the Confluent Schema Registry and standalone (registry-less) flows, using Confluent-compatible framing — without requiring generated Go protobuf types in your project.

import { SchemaRegistry, SCHEMA_TYPE_PROTOBUF } from "k6/x/kafka";

const sr = new SchemaRegistry({ url: "http://localhost:8081" });

const schema = sr.createSchema({
  subject: "my-topic-value",
  schema: protoSource,        // inline .proto text
  schemaType: SCHEMA_TYPE_PROTOBUF,
  // messageName: "com.example.MyMessage", // required when a .proto has >1 top-level message
});

const data = sr.serialize({ topic: "my-topic", value: { id: 1, name: "k6" }, schema, schemaType: SCHEMA_TYPE_PROTOBUF });

• New schema metadata fields: messageName (fully-qualified message, required when a .proto defines multiple top-level messages) and dependencies (import path → raw .proto source, for multi-file standalone resolution).
• New container option protobufFormat: "object" | "bytes" (default "object"): object uses canonical protojson semantics; bytes accepts/returns raw protobuf binary payloads.
• Subject-naming strategies (TopicNameStrategy, RecordNameStrategy, TopicRecordNameStrategy) supported; record-based naming uses messageName.
• Performance: compiled FileDescriptors are now cached by schema content, avoiding recompilation on every serialize/deserialize call in hot loops (#398).

See docs/v2.1.0-protobuf.md for the full design and behavior.

Microsoft Entra (OAuth) authentication (#390)

xk6-kafka can now authenticate to Azure Event Hubs using Microsoft Entra ID (OAuth). The change also introduces a general framework for OAuth providers that issue dynamic credentials, paving the way for AWS MSK and GCP managed Kafka token flows.

What's New

• Protobuf serde for Schema Registry and standalone flows, with object/bytes modes and protojson semantics (#389)
• FileDescriptor caching keyed by schema content for faster repeated protobuf serdes (#398)
• Azure Entra / OAuth authentication for Azure Event Hubs, plus an extensible dynamic-credential provider framework (#390)
• Internal refactor: Go source moved under pkg/kafka for a cleaner package layout (#388)

Fixes

• Docker image: switched the runtime base image to ubuntu:24.04 so the glibc-linked binary runs correctly (#396)
• CI/security: resolved open workflow code-scanning alerts and updated the api-docs toolchain / d.ts validation

Compatibility

• Fully backward compatible with v2.0.0: no metric renames, no config-shape changes.
• v1 constructor aliases (Writer, Reader, Connection) remain available.

Building

# Install xk6
go install go.k6.io/xk6/cmd/xk6@latest

# Build k6 with xk6-kafka v2.1.0
xk6 build --with github.com/mostafa/xk6-kafka/v2@v2.1.0  # Notice the /v2 at the end of the path

CGO must be enabled (CGO_ENABLED=1) and platform build tools are required:

• Linux: apt install pkg-config librdkafka-dev
• macOS: brew install pkg-config librdkafka
• Windows: MSYS2 with mingw-w64-x86_64-gcc

Or use the pre-built Docker image:

docker pull mostafamoradian/xk6-kafka:2.1.0

Release Artifacts

The Docker image is signed with cosign (keyless, OIDC).

Dependencies

• confluent-kafka-go/v2 v2.14.0
• go.k6.io/k6 v1.7.1
• Go 1.26+

What's Next

k6 subcommand support was supposed to be released in v2.2.0 of xk6-kafka, however #393 upgrades xk6-kafka to use k6 v2, which requires moving to xk6-kafka v3. So, the next release, v3.0.0, will include this feature.

Full Changelog: v2.0.0...v2.1.0

xk6-kafka v2.0.0

A major release that migrates the runtime from segmentio/kafka-go to confluentinc/confluent-kafka-go (#387), delivering up to 3.3x higher throughput while preserving backward compatibility with existing scripts.

Highlights

Performance

Up to ~383,000 msgs/sec (unacked, 50 VUs) on the same benchmark (scripts/test_json.js) that reached ~115,600 msgs/sec on v1 (a ~3.3x improvement). The gains come from the Confluent-backed runtime and producer-path optimizations.

New v2 API

New primary constructors with clearer naming:

import { Producer, Consumer, AdminClient } from "k6/x/kafka";

const producer = new Producer({ brokers: ["localhost:9092"], topic: "my-topic" });
const consumer = new Consumer({ brokers: ["localhost:9092"], topic: "my-topic", groupId: "my-group" });
const admin    = new AdminClient({ brokers: ["localhost:9092"] });

The v1 constructors (Writer, Reader, Connection) remain available as deprecated aliases throughout v2.x and route through the same Confluent-backed runtime, so you can migrate incrementally.

Schema Registry

The Schema Registry client has been migrated from srclient to the Confluent Schema Registry client. Avro and JSON Schema are fully supported. Protobuf Schema Registry serdes are planned for v2.1.

What's New

• Confluent runtime: all producer, consumer, and admin operations now run on confluent-kafka-go/v2 (v2.14.0), wrapping librdkafka for native performance
• Producer/Consumer/AdminClient: new primary JS constructors; Writer/Reader/Connection kept as deprecated aliases
• Typed error handling: Xk6KafkaError across all code paths; removed panic-prone constructor paths
• Avro Fixed type: serialization support added (#385)
• ~80% test coverage: table-driven config tests, MockCluster smoke tests, metrics unit tests, race-detector CI
• Versioned API docs: TypeScript definitions at api-docs/v2/index.d.ts, generated reference at api-docs/v2/docs/
• Migration guide: MIGRATION.md covers constructor, method, config, and metric mapping

Compatibility

• Metric names preserved: all kafka_writer_* and kafka_reader_* metrics keep their existing names; dashboards and thresholds continue to work
• Config shapes unchanged: WriterConfig, ReaderConfig, and ConnectionConfig remain the input shapes
• No removed or renamed metrics in v2.0.0

Breaking Changes

• Go module path: now github.com/mostafa/xk6-kafka/v2 (required by Go's v2+ module rules)
• AdminClient.listTopics() returns structured topic metadata objects instead of string[] (the deprecated Connection.listTopics() alias preserves the old string[] shape)
• Custom writer balancer: not supported on the Confluent path; scripts using balancer callbacks should stay on v1 until a replacement is added
• Protobuf serdes: SCHEMA_TYPE_PROTOBUF is exported but not implemented in v2.0.0
• There won't be a Windows ARM64 build anymore.

Building

# Install xk6
go install go.k6.io/xk6/cmd/xk6@latest

# Build k6 with xk6-kafka v2
xk6 build --with github.com/mostafa/xk6-kafka/v2@v2.0.0  # Notice the /v2 at the end of the path

CGO must be enabled (CGO_ENABLED=1) and platform build tools are required:

• Linux: apt install pkg-config librdkafka-dev
• macOS: brew install pkg-config librdkafka
• Windows: MSYS2 with mingw-w64-x86_64-gcc

Or use the pre-built Docker image:

docker pull mostafamoradian/xk6-kafka:2.0.0

Release Artifacts

Dependencies

• confluent-kafka-go/v2 v2.14.0
• go.k6.io/k6 v1.7.1
• Go 1.26+

What's Next

• v2.1: Protobuf Schema Registry serdes
• v2.2: k6 subcommand support

Full Changelog: v1.0.0...v2.0.0

What changed

• Added support for wrapped primitive types inside Avro union fields, with new conversion and serde test coverage.
• Updated toolchain/runtime dependencies, including Go (1.26.1) and k6 (v1.6.1).
• Improved producer partition argument handling (len(partitions) usage).

Documentation and examples

• Refined README and schema-registry docs to reflect current behavior and usage.
• Updated Avro auth/schema-registry example scripts and added a dedicated union/logical-type test script.

Release and CI reliability fixes (included in republished tag)

• Fixed Docker image builds by removing fragile Alpine package version pinning in Dockerfile.
• Updated release workflow so Trivy scan failures no longer block publishing release artifacts; SARIF upload runs when a report is present.

Notes

This v1.3.0 tag was republished to include release-pipeline fixes required for successful artifact/image publication.

This release brings an updated k6 baseline (to v1.4.2), a new Avro implementation (based on hamba/avro), better precision and resiliency around time handling, balancer functions in JS (thanks to @yonesko), plus a handful of quality-of-life and security linting fixes.

This marks full migration from linkedin/goavro to hamba/avro plus resolvers for auto-resolving nested and referenced schemas and proper handling of Avro union types.

Please test and provide feedback by reporting issues. 🙏

What's Changed

• Migrate to hamba/avro by @mostafa in #346
• Update k6 to v1.3.0 and update deps by @mostafa in #359
• Balancer func plus other fixes by @yonesko in #361
• Update deps by @mostafa in #369
• Fix zizmor reported issues by @mostafa in #370
• Resolvers and all the fun by @mostafa in #371
• Fix subsecond precision by @mostafa in #372
• Gzip support in HTTP client for srclient by @mostafa in #373
• Fix deps and update Alpine image version by @mostafa in df6f73f

New Contributors

• @yonesko made their first contribution in #361

Full Changelog: v1.1.0...v1.2.0

xk6-kafka now ships with k6 v1.2.3. 🚀

What's Changed

• docs(usage) : Add documentation for managing readers and writers with xk6-kafka by @lewandowski-anthony in #351
• Update deps by @mostafa in #357
• Update Alpine deps in Dockerfile by @mostafa in 911dee5

New Contributors

• @lewandowski-anthony made their first contribution in #351

Full Changelog: v1.0.0...v1.1.0

xk6-kafka v1.0.0 is here! 🎉

After nearly 5 years of continuous development, from the initial commit on May 9, 2020, to today's milestone, xk6-kafka v1.0.0 represents our commitment to stability, clarity, and long-term support for everyone load-testing Kafka with k6. In that time, this extension has seen 277 commits, amassed 176 stars and 81 forks on GitHub, and benefitted from 21+ community contributors, with 72,000+ Docker pulls and 9,000+ binary downloads. With v1.0.0, we're formalizing versioning guarantees, polishing our public API, and delivering a rock-solid foundation you can build on with confidence.

Thank you, xk6-kafka community! 🌍

This release wouldn't be possible without you:

• ⭐️ 176 stars on GitHub.
• 🍴 81 forks to explore and extend.
• 🧠 277 commits driving the project forward.
• 🤝 21+ contributors shaping features and fixes.
• 📦 72,000+ Docker pulls of mostafamoradian/xk6-kafka.
• 📥 9,000+ GitHub artifact downloads for prebuilt binaries.

From enhancing serialization formats and authentication to expanding metrics coverage and improving developer ergonomics, your issues, PRs, and feedback have been invaluable. Thank you for every test run, discussion, and contribution that brought us here.

What's New in xk6-kafka v1.0.0?

1. Stability You Can Trust

• ✅ Semantic Versioning: We've adopted SemVer 2.0.0, breaking changes only in major releases, with deprecation warnings in advance.
• 🔒 2-Year Support Guarantee: Critical fixes and security patches for all v1.x releases, for at least two years.
• 📦 Public API Surface: A clearly documented, supported set of functions (Writer, Reader, Connection, SchemaRegistry) and options you can rely on.

2. First-Class TypeScript Definitions

Leverage TypeScript's type-safety when writing Kafka tests:

import { Writer, Reader, SchemaRegistry, SCHEMA_TYPE_AVRO } from 'k6/x/kafka';

const writer = new Writer({
  brokers: ['localhost:9092'],
  topic: 'orders',
  keySchema: { type: SCHEMA_TYPE_AVRO, schema: orderKeySchema },
  valueSchema: { type: SCHEMA_TYPE_AVRO, schema: orderValueSchema },
});

export default function () {
  writer.produce([{ key: '123', value: orderPayload }]);
}

3. Streamlined Extension Workflow

No more toolchain quirks, just build and run:

xk6 build --with github.com/mostafa/xk6-kafka@v1.0.0
k6 run script.js

Or use the official Docker image:

docker pull mostafamoradian/xk6-kafka:v1.0.0
docker run --rm -v "$(pwd)":/scripts \
  mostafamoradian/xk6-kafka:v1.0.0 \
  run /scripts/test_avro.js

4. Enhanced Metrics & Monitoring

• 📊 New consumer metrics: Track kafka_consumer_lag, kafka_consumer_bytes, and more.
• ⚖️ Improved thresholds: Apply thresholds on Kafka-specific metrics for automated pass/fail criteria.
• 🔍 Hierarchical summaries: Group results by topic, partition, and metric category in the end-of-test report.

5. Quality-of-Life Upgrades

• ✔️ Full Schema Registry support: Auto-fetch Avro/JSON Schema from your registry or embed inline.
• 🔐 Expanded auth options: SASL/PLAIN, SCRAM, SSL, and AWS IAM are all first-class citizens.
• ⚡ Performance optimizations: Bulk-produce mode and connection pooling for high-throughput scenarios.

Ready to get started? Download the binaries or grab the Docker image, and let v1.0.0 supercharge your Kafka load tests. If you encounter any issues or have feature ideas, we'd love to hear from you on GitHub!

What's Changed

• Add option to set AWS profile in SASL config (as requested in #337) by @mostafa in #338
• Update Trivy Action in b4d9447
• Update deps to fix CVE-2025-22870 in 64bb0e6

Full Changelog: v0.31.2...v0.31.5

What's Changed

• bump-crypto-to-v0.35.0-CVE-2025-22869 by @kempsterc in #336

New Contributors

• @kempsterc made their first contribution in #336

Full Changelog: v0.31.1...v0.31.2

What's Changed

• Update k6 to v0.57.0
• Update Go to v1.24
• Update deps
• Adding docker build steps to README. by @beachwood23 in #326

New Contributors

• @beachwood23 made their first contribution in #326

Full Changelog: v0.30.0...v0.31.1

Full Changelog: v0.29.0...v0.30.0