Skip to content

Bug: Kafka consumer fails esbuild bundling unless unused Avro and Protobuf dependencies are installed #5557

Description

@Malanius

Expected Behavior

A primitive-only consumer should bundle with only @aws-lambda-powertools/kafka installed. Format-specific dependencies and implementation code should only be required when Avro or Protobuf is configured.

Current Behavior

Using @aws-lambda-powertools/kafka@2.34.0 with primitive deserialization fails during esbuild bundling when avro-js and protobufjs are not installed.

The consumer dynamically imports all supported deserializers using literal paths. Esbuild resolves these modules during bundling, even though the handler does not configure Avro or Protobuf:

Could not resolve "protobufjs"

@aws-lambda-powertools/kafka/lib/esm/deserializer/protobuf.js:
import { BufferReader } from 'protobufjs';

Could not resolve "avro-js"

@aws-lambda-powertools/kafka/lib/esm/deserializer/avro.js:
import avro from 'avro-js';

The documentation indicates that these libraries are required only when their respective formats are used. They are also absent from the Kafka package’s dependencies and optional peer dependencies.

Installing both packages allows bundling, but esbuild includes both unused codec implementations in the single-file Lambda bundle because they remain runtime-selectable.

Occurs in invoking esbuild directly (see below) and CDK NodeJsFunctio bundling.

Code snippet

import { kafkaConsumer } from '@aws-lambda-powertools/kafka';

export const handler = kafkaConsumer(async (event) => {
  for (const record of event.records) {
    console.log(record.value);
  }
});

Steps to Reproduce

mkdir powertools-kafka-repro
cd powertools-kafka-repro
npm init -y
npm install @aws-lambda-powertools/kafka@2.34.0
npm install --save-dev esbuild

Create index.ts:

import { kafkaConsumer } from '@aws-lambda-powertools/kafka';

export const handler = kafkaConsumer(async (event) => {
  for (const record of event.records) {
    console.log(record.value);
  }
});

Bundle it:

npx esbuild index.ts \
  --bundle \
  --platform=node \
  --target=node24 \
  --format=esm \
  --outfile=dist/index.mjs

Possible Solution

No response

Powertools for AWS Lambda (TypeScript) version

latest

AWS Lambda function runtime

24.x

Packaging format used

npm

Execution logs

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions