Skip to content

More dynamic way of defining topics for KafkaConsumer #693

Description

@tandev

Description
In a Symfony-Application using ecotone with Usage of Kafka its possible to
Use the KafkaConsumer-Attribute to define consuming from multiple topics in the following ways:

Using env with Symfony EnvVarProcessor casting to array. Requires excplicit configuration of topics

KAFKA_CONSUMER_ORDER_TOPICS=orders_eu,orders_us
    #[KafkaConsumer(
        endpointId: 'orders',
        topics: '%env(csv:KAFKA_CONSUMER_ORDER_TOPICS)%',
    )]

Using Wildcard. Using with with Tenant-Resolver might make the Resolving fail if there is a topic for a tenant not yet configured.

    #[KafkaConsumer(
        endpointId: 'orders',
        topics: 'orders_.*',
    )]

Using array in code. Requires excplicit configuration of topics

    #[KafkaConsumer(
        endpointId: 'orders',
        topics: ['orders_eu', 'orders_us'],
    )]

For Multi-Tenant-Applications, there might be need for a more dynamic possibility

Example

Using Symfony-Parameters, which could be set in an DI-Extension based on all available tenants

    #[KafkaConsumer(
        endpointId: 'orders',
        topics: '%app.multi_tenancy.kafka.orders.topics%',
    )]

Using Expression:

    #[KafkaConsumer(
        endpointId: 'orders',
        topics: new Expression("reference('config').getOrderTopics()")
    )]

Alternatively it could be an option to allow defining a KafkaConsumer outside of using the Attribute like in EcotoneConfiguration which might reference a MessageConsumer

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions