Amazon recently announced that AWS Lambda supports content filtering options for Amazon MSK, Self-Managed Kafka, Amazon MQ for Apache ActiveMQ, and Amazon MQ for RabbitMQ as event sources. The new options extend the filtering to data store and broker services and reduce traffic to Lambda functions, simplifying application logic and reducing costs.

Event filtering limits function invocations for microservices that only use a subset of events available, removing the need for the target Lambda function or downstream applications to perform filtering. Jeremy Daly, author of the weekly serverless newsletter Off-by-none, writes:

Filtering was already available for SQS, DynamoDB, and Kinesis, so this seems to round out currently supported event source mappings for Lambda. Other event triggers work differently (generally attached to the publishing service), but it’ll be interesting to see if they make a move to consolidate the underlying semantics to make everything an event source mapping.

For Kafka, the serverless platform commits offsets for matched and unmatched messages after successfully invoking the function. For Amazon MQ, it acknowledges matched messages after successfully invoking the function and unmatched messages when filtering them. While for Amazon MQ sources the message field is data, for Kafka sources, both managed and not, there are two message fields, key and value.

Both Kafka and Amazon MQ messages must be UTF-8 encoded strings, either plain strings or in JSON format. For filter rules, Lambda supports the same syntax as EventBridge.

{
    "data": {
        "Temperature": [ {"numeric": [ ">", 50, "<=", 60 ] } ]
    }
}

Source: https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html

When a function is triggered by an event source that supports filtering, developers can define up to five filter criteria when creating or updating the event source mappings. The filters are combined using OR logic: only an event matching at least one of the filtering criteria will trigger a Lambda function. Randall Hunt, VP cloud at Caylent, highlights the pricing benefits:

"There is no additional cost for using this feature beyond the standard price for AWS Lambda." A huge potential price drop with no code changes, just some config changes.

Julian Wood, senior serverless developer advocate at AWS, agrees:

This is a huge additional feature. Lambda running event/stream pollers for you is just a massive productivity and simplicity boost, serverless consumers FTW! And a simple cost reducer. Don’t invoke needlessly!

According to the documentation, it can take up to 15 minutes to apply the filtering rules to events after a filter criteria is attached to a Kafka or Amazon MQ event source mapping.