Atlas Stream Processing Changelog

Supports VPC Peering with Confluent on AWS.

Supports using the Atlas Administration API to edit stream processors.

Supports AWS PrivateLink with Confluent.

Supports Azure Private Link with EventHubs.

Increases Apache Kafka Timeout from 10 seconds to 30 seconds.

Always retries processors when the target cluster is in the REPAIRING state.

Fixes the partitionIdleTimeout after the checkpoint restore.

Adds more watermark and window information to verbose stats.

Ensures calls to produce messages to Apache Kafka don't hang indefinitely, even when the producer queue is full.

Supports ingesting multitopic Apache Kafka sources.

Updates Apache Kafka offsets for consumer groups to near-real time.

Supports the compression.type and acks settings in the Apache Kafka $emit configuration.

Supports collectionless $lookup syntax.

Standardizes the way Atlas Stream Processing serializes Apache Kafka errors into a string.

Increases the $emit and $merge dynamic expression limit to 1000.

Handles $lookup with unique connectionName inside window pipeline.

Improves paused cluster errors.

Supports converting from Binary data to various data types using $convert. This allows you to transform Kafka headers into specific data types as needed.

Stream processors that use Apache Kafka as a source expose per-partition watermarks in sp.processor.stats().

Fixes a bug where users were unable to create stream processing instances in certain regions through the API.