Real iter example

Mike

2 min read

·

Jul 31, 2026

--

Listen

Share

Some time ago, I wrote a post about the `iter` package.

All my examples were related to slices, without a real application in production code.

Let’s fix it!

Do you use Kafka or another message broker? Almost everyone does now.

So, let’s look at an example from confluent-lib:

consumer, err := kafka.NewConsumer(ToConsumerConfig(cfg, "earliest"))
if err != nil {
    log.Fatalf("Failed to create consumer: %v", err)
}

defer consumer.Close()

err = consumer.SubscribeTopics([]string{topic}, nil)
if err != nil {
    log.Fatalf("Failed to subscribe to topics: %v", err)
}

for {
    msg, err := consumer.ReadMessage(1 * time.Second)
    if err == nil {
        fmt.Println("Received message: ", string(msg.Value))
    } else if !err.(kafka.Error).IsTimeout() { 
        log.Fatalf("Consumer error: %v", err)
    }
}

What do we see here? Standard setup around the topic and then an infinite loop

(in real systems, of course, it is usually controlled by shutdown logic).

If we transform this example into an iter-style version:

for msg, readErr := range consumeMessagesIter(consumer, 1*time.Second) {
  if readErr != nil {
   log.Fatalf("Consumer error: %v", readErr)
  }
  fmt.Println("Received message: ", string(msg.Value))
 }


func consumeMessagesIter(consumer *kafka.Consumer, timeout time.Duration) iter.Seq2[*kafka.Message, error] {
  return func(yield func(*kafka.Message, error) bool) {
   for {
     msg, err := consumer.ReadMessage(timeout)
     if err != nil {
       var kafkaErr kafka.Error
       if errors.As(err, &kafkaErr) && kafkaErr.IsTimeout() {
         continue
       }
       if !yield(nil, err) {
         return
       }
       return
     }

     if !yield(msg, nil) {
      return
     }
    }
  }
}

What is the difference? In this Kafka case, there is no memory win from iter, because the original version is already streaming and does not build large intermediate slices either.

So one classic iter advantage does not apply here. Still, iter helps with composition, reuse, and cleaner separation between reading and processing.

For example, composition here means we can build a pipeline like:

`consumeMessagesIter -> parse JSON -> filter by event type -> handle business logic`

Each step can be a small reusable function, and we can chain them without rewriting the Kafka read loop every time.

Full example here.