Saturday, June 11, 2011

Channels in Go - range and select

Channels and range

This is the second part of tutorial on channels in Go. If you haven’t yet gone through the first part, Channels in Go, please go through it now.

Receivers of data have a problem of knowing when to stop waiting for data. Is there more to come or is it all done? Should we wait or should we move on? One option is to constantly poll the source and check if the channel is closed or not, but that isn’t very efficient. Go provides the range keyword which when used with a channel will wait on the channel until it is closed.

Full code
package main                                                                                             
import (
    "fmt"
    "time"
    "strconv"
)

func makeCakeAndSend(cs chan string, count int) {
    for i := 1; i <= count; i++ {
        cakeName := "Strawberry Cake " + strconv.Itoa(i)
        cs <- cakeName //send a strawberry cake
    }   
}

func receiveCakeAndPack(cs chan string) {
    for s := range cs {
        fmt.Println("Packing received cake: ", s)
    }
}

func main() {
    cs := make(chan string)
    go makeCakeAndSend(cs, 5)
    go receiveCakeAndPack(cs)

    //sleep for a while so that the program doesn’t exit immediately
    time.Sleep(3 * 1e9)
}

Packing received cake: Strawberry Cake 1
Packing received cake: Strawberry Cake 2
Packing received cake: Strawberry Cake 3
Packing received cake: Strawberry Cake 4
Packing received cake: Strawberry Cake 5

To the cake maker we have indicated that we want 5 cakes made, but the receiver does not know that. In earlier versions of the code, we hard coded the exact number for the receipt also. But by using the range keyword on the channel in the code above, we avoid that need - now when the channel is closed, the for loop automatically stops.

Channels and select

The select keyword when used in conjunction with many channels works like a are-you-ready polling mechanism across different channels. The case blocks within it can be for sending or receiving - when either a send or a receive is initiated with the <- operator, then that channel is ready. There can also be a default case block, which is always ready. The algorithm with which the select keyword works on blocks can be approximated to this:
* check each of the case blocks
* if any one of them is sending or receiving, execute the code block corresponding to it
* if more than one is sending or receiving, randomly pick any of them and execute its code block
* if none of them are ready, wait
* if there is a default block, and none of the other case blocks are ready, then execute the default block (I’m not very sure about this, but from coding experiments, it appears that default gets last priority)

In the program below, we have extended our cake making factory to simulate the production of more than 1 flavor of cake - both strawberry and chocolate now! But the packing mechanism is the same. Also it takes more time to make a cake than pack one, which means that we can gain efficiency from packing different types of cake with the same packing machine. Since cakes come from different channels and the packer cannot know the exact moment when a cake is placed on either or many channels, it can use the select statement and wait across all of them - as soon as one of the channels is receiving a cake/data, it will implement its code block.

Note also how we have used multiple return values case cakeName, strbry_ok := <-strbry_cs:. The second return value is a bool which when false indicates that the channel is closed. If it is true, it indicates that a value was transferred. I have used it to check whether I can stop waiting for all cakes.

Full code
package main

import (
    "fmt"
    "time"
    "strconv"
)

func makeCakeAndSend(cs chan string, flavor string, count int) {
    for i := 1; i <= count; i++ {
        cakeName := flavor + " Cake " + strconv.Itoa(i)
        cs <- cakeName //send a strawberry cake
    }   
    close(cs)
}

func receiveCakeAndPack(strbry_cs chan string, choco_cs chan string) {
    strbry_closed, choco_closed := false, false

    for {
        //if both channels are closed then we can stop
        if (strbry_closed && choco_closed) { return }
        fmt.Println("Waiting for a new cake ...")
        select {
        case cakeName, strbry_ok := <-strbry_cs:
            if (!strbry_ok) {
                strbry_closed = true
                fmt.Println(" ... Strawberry channel closed!")
            } else {
                fmt.Println("Received from Strawberry channel.  Now packing", cakeName)
            }   
        case cakeName, choco_ok := <-choco_cs:
            if (!choco_ok) {
                choco_closed = true
                fmt.Println(" ... Chocolate channel closed!")
            } else {
                fmt.Println("Received from Chocolate channel.  Now packing", cakeName)
            }   
        }   
    }   
}

func main() {
    strbry_cs := make(chan string)
    choco_cs := make(chan string)

    //two cake makers
    go makeCakeAndSend(choco_cs, "Chocolate", 3)  //make 3 chocolate cakes and send
    go makeCakeAndSend(strbry_cs, "Strawberry", 3)  //make 3 strawberry cakes and send

    //one cake receiver and packer
    go receiveCakeAndPack(strbry_cs, choco_cs)  //pack all cakes received on these cake channels

    //sleep for a while so that the program doesn’t exit immediately
    time.Sleep(2 * 1e9)
}
Waiting for a new cake ...
Received from Strawberry channel. Now packing Strawberry Cake 1
Waiting for a new cake ...
Received from Chocolate channel. Now packing Chocolate Cake 1
Waiting for a new cake ...
Received from Chocolate channel. Now packing Chocolate Cake 2
Waiting for a new cake ...
Received from Strawberry channel. Now packing Strawberry Cake 2
Waiting for a new cake ...
Received from Strawberry channel. Now packing Strawberry Cake 3
Waiting for a new cake ...
Received from Chocolate channel. Now packing Chocolate Cake 3
Waiting for a new cake ...
... Strawberry channel closed!
Waiting for a new cake ...
... Chocolate channel closed!

13 comments:

  1. It looks like a channel can only be used by one goroutine to send. Can a goroutine leave the channel open for another goroutine to send?

    ReplyDelete
    Replies
    1. There is no such coupling between go routines and channels. If you have a reference to a channel, it is open, and it's type allows it, you can send or receive on it at whim from wherever.

      I mention type because you can limit how a channel can be used with the type system, but it has nothing to do with go routines per se.

      Delete
  2. Great stuff!

    If I'm not mistaken, doesn't your first example forget to close the channel, in order to trigger the range statement?

    ReplyDelete
    Replies
    1. Tested with and without close - both works. Does the garbage collector closes channel automatically?

      Delete
    2. I found this not to be true, and I am recording it here. What happens is that the range statement never finishes, but the program exits once the main thread exits, so it does not matter that the range statement never finishes.

      The garbage collector cannot collect the channel since there is still a goroutine waiting on it, and it cannot know that none of the references held to the channel will ever write to it again, so that cannot be the answer.

      The proper way to write a program like this is to:
      a) have the sender close the channel once the sender has stopped sending, eg by adding a close(cs) to makeCakeAndSend
      b) have the program terminate once the receiver has stopped receiving. This can be done just by removing the "go" from "go receiveCakeAndPack".
      c) Remove the time.Sleep.

      Delete
    3. @Andrew's solution is the correct implementation

      Delete
  3. My Solution:

    func receiveCakeAndPack(strbry_cs chan string, choco_cs chan string) {
    strbry_open, choco_open := true, true
    for cakeName := ""; strbry_open || choco_open; {
    fmt.Println("Waiting for a new cake ...")
    if cakeName, strbry_open = <- strbry_cs; strbry_open {
    fmt.Println("Received from Strawberry channel. Now packing", cakeName)
    }
    if cakeName, choco_open = <- choco_cs; choco_open {
    fmt.Println("Received from Chocolate channel. Now packing", cakeName)
    }
    }
    fmt.Println("Both channels closed!")
    }

    ReplyDelete
    Replies
    1. That works, but the semantics have changed. Your program makes the packer alternate between strawberry and chocolate evenly.

      When using the `select` statement, the packer can receive strawberry or chocolate cakes in any order.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Thanks a lot for these tutorials. This is hands down the best available resource for newcomers to the language. Well done!

    ReplyDelete
  6. When using buffered channels, closing is not the right solution ; every one in the chain must be drained first. Recently, I solved this problem by passing an EOF-like token. Each channel in my chain detect this and passes this to the next. The consumer and the end of the chain now knows when to terminate.

    ReplyDelete
    Replies
    1. The code in the second example works almost as-is (after changing the sleep at the end to "time.Sleep( 2 * time.Second )".

      However, it's making the same number of both types of cake. If you try to change the code to make (eg) 3 strawberry and 8 chocolate cakes, the program never terminates.

      Ernest's EOF-like token (send a cake named "END" when you're finished, and have the packer code look for that) instead of closing the channel worked.

      Delete
  7. Please remove the thread sleep and update as Andrew recommended. This page turns up at the top of google search result and this code has errors that should not be using for teaching channels.

    ReplyDelete

If you think others also will find these tutorials useful, kindly "+1" it above and mention the link in your own blogs, responses, and entries on the net so that others also may reach here. Thank you.

Note: Only a member of this blog may post a comment.