Fan-In & Fan-Out Messaging Patterns
Jan 2, 2016
2 minutes read

Fan-In Pattern

Fan-In is a messaging pattern used to create a funnel for work amongst workers (clients: source, server: destination).

We can model these messaging patterns using the Go channels.

// Merge different channels in one channel
func Merge(cs ...<-chan int) <-chan int {
	var wg sync.WaitGroup

	out := make(chan int)

	// Start an send goroutine for each input channel in cs. send
	// copies values from c to out until c is closed, then calls wg.Done.
	send := func(c <-chan int) {
		for n := range c {
			out <- n
		}
		wg.Done()
	}

	wg.Add(len(cs))
	for _, c := range cs {
		go send(c)
	}

	// Start a goroutine to close out once all the send goroutines are
	// done.  This must start after the wg.Add call.
	go func() {
		wg.Wait()
		close(out)
	}()
	return out
}

The Merge function converts a list of channels to a single channel by starting a goroutine for each inbound channel that copies the values to the sole outbound channel.

Once all the output goroutines have been started, Merge a goroutine is started to close the main channel.

Fan-Out Pattern

Similarly, Fan-Out is used for distributing work amongst workers (producer: source, consumers: destination)

// Split a channel into n channels that receive messages in a round-robin fashion.
func Split(ch <-chan int, n int) []<-chan int {
	cs := make([]chan int)
	for i := 0; i < n; i++ {
		cs = append(cs, make(chan int))
	}

	// Distributes the work in a round robin fashion among the stated number
	// of channels until the main channel has been closed. In that case, close
	// all channels and return.
	distributeToChannels := func(ch <-chan int, cs []chan<- int) {
		// Close every channel when the execution ends.
		defer func(cs []chan<- int) {
			for _, c := range cs {
				close(c)
			}
		}(cs)

		for {
			for _, c := range cs {
				select {
				case val, ok := <-ch:
					if !ok {
						return
					}

					c <- val
				}
			}
		}
	}

	go distributeToChannels(ch, cs)

	return cs
}

The Split function converts a single channel into a list of channels by using a goroutine to copy received values to channels in the list in a round-robin fashion.

Go’s Forte

Go’s built-in communication model (i.e. chan, select, range) makes working with communication patterns surprisingly easy and pleasant.

These patterns are very helpful in service-oriented/microservice architectures.


Back to posts