Golang concurrent function (goroutine) runner with quota

A concurrent function runner with quota on how many functions can be executing at the same time.

Features

A concurrent runner purpose is to enforce maximum number of goroutines that can execute simultaneously (quota). When the quota is reached scheduling new function executions is blocked until some of the running functions are finished.

It also maintains an atomic counter of how many functions are executing at any point of time.

Implementation

There are two flavors of concurrent runners are implemented. One that uses semaphore synchronization primitive and the other uses channels.

Both have common functionality described by the interface:

type Runner interface {
   // Concurrently executes a function wrapped in a goroutine.
   Run(task func()) error

   // Waits for all running functions to complete and frees resources.
   WaitAndClose() error

   // Returns the number of currently executing functions.
   GetNumberOfRunningTasks() int

   // Returns the quota limit
   GetQuota() int
}

Usage

Get the package

go get github.com/paulshpilsher/concurrent-go

In code import runner.

To use channel-based runner:

import "github.com/paulshpilsher/concurrent-go/concurrency/chan/runner"

To use sync-based runner:

import "github.com/paulshpilsher/concurrent-go/concurrency/sync/runner"

Use the runner

 theRunner := runner.New(quota)
 if err != nil {
    panic(err)
 }
 
 for i := 0; i < 1000; i++ {
  theRunner.Run(func() {
   // put some code to be exectuted
  })
 }

 theRunner.WaitAndClose()

Examples

The exapmples are in the ./examples/ directory.

Running examples using make utility:

make run-example-channel

or

make run-example-sync

Testing

Running unit tests using make utility:

make test

Benchmarks:

make bench

Acknowledgements

• Inspiration "Simple Made Easy" - Rich Hickey (2011)
• References