Reading Command Output Line by Line

November 12, 2019
golang | pattern

Sometimes, you want to execute a system command from within a Go app and process it’s output line-by-line in a streaming fashion.

We of course want to avoid that we need to buffer all the output, wait for the command to finish and then process each line.

We also want to check in the end if the command ran succesfully or not and we also want to capture both standard out and standard error.

Well, here’s an example that shows exactly how to do this:

package main

import (
    "bufio"
    "context"
    "os/exec"

    "github.com/pieterclaerhout/go-log"
    "golang.org/x/sync/errgroup"
)

func main() {

    // Print the log timestamps
    log.PrintTimestamp = true

    // The command you want to run along with the argument
    cmd := exec.Command("brew", "info", "golang")

    // Get a pipe to read from standard out
    r, _ := cmd.StdoutPipe()

    // Use the same pipe for standard error
    cmd.Stderr = cmd.Stdout

    // Create a scanner which scans r in a line-by-line fashion
    scanner := bufio.NewScanner(r)

    // Start the command and check for errors
    err := cmd.Start()
    log.CheckError(err)

    // Start a new error group, run the command and wait for it to finish
    // We are running this in a go routine so that we don't block
    errs, _ := errgroup.WithContext(context.Background())
    errs.Go(func() error {
        return cmd.Wait()
    })

    // Use the scanner to scan the output line by line and log ti
    for scanner.Scan() {
        line := scanner.Text()
        log.Info(line)
    }

    // Wait for the command to finish and check for errors
    err = errs.Wait()
    log.CheckError(err)

}

The full source code can be found here.