Reading Command Output Line by Line

November 12, 2019
golang | pattern

Sometimes, you want to execute a system command from within a Go app and process it's output line-by-line in a streaming fashion.

We of course want to avoid that we need to buffer all the output, wait for the command to finish and then process each line.

We also want to check in the end if the command ran succesfully or not and we also want to capture both standard out and standard error.

Well, here's an example that shows exactly how to do this:

package main

import (
	"bufio"
	"os/exec"

	"github.com/pieterclaerhout/go-log"
)

func main() {

	// Print the log timestamps
	log.PrintTimestamp = true

	// The command you want to run along with the argument
	cmd := exec.Command("brew", "info", "golang")

	// Get a pipe to read from standard out
	r, _ := cmd.StdoutPipe()

	// Use the same pipe for standard error
	cmd.Stderr = cmd.Stdout

	// Make a new channel which will be used to ensure we get all output
	done := make(chan struct{})

	// Create a scanner which scans r in a line-by-line fashion
	scanner := bufio.NewScanner(r)

	// Use the scanner to scan the output line by line and log it
	// It's running in a goroutine so that it doesn't block
	go func() {

		// Read line by line and process it
		for scanner.Scan() {
			line := scanner.Text()
			log.Info(line)
		}

		// We're all done, unblock the channel
		done <- struct{}{}

	}()

	// Start the command and check for errors
	err := cmd.Start()
	log.CheckError(err)

	// Wait for all output to be processed
	<-done

	// Wait for the command to finish
	err = cmd.Wait()
	log.CheckError(err)

}

The full source code can be found here.