Skip to main content
A line filter is a common type of program that reads input from stdin, processes it line by line, and prints results to stdout. Tools like grep and sed are classic examples of line filters.

​
What is a Line Filter?

Line filters are programs that:
  • Read input from standard input (stdin)
  • Process data line by line
  • Write output to standard output (stdout)
  • Can be chained together using Unix pipes
Line filters are fundamental building blocks in Unix/Linux command-line processing and can be combined using pipes to create powerful data processing pipelines.

​
Basic Line Filter Example

Here’s a simple line filter that converts input text to uppercase: 
package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"
)

func main() {
	// Create buffered scanner for stdin
	scanner := bufio.NewScanner(os.Stdin)

	for scanner.Scan() {
		// Get current line
		ucl := strings.ToUpper(scanner.Text())
		
		// Write uppercased line to stdout
		fmt.Println(ucl)
	}

	// Check for scanning errors
	if err := scanner.Err(); err != nil {
		fmt.Fprintln(os.Stderr, "error:", err)
		os.Exit(1)
	}
}

​
How It Works

​
Buffered Scanner

scanner := bufio.NewScanner(os.Stdin)
Wrapping os.Stdin with a buffered scanner provides a convenient Scan method that advances to the next token (line by default).

​
Reading Lines

for scanner.Scan() {
	// scanner.Text() returns the current line
	ucl := strings.ToUpper(scanner.Text())
	fmt.Println(ucl)
}
The Scan() method:
  • Returns true when a line is available
  • Returns false at end of file or on error
  • Automatically handles line ending characters

​
Error Handling

if err := scanner.Err(); err != nil {
	fmt.Fprintln(os.Stderr, "error:", err)
	os.Exit(1)
}
Always check scanner.Err() after the scan loop. End of file is expected and not reported as an error, but other errors should be handled.

​
Usage Examples

​
Direct Input

echo "hello world" | ./line-filter
# Output: HELLO WORLD

​
File Input

cat input.txt | ./line-filter

​
Pipeline Processing

cat log.txt | ./line-filter | grep "ERROR" | sort

​
Advanced Line Filter Patterns

scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
    line := scanner.Text()
    // Only output lines containing "error"
    if strings.Contains(strings.ToLower(line), "error") {
        fmt.Println(line)
    }
}

​
Custom Split Functions

You can customize how the scanner splits input: 
import "bytes"

scanner := bufio.NewScanner(os.Stdin)

// Split on custom delimiter
scanner.Split(bufio.ScanWords) // Split on words instead of lines

for scanner.Scan() {
	word := scanner.Text()
	fmt.Println(strings.ToUpper(word))
}
Available split functions:
  • bufio.ScanLines (default) - split on newlines
  • bufio.ScanWords - split on whitespace
  • bufio.ScanRunes - split into individual runes
  • bufio.ScanBytes - split into individual bytes

​
Real-World Use Cases

Log Processing

Filter and format log files, extract error messages, or highlight specific patterns

Data Transformation

Convert data formats, normalize text, or apply transformations to each line

Text Analysis

Count words, analyze patterns, or extract statistics from text streams

Pipeline Integration

Integrate with other command-line tools in Unix pipelines

​
Best Practices

Always use bufio.Scanner for reading line by line. It’s more efficient than reading character by character.
Check scanner.Err() after the scan loop to catch any errors that occurred during reading.
Use fmt.Fprintln(os.Stderr, ...) for error messages so they don’t interfere with stdout output.
Use os.Exit(1) for errors and os.Exit(0) (or return from main) for success.
Each filter should do one thing well. Combine multiple simple filters for complex operations.
For very large files, consider processing in chunks or using goroutines for parallel processing.

​
Testing Line Filters

package main

import (
	"bytes"
	"strings"
	"testing"
)

func TestLineFilter(t *testing.T) {
	input := "hello\nworld"
	expected := "HELLO\nWORLD\n"
	
	// Create reader from string
	reader := strings.NewReader(input)
	
	// Capture output
	var output bytes.Buffer
	
	// Process (pseudo-code - adapt to your filter logic)
	scanner := bufio.NewScanner(reader)
	for scanner.Scan() {
		output.WriteString(strings.ToUpper(scanner.Text()) + "\n")
	}
	
	if output.String() != expected {
		t.Errorf("Expected %q, got %q", expected, output.String())
	}
}

Reading Files

Learn more about file reading in Go

Writing Files

Learn how to write data to files

Build docs developers (and LLMs) love

Get started for freeTalk to us