feat/internal/memcmd: add internal/memcmd package to allow for memory tracking of exec.Cmd processes

[ggilmore]

This PR adds a new package memcmd, that adds a new abstraction called "Observer" that allows you to track the memory that a command (and all of its children) is using. (This package uses a polling approach with procfs, since maxRSS on Linux is otherwise unreliable for our purposes).

Example usage

import (
	"context"
	"fmt"
	"os/exec"
	"time"

	"github.com/sourcegraph/sourcegraph/internal/memcmd"
)

func Example() {
	const template = `
#!/usr/bin/env bash
set -euo pipefail

word=$(head -c "$((10 * 1024 * 1024))" </dev/zero | tr '\0' '\141') # 10MB worth of 'a's
sleep 1
echo ${#word}
`

	cmd := exec.Command("bash", "-c", template)
	err := cmd.Start()
	if err != nil {
		panic(err)
	}

	observer, err := memcmd.NewLinuxObserver(context.Background(), cmd, 1*time.Millisecond)
	if err != nil {
		panic(err)
	}

	observer.Start()
	defer observer.Stop()

	err = cmd.Wait()
	if err != nil {
		panic(err)
	}

	memoryUsage, err := observer.MaxMemoryUsage()
	if err != nil {
		panic(err)
	}

	fmt.Println((0 < memoryUsage && memoryUsage < 50*1024*1024)) // Output should be between 0 and 50MB

	// Output:
	// true
}

Test plan

Unit tests

Note that some tests only work on darwin, so you'll have to run those locally.

Changelog

This feature adds a package that allows us to track the memory usage of commands invoked via exec.Cmd.

[ggilmore]

• feat/cmd/gitserver: add memory tracking for both linux and macos #63114
• feat/internal/memcmd: add internal/memcmd package to allow for memory tracking of exec.Cmd processes #62803 👈
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @ggilmore and the rest of your teammates on Graphite

[eseliger]

LGTM pending the bazel question from Noah and an approving review from the dev-infra team who own bazel

[graphite-app]

[ggilmore]

Merge activity

• Jun 10, 5:20 PM EDT: @ggilmore merged this pull request with Graphite.