? Python subprocess Module

⚡ Quick Overview

The subprocess module lets your Python program start new processes, connect to their input/output/error pipes, and obtain their return codes. It is the recommended way to run external commands and replace older modules like os.system() and os.popen().

Common use cases include automating command-line tools, running shell scripts, capturing command output, and integrating system utilities into Python applications.

Import it like this: ✅ import subprocess

? Key Concepts

• subprocess.run() – High-level API to run a command and wait for it to finish.
• Popen class – Low-level interface for advanced control over processes.
• CompletedProcess – Object returned by run() containing output and status.
• Standard streams – stdin, stdout, and stderr pipes for communicating with the process.
• check=True – Raises an exception if the command exits with a non-zero status.
• capture_output=True / stdout & stderr – Capture the command’s output.
• text=True / encoding=... – Get output as strings instead of bytes.
• shell=True – Runs the command through the shell (use with care for security).

? Syntax and Core APIs

? Using subprocess.run()

Basic pattern:

import subprocess

# General pattern
result = subprocess.run(
    ["command", "arg1", "arg2"],  # list of program + arguments
    capture_output=False,         # True to capture stdout/stderr
    text=False,                   # True to get str instead of bytes
    check=False,                  # True to raise error on non-zero exit
    shell=False                   # True to execute through the shell
)

print(result.returncode)

The run() function returns a CompletedProcess object with attributes like:

• args – The command that was run.
• returncode – Exit status (0 means success on most systems).
• stdout – Captured standard output (if requested).
• stderr – Captured standard error (if requested).

? Using Popen for Advanced Control

Popen gives you full control: you can interact with the process while it is running, send data, and read incremental output.

import subprocess

# Start a subprocess and set up pipes for communication
proc = subprocess.Popen(
    ["command", "arg1"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
    text=True
)

# Send optional input and read all output
stdout_data, stderr_data = proc.communicate(input="optional input\n")
print("Return code:", proc.returncode)

? Code Examples

? Run a Simple Command

Example: run the echo command and print its output.

import subprocess

# Run a simple echo command and capture its output
result = subprocess.run(
    ["echo", "Hello from subprocess!"],
    capture_output=True,
    text=True
)

# Inspect the return code and captured streams
print("Return code:", result.returncode)
print("STDOUT:", result.stdout.strip())
print("STDERR:", result.stderr)

? Capture Output and Handle Errors

Use check=True to raise an exception if the command fails.

import subprocess

# Run a command and raise an error if it fails
try:
    result = subprocess.run(
        ["python", "--version"],
        capture_output=True,
        text=True,
        check=True
    )
    print("Command succeeded!")
    print("Output:", result.stdout or result.stderr)
# Handle the case where the command exits with a non-zero status
except subprocess.CalledProcessError as e:
    print("Command failed with return code:", e.returncode)
    print("Output:", e.stdout)
    print("Error:", e.stderr)

? Piping Input to a Command

Send data to a command via stdin using input.

import subprocess

text = "line 1\nline 2\nline 3\n"

# Run wc -l and pass the text through stdin
result = subprocess.run(
    ["wc", "-l"],            # counts lines on Unix-like systems
    input=text,
    capture_output=True,
    text=True
)

print("Input text:")
print(text)
print("\nNumber of lines reported by wc -l:", result.stdout.strip())

⚠️ Using shell=True Carefully

shell=True lets you use shell features like wildcards and pipes, but can be dangerous if you use untrusted input (command injection risk).

import subprocess

# Example that uses shell features (pipe + grep)
cmd = "ps aux | grep python"

result = subprocess.run(
    cmd,
    shell=True,
    capture_output=True,
    text=True
)

print("Running:", cmd)
print(result.stdout)

?️ Live Output and Explanation

# Run the echo command and capture its output
result = subprocess.run(
    ["echo", "Hello from subprocess!"],
    capture_output=True,
    text=True
)

Return code: 0
STDOUT: Hello from subprocess!
STDERR: None

? Common Use Cases

• Automating system maintenance scripts (backups, cleanups, deployments).
• Calling existing CLI tools (e.g., git, ffmpeg, curl) from Python.
• Integrating shell scripts into larger Python applications.
• Collecting diagnostics by running tools like ping, traceroute, or top.
• Building wrappers around legacy command-line programs.