FFmpeg Error Handling: How to Find the Key Points in a Pile of Noise

When using Python's subprocess module to call external tools, especially ffmpeg, you often encounter a headache: when a command fails, the thrown subprocess.CalledProcessError exception throws the standard error output (stderr) at you wholesale. This output is often scarily long, mixed with version numbers, compilation information, configuration parameters, etc., while the truly useful error clues may be just a line or two, hidden in this large pile of information and difficult to find.

Problem: FFmpeg Error Reports, Logs Full of "Noise"

For example, suppose you want to use ffmpeg to convert a file that doesn't exist at all:

import subprocess
import logging

logger = logging.getLogger("FFmpegRunner")

cmd = ["ffmpeg", "-hide_banner", "-i", "no_such_file.mp4", "output.mp4"]

try:
    subprocess.run(cmd, check=True, capture_output=True, text=True, encoding="utf-8")
except subprocess.CalledProcessError as e:
    logger.error(f"FFmpeg failed!\nCommand: {' '.join(cmd)}\nError output:\n{e.stderr}")

After running this code, e.stderr may spit out a lot of stuff: FFmpeg's version information, a list of supported encoders... Only at the end can you see a simple no_such_file.mp4: No such file or directory. If this is in a production environment or a complex task flow, facing such a long log, quickly figuring out where the problem is is a nightmare.

C:\Users\c1\Videos>ffmpeg -c:v h264_amf -i 480.mp4  -c:v 152.mp4
ffmpeg version N-112170-gb61733f61f-20230924 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.2.0 (crosstool-NG 1.25.0.232_c175b21)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20230924
  libavutil      58. 25.100 / 58. 25.100
  libavcodec     60. 27.100 / 60. 27.100
  libavformat    60. 13.100 / 60. 13.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Trailing option(s) found in the command: may be ignored.
Unknown decoder 'h264_amf'
Error opening input file 480.mp4.
Error opening input files: Decoder not found

We need a way to pull out the truly critical error information and prevent it from being buried in the "noise."

Solution: Intelligently Extract Key Information

Directly printing the entire e.stderr is definitely not feasible; it's too messy. A better approach is to pick out the sentences that best illustrate the problem from this pile of output.

Observe the error messages of ffmpeg; they usually have a few patterns:

1. Key information is often in the last few lines, such as prompts for files not found or unsupported formats.
2. It will have some obvious keywords, such as "Error," "Invalid," "No such file," "Permission denied," etc.

Based on these characteristics, we can write a function to specifically dig out the useful parts from stderr:

def extract_concise_error(stderr_text: str, max_lines=3, max_length=250) -> str:
    """Extract concise error information from stderr, usually the last few lines with keywords."""
    if not stderr_text:
        return "Unknown error (stderr is empty)"

    # Split stderr by line
    lines = stderr_text.strip().splitlines()
    if not lines:
        return "Unknown error (stderr has no content)"

    # Common error keywords
    error_keywords = ["error", "invalid", "fail", "could not", "no such",
                      "denied", "unsupported", "unable", "can't open", "conversion failed"]

    # Only look at the last few lines (maximum 3 lines by default)
    start = max(0, len(lines) - max_lines)
    for i in range(len(lines) - 1, start - 1, -1):  # Find from back to front
        line = lines[i].strip()
        if not line:  # Skip empty lines
            continue

        # If this line has keywords, it's basically what we're looking for
        if any(keyword in line.lower() for keyword in error_keywords):
            # Add the previous line as context, which may be more useful
            if i > 0 and lines[i-1].strip():
                return f"{lines[i-1].strip()}\n{line}"[:max_length] + ("..." if len(line) > max_length else "")
            return line[:max_length] + ("..." if len(line) > max_length else "")

    # If no keywords are found, take the last line
    for line in reversed(lines):
        if line.strip():
            return line[:max_length] + ("..." if len(line) > max_length else "")
    
    return "Unknown error (no specific problem found)"

# Use it like this:
try:
    subprocess.run(cmd, check=True, capture_output=True, text=True, encoding="utf-8")
except subprocess.CalledProcessError as e:
    short_error = extract_concise_error(e.stderr)
    logger.error(f"FFmpeg failed (exit code: {e.returncode})! Command: {' '.join(cmd)}, Error: {short_error}")
    # If you need the complete output, you can record it at the DEBUG level
    # logger.debug(f"Full error output:\n{e.stderr}")

Some Thoughts and Pitfalls When Writing This Function

1. Keywords are not necessarily complete The error_keywords I listed are based on experience and may miss some special error messages from ffmpeg. In actual use, you may need to add a few keywords when encountering new situations.

2. Context is very important Sometimes you don't understand just by looking at the error line, such as "file cannot be opened." You have to look at the previous line to know which file it is. So I added some code to try to bring the previous line as well.

3. What if no keywords are found? If no keywords are matched, I take the last line as the result as a fallback. It's better than throwing out the entire page of logs, but it's not necessarily accurate every time.

4. Trouble with character encodingffmpeg's output sometimes pops up with strange characters that are not standard UTF-8. To prevent the program from crashing, encoding="utf-8" is added to subprocess.run, and errors="replace" can be used as a backup if necessary.

5. How to log it well? My approach is to record the short error message at the ERROR level to easily identify the problem; if you need to check the details, record the complete output at the DEBUG level. This is both clear and doesn't lose information.

With this method, we can quickly retrieve key information from the stderr of subprocess.CalledProcessError, making the logs much easier to read and troubleshooting faster. This approach can be used not only on ffmpeg but also on other command-line tools with verbose output.

The core is to understand the patterns of the tool's error output and then "streamline" it.

Although it's not guaranteed to be perfect every time, it can at least help you flip through fewer pages of logs and scratch your head less.