“Killed” and exit code 137: how to tell if it was the out-of-memory killer, and what to do

Exit code 137 means the process didn’t crash by itself: something sent it SIGKILL, signal 9, which can’t be caught or cleaned up after. Most of the time that something is the kernel’s out-of-memory killer, which picks a big process and stops it to save the machine.

How the number is built

When a Unix process is ended by a signal, shells and container runtimes report 128 + the signal number. 137 is 128 + 9 (SIGKILL); in binary that’s 10001001, as the number base converter shows. Neighbouring codes tell different stories:

Confirm it was memory

• Linux server or VM: sudo dmesg -T | grep -i -E "out of memory|oom-kill|killed process" or journalctl -k | grep -i oom. A line like Out of memory: Killed process 4121 (node) total-vm:… anon-rss:1650212kB names the victim and how much it was using. (dmesg without -T prints seconds since boot rather than clock time.)
• systemd service: systemctl status myapp shows code=killed, status=9/KILL and often Failed with result 'oom-kill'.
• Docker: docker inspect --format '{{.State.OOMKilled}} {{.State.ExitCode}}' myapp prints true 137 when the container hit its memory limit. docker stats shows live use against the limit.
• Kubernetes: kubectl describe pod mypod shows Last State: Terminated, Reason: OOMKilled, Exit Code: 137. kubectl top pod shows current use.
• CI runners: logs that end with Killed, The runner has received a shutdown signal or exit code 137 during a build or test step are almost always memory.

137 without running out of memory

• Stop timeout: docker stop sends SIGTERM and, after 10 seconds, SIGKILL. An app that ignores SIGTERM always exits 137. Kubernetes does the same after terminationGracePeriodSeconds (30 s by default), for example when a liveness probe keeps failing. Handle SIGTERM and shut down quickly.
• Someone ran kill -9, or a supervisor or watchdog did.
• PID 1 in a container that doesn’t forward signals (a shell script wrapper): use exec in the script or docker run --init so your app receives SIGTERM.

Fixes when it really is memory

• Measure first: free -m, top sorted by memory (Shift+M), /usr/bin/time -v cmd (“Maximum resident set size”), or docker stats.
• Raise the limit to what it actually needs: docker run -m 1g, resources.limits.memory in Kubernetes (set requests close to real use too), a bigger CI runner.
• Add swap on small VMs: on a 512 MB or 1 GB server, a 1 GB swap file turns sudden kills during builds and package installs into slower but successful runs.
• Build somewhere bigger: npm run build, webpack, TypeScript, Rust and C++ compiles can need far more memory than the app needs to run. Build in CI or locally and ship the output; limit parallel jobs (make -j1, cargo build -j 1).
• Node.js: V8’s heap limit is separate from the container’s. Too high and the kernel kills it (137); too low and you get JavaScript heap out of memory instead. Set --max-old-space-size (in MB) to about 75% of the container limit.
• Java: use -XX:MaxRAMPercentage=75 so the JVM sizes its heap from the container limit rather than the host.
• Python and data jobs: read big files in chunks or stream them (pandas.read_csv(..., chunksize=…), iterate lines rather than .read()), and lower worker counts (Gunicorn --workers, multiprocessing pool size); each worker is a full copy.
• Leaks: if memory climbs steadily over hours until the kill, restarting only hides it. Look for caches without a size limit and lists that only grow.

Related

The disk filling up gives a different error, not 137; see ENOSPC: no space left on device. If a web server behind a proxy is being killed, users see 502 Bad Gateway until it restarts. An old webpack/CRA build that crashes with digital envelope routines::unsupported instead of 137 is a different bug: Node OpenSSL envelope error. V8 refusing to grow the JS heap (process still alive long enough to print FATAL ERROR) is JavaScript heap out of memory, not 137. · EMFILE too many open files

Questions

Published 4 November 2026.