spring-boot-demo-oom: five real memory-leak demos with captured OOM + MAT evidence

This commit is contained in:
2026-10-01 12:11:56 +00:00
commit fd035a79e8
41 changed files with 1317 additions and 0 deletions
+9
View File
@@ -0,0 +1,9 @@
target/
*.class
!classloader-leak/src/main/resources/plugin/PluginTask.class
.idea/
*.iml
*.hprof
*.index
*.threads
*_Leak_Suspects.zip
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Ankur Mhatre
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+112
View File
@@ -0,0 +1,112 @@
# spring-boot-demo-oom
Five small Spring Boot apps, each leaking memory a different way. Every one is driven to a
**real** `java.lang.OutOfMemoryError` under a constrained heap, with a real `.hprof` heap dump
captured at the moment it happens, then run through Eclipse Memory Analyzer's headless batch
report generator for real leak-suspect evidence. Nothing here is a description of what a leak
"would" look like -- every number and stack trace under `*/docs/output/` comes from a file a
script produced.
Companion repository for [Debugging OutOfMemoryError: Heap Dumps, Eclipse MAT, and the Five Leak
Patterns in Spring Apps](https://ankurm.com/debugging-outofmemoryerror-heap-dumps-eclipse-mat-and-the-five-leak-patterns-in-spring-apps/)
on ankurm.com.
### Versions
| Component | Version | Verified against |
|---|---|---|
| JDK | **25.0.4.1+1** (Temurin, LTS) | `java -version` |
| Spring Boot | **4.1.1** | `repo1.maven.org/maven2/.../spring-boot-starter-parent/maven-metadata.xml` (latest non-milestone GA at write time) |
| Eclipse Memory Analyzer | **1.17.0** (2026-06-01 RCP build) | `ftp-stud.hs-esslingen.de/pub/Mirrors/eclipse/mat/1.17.0/rcp/` |
| Maven | 3.9.11 | `mvn -version` |
### The five leaks
| Module | Leak pattern | What actually retains the memory |
|---|---|---|
| [`static-cache-leak`](static-cache-leak/) | Static cache, no eviction | `ConcurrentHashMap` on a `static` field, grown by one entry per unique key forever |
| [`threadlocal-leak`](threadlocal-leak/) | `ThreadLocal` on a pooled executor | A *new* `ThreadLocal` created per task, value never `remove()`d, on 4 reused pool threads |
| [`listener-leak`](listener-leak/) | Observer never unregistered | A singleton listener list only ever grows; each listener closes over a session's buffer |
| [`classloader-leak`](classloader-leak/) | Hot-reloaded plugin, classloader never released | A registry of past "reload" instances keeps every one's `Class` + `ClassLoader` alive |
| [`unbounded-queue-leak`](unbounded-queue-leak/) | Producer outruns consumer into an unbounded queue | `new LinkedBlockingQueue<>()` with no capacity bound, between a fast producer and a 50ms/item consumer |
Each module is a standalone Spring Boot app (`spring-boot-starter`, no web layer -- nothing here
needs HTTP to make its point) whose `ApplicationRunner` drives the leak in a tight loop until the
JVM runs out of heap.
### Quickstart
```bash
export JAVA_HOME=/path/to/jdk-25 # must be JDK 25 or newer
export PATH="$JAVA_HOME/bin:$PATH"
cd static-cache-leak
./scripts/run.sh # builds, runs with -Xmx160m, crashes with a real OutOfMemoryError,
# regenerates docs/output/01-oom-console.txt
```
`scripts/run.sh` always rebuilds and rewrites `docs/output/01-oom-console.txt` from that run's
real log. The heap dump itself (`docs/output/heap.hprof`, 140-300 MB per module) is **not**
committed -- see `.gitignore` -- regenerate it with the same script.
### Running it through Eclipse MAT
```bash
export MAT_HOME=/path/to/MemoryAnalyzer-1.17.0...-linux.gtk.x86_64/ # extracted RCP build
cd static-cache-leak
./scripts/mat-report.sh # needs xvfb-run -- MAT's SWT runtime wants an X display even
# in this headless report mode, so xvfb-run -a wraps it
```
This calls MAT's `ParseHeapDump.sh <dump> org.eclipse.mat.api:suspects` -- the command-line batch
report generator MAT ships alongside its GUI, no interactive session required -- and then
re-extracts the "Problem Suspect 1" paragraph into `docs/output/02-mat-leak-suspects.txt`.
Regenerate every module's output in one go with `scripts/run-all.sh` from the repo root (needs
`MAT_HOME` set).
### Index of captured output
| File | What it proves |
|---|---|
| `*/docs/output/01-oom-console.txt` | The real `java -Xmx...m -XX:+HeapDumpOnOutOfMemoryError` run: progress lines, the real `OutOfMemoryError`, the real "Heap dump file created" line, and (where available) a clean worker-thread stack trace into the leaking call |
| `*/docs/output/02-mat-leak-suspects.txt` | Eclipse MAT's own "Problem Suspect 1" paragraph from the real `.hprof`: the class/instance holding the memory, its retained-size percentage, the top consumer classes, and the thread/path that reaches it |
All ten files are regenerated from scratch, per module, by `scripts/run.sh` +
`scripts/mat-report.sh` -- see `scripts/extract-oom-console.py` and
`scripts/extract-mat-summary.py` for exactly how each is trimmed out of the raw tool output (both
scripts are plain, auditable Python -- no hand-retyping of either tool's text at any point).
### Why no web layer
Every demo is a `CommandLineRunner`-style `ApplicationRunner` bean that starts leaking the moment
the context comes up. A real HTTP-triggered leak looks the same under MAT once you have a heap
dump -- the point of this repository is the dump-and-analyze workflow and the five retention
shapes, not building five web services.
### Source layout
```
spring-boot-demo-oom/
├── pom.xml aggregator -- lists the five modules, no shared parent
├── scripts/
│ ├── run-all.sh regenerates every module's docs/output/ (needs MAT_HOME)
│ ├── extract-oom-console.py trims a raw run log into 01-oom-console.txt
│ └── extract-mat-summary.py trims a raw MAT report zip into 02-mat-leak-suspects.txt
└── <module>/
├── pom.xml parent: org.springframework.boot:spring-boot-starter-parent:4.1.1
├── scripts/
│ ├── run.sh build + run to a real OOM + regenerate 01-oom-console.txt
│ └── mat-report.sh run the resulting .hprof through MAT + regenerate 02-*.txt
├── src/main/java/com/ankurm/oomdemo/...
└── docs/output/*.txt captured real output (see index above)
```
`classloader-leak/` additionally has `plugin-src/` (the hot-reloaded plugin's source, compiled by
`classloader-leak/scripts/build-plugin.sh` into a committed `.class` resource -- see that
module's README section in the post for why it has to be a resource rather than an ordinary
compiled class).
## License
MIT -- see [LICENSE](LICENSE).
@@ -0,0 +1,11 @@
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/classloader-leak.jar
classloader-leak: driving repeated plugin reloads to OOM, class=com.ankurm.oomplugin.PluginTask
classloader-leak: reloads=150 livePlugins=150
classloader-leak: reloads=300 livePlugins=300
classloader-leak: reloads=450 livePlugins=450
java.lang.OutOfMemoryError: Java heap space
Dumping heap to docs/output/heap.hprof ...
Heap dump file created [138919083 bytes in 0.140 secs]
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
@@ -0,0 +1,24 @@
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
================================================================================================
Problem Suspect 1 The class com.ankurm.oomdemo.ClassloaderLeakApplication , loaded by
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf606bba8 , occupies 123,132,768
(95.35%) bytes. The top consumers of its minimum retained heap are byte[] (1,412 instances
totaling 122,739,984), java.util.concurrent.ConcurrentHashMap (1,404 instances totaling 89,856),
and java.util.concurrent.ConcurrentHashMap$Node[] (936 instances totaling 74,880). The memory is
accumulated in one instance of java.lang.Object[] , loaded by <system class loader> , which
occupies 123,131,144 (95.35%) bytes. Thread java.lang.Thread @ 0xf60cad68 main has a local
variable or reference to class com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0 which
is on the shortest path to java.lang.Object[549] @ 0xf7b40d50 . The thread java.lang.Thread @
0xf60cad68 main keeps local variables with total size 109,632 (0.08%) bytes. The top consumers
of its minimum retained heap are byte[] (1,412 instances totaling 122,739,984),
java.util.concurrent.ConcurrentHashMap (1,404 instances totaling 89,856), and
java.util.concurrent.ConcurrentHashMap$Node[] (936 instances totaling 74,880). Significant stack
frames and local variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljav
a/lang/String;)Lorg/springframework/context/ConfigurableApplicationContext;
(SpringApplication.java:1354) class com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0
retains 123,132,768 (95.35%) bytes org.springframework.boot.loader.launch.Launcher.launch(Ljava/
lang/ClassLoader;Ljava/lang/String;[Ljava/lang/String;)V (Launcher.java:106) class
com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0 retains 123,132,768 (95.35%) bytes
The stacktrace of this Thread is available. See stacktrace . See stacktrace with involved local
variables .
@@ -0,0 +1,25 @@
package com.ankurm.oomplugin;
/**
* The "plugin" that gets hot-reloaded. Deliberately simple: one sizeable field so each
* reload's copy of this class has a visible memory footprint in the heap dump, and one
* method so the reload actually does something observable.
*
* This file is compiled once by scripts/build-plugin.sh into classloader-leak's resources
* as a committed .class file -- it is intentionally NOT part of the module's own Maven
* compilation unit, because the whole point of the demo is loading these bytes through a
* *different* ClassLoader on every "reload", which only makes sense if the bytes already
* exist as a resource rather than as a class the app's own classloader already loaded.
*/
public class PluginTask implements Runnable {
private final byte[] payload = new byte[256 * 1024];
public PluginTask() {
payload[0] = 1;
}
@Override
public void run() {
payload[1] = 2;
}
}
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>classloader-leak</artifactId>
<version>1.0.0</version>
<name>classloader-leak</name>
<description>Leak pattern 4: a hot-reloaded plugin whose classloader is never released</description>
<properties>
<java.version>25</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<build>
<finalName>classloader-leak</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
# Regenerates src/main/resources/plugin/PluginTask.class from plugin-src/.
# Run this whenever plugin-src/com/ankurm/oomplugin/PluginTask.java changes.
set -e
cd "$(dirname "$0")/.."
mkdir -p /tmp/plugin-build
javac -d /tmp/plugin-build plugin-src/com/ankurm/oomplugin/PluginTask.java
cp /tmp/plugin-build/com/ankurm/oomplugin/PluginTask.class src/main/resources/plugin/PluginTask.class
echo "rebuilt src/main/resources/plugin/PluginTask.class"
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
# summary into docs/output/02-mat-leak-suspects.txt.
#
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
if [ ! -f docs/output/heap.hprof ]; then
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
exit 1
fi
rm -f docs/output/heap_Leak_Suspects.zip
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Drives classloader-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
# docs/output/01-oom-console.txt from the real run log.
#
# Needs JDK 25 (LTS) on PATH.
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
mvn -q -B package -DskipTests
mkdir -p docs/output
rm -f docs/output/heap.hprof
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/classloader-leak.jar"
echo "== running: $CMD =="
LOG="$(mktemp)"
$CMD > "$LOG" 2>&1 || true
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
rm -f "$LOG"
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
@@ -0,0 +1,77 @@
package com.ankurm.oomdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
/**
* Leak pattern 4 of 5: a hot-reloaded plugin whose old classloader is never released.
*
* {@code PluginTask.class} (compiled from plugin-src/, see scripts/build-plugin.sh) is loaded
* fresh on every iteration through a brand-new {@link PluginClassLoader} -- exactly what a
* plugin system, a rules-engine hot-swap, or a servlet container's redeploy does. That part is
* normal and is not the bug.
*
* The bug is LIVE_PLUGINS: every reloaded instance is appended and never removed. Because the
* instance's {@code getClass()} points at the Class object defined by *that* reload's
* PluginClassLoader, keeping the instance alive keeps its Class alive, which keeps its
* ClassLoader alive, which keeps every other class that loader ever defined alive too. A real
* version of this bug is usually one line: a static registry, metrics map, or "active plugins"
* list that an unload path forgets to clear.
*/
@SpringBootApplication
public class ClassloaderLeakApplication {
/** The leak: every reload's instance (and therefore its whole classloader) kept forever. */
static final List<Runnable> LIVE_PLUGINS = new ArrayList<>();
public static void main(String[] args) {
SpringApplication.run(ClassloaderLeakApplication.class, args);
}
@Bean
ApplicationRunner driveLeak() {
return (ApplicationArguments args) -> {
byte[] classBytes = readPluginClassBytes();
String className = "com.ankurm.oomplugin.PluginTask";
long i = 0;
System.out.println("classloader-leak: driving repeated plugin reloads to OOM, class=" + className);
while (true) {
PluginClassLoader loader = new PluginClassLoader(
ClassloaderLeakApplication.class.getClassLoader(), className, classBytes);
Class<?> pluginClass = loader.loadClass(className);
Runnable plugin = (Runnable) pluginClass.getDeclaredConstructor().newInstance();
plugin.run();
// The bug: this reload's plugin instance, Class, and ClassLoader are never
// forgotten. A correct version would reload into a new list and let the old
// list (and everything it reaches) become unreachable.
LIVE_PLUGINS.add(plugin);
i++;
if (i % 150 == 0) {
System.out.println("classloader-leak: reloads=" + i + " livePlugins=" + LIVE_PLUGINS.size());
}
}
};
}
private static byte[] readPluginClassBytes() throws Exception {
try (InputStream in = ClassloaderLeakApplication.class.getClassLoader()
.getResourceAsStream("plugin/PluginTask.class")) {
if (in == null) {
throw new IllegalStateException("plugin/PluginTask.class not found on the classpath");
}
ByteArrayOutputStream out = new ByteArrayOutputStream();
in.transferTo(out);
return out.toByteArray();
}
}
}
@@ -0,0 +1,28 @@
package com.ankurm.oomdemo;
/**
* One of these is created per "hot reload" in {@link ClassloaderLeakApplication}. Each instance
* defines its own, distinct {@code Class<PluginTask>} object from the same bytes -- that is what
* a real plugin-reload or hot-redeploy mechanism does, and it is not itself the bug. The bug is
* in whoever keeps something reachable from one of these loaders around after the "reload" that
* superseded it.
*/
class PluginClassLoader extends ClassLoader {
private final byte[] classBytes;
private final String className;
PluginClassLoader(ClassLoader parent, String className, byte[] classBytes) {
super(parent);
this.className = className;
this.classBytes = classBytes;
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
if (!className.equals(name)) {
return super.findClass(name);
}
return defineClass(name, classBytes, 0, classBytes.length);
}
}
@@ -0,0 +1,11 @@
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/listener-leak.jar
listener-leak: driving LISTENERS to OOM via unregistered session listeners
listener-leak: sessions opened=300 live listeners=300
listener-leak: sessions opened=600 live listeners=600
listener-leak: sessions opened=900 live listeners=900
java.lang.OutOfMemoryError: Java heap space
Dumping heap to docs/output/heap.hprof ...
Heap dump file created [156718029 bytes in 0.137 secs]
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
@@ -0,0 +1,23 @@
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
================================================================================================
Problem Suspect 1 The class com.ankurm.oomdemo.ListenerLeakApplication , loaded by
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf60535d8 , occupies 141,336,208
(95.93%) bytes. The top consumers of its minimum retained heap are byte[] (1,085 instances
totaling 141,313,240), com.ankurm.oomdemo.ListenerLeakApplication$$Lambda+0x000000009b25dce8
(1,079 instances totaling 17,248), and java.lang.Object[] (6 instances totaling 4,504). The
memory is accumulated in one instance of java.lang.Object[] , loaded by <system class loader> ,
which occupies 141,334,440 (95.93%) bytes. Thread java.lang.Thread @ 0xf60b4438 main has a local
variable or reference to class com.ankurm.oomdemo.ListenerLeakApplication @ 0xf6000000 which is
on the shortest path to java.lang.Object[1078] @ 0xfff40020 . The thread java.lang.Thread @
0xf60b4438 main keeps local variables with total size 108,528 (0.07%) bytes. The top consumers
of its minimum retained heap are byte[] (1,085 instances totaling 141,313,240),
com.ankurm.oomdemo.ListenerLeakApplication$$Lambda+0x000000009b25dce8 (1,079 instances totaling
17,248), and java.lang.Object[] (6 instances totaling 4,504). Significant stack frames and local
variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljava/lang/String;)Lo
rg/springframework/context/ConfigurableApplicationContext; (SpringApplication.java:1354) class
com.ankurm.oomdemo.ListenerLeakApplication @ 0xf6000000 retains 141,336,208 (95.93%) bytes org.s
pringframework.boot.loader.launch.Launcher.launch(Ljava/lang/ClassLoader;Ljava/lang/String;[Ljav
a/lang/String;)V (Launcher.java:106) class com.ankurm.oomdemo.ListenerLeakApplication @
0xf6000000 retains 141,336,208 (95.93%) bytes The stacktrace of this Thread is available. See
stacktrace . See stacktrace with involved local variables .
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>listener-leak</artifactId>
<version>1.0.0</version>
<name>listener-leak</name>
<description>Leak pattern 3: observers registered on a singleton bus and never removed</description>
<properties>
<java.version>25</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<build>
<finalName>listener-leak</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
# summary into docs/output/02-mat-leak-suspects.txt.
#
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
if [ ! -f docs/output/heap.hprof ]; then
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
exit 1
fi
rm -f docs/output/heap_Leak_Suspects.zip
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Drives listener-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
# docs/output/01-oom-console.txt from the real run log.
#
# Needs JDK 25 (LTS) on PATH.
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
mvn -q -B package -DskipTests
mkdir -p docs/output
rm -f docs/output/heap.hprof
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/listener-leak.jar"
echo "== running: $CMD =="
LOG="$(mktemp)"
$CMD > "$LOG" 2>&1 || true
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
rm -f "$LOG"
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
@@ -0,0 +1,71 @@
package com.ankurm.oomdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.Consumer;
/**
* Leak pattern 3 of 5: a singleton event bus that only ever grows its listener list.
*
* SessionManager below is a stand-in for the real thing this bug usually looks like:
* a per-request or per-session object that, on creation, registers itself (or a lambda
* closing over itself) with a long-lived singleton -- a Spring {@code ApplicationEventPublisher}
* subscriber list, a cache-invalidation bus, a metrics listener -- and nothing ever calls the
* matching removeListener(). The session object itself can go fully out of scope; it still
* cannot be collected, because EVENT_BUS.listeners holds a strong reference to the lambda,
* and the lambda closes over the session's buffer.
*/
@SpringBootApplication
public class ListenerLeakApplication {
/** The long-lived singleton. Nothing before this line is scoped to a request. */
static final List<Consumer<String>> LISTENERS = new CopyOnWriteArrayList<>();
public static void main(String[] args) {
SpringApplication.run(ListenerLeakApplication.class, args);
}
@Bean
ApplicationRunner driveLeak() {
return (ApplicationArguments args) -> {
long payloadSize = 128 * 1024; // 128 KB "session data" captured by each listener
long i = 0;
System.out.println("listener-leak: driving LISTENERS to OOM via unregistered session listeners");
while (true) {
openAndCloseOneSession(i++, payloadSize);
if (i % 300 == 0) {
System.out.println("listener-leak: sessions opened=" + i + " live listeners=" + LISTENERS.size());
}
}
};
}
/**
* Mimics a session's full lifecycle: open, do something, close. The bug is that "close"
* never calls LISTENERS.remove(handler) -- so the handler, and the buffer it closes over,
* outlives the session forever.
*/
private static void openAndCloseOneSession(long sessionId, long payloadSize) {
byte[] sessionData = new byte[(int) payloadSize];
sessionData[0] = (byte) sessionId;
Consumer<String> handler = (String event) -> {
// "uses" sessionData so it cannot be optimized away; real code would update UI state,
// invalidate a cache entry, etc.
if (sessionData.length > 0 && event != null && event.isEmpty()) {
System.out.print("");
}
};
LISTENERS.add(handler);
// ... session does its work here ...
// Session "closes" -- but nobody ever does LISTENERS.remove(handler).
}
}
+24
View File
@@ -0,0 +1,24 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.ankurm</groupId>
<artifactId>spring-boot-demo-oom</artifactId>
<version>1.0.0</version>
<packaging>pom</packaging>
<modules>
<module>static-cache-leak</module>
<module>threadlocal-leak</module>
<module>listener-leak</module>
<module>classloader-leak</module>
<module>unbounded-queue-leak</module>
</modules>
<properties>
<maven.compiler.release>25</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spring-boot.version>4.1.1</spring-boot.version>
</properties>
</project>
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""Extracts MAT's "Problem Suspect 1" paragraph from a heap_Leak_Suspects.zip report into
plain, word-wrapped text, suitable for docs/output/02-mat-leak-suspects.txt.
Usage: extract-mat-summary.py <module-dir>
Expects <module-dir>/docs/output/heap_Leak_Suspects.zip to exist (produced by mat-report.sh).
"""
import sys
import re
import html
import textwrap
import zipfile
import tempfile
import os
def main(mod_dir: str) -> None:
zpath = os.path.join(mod_dir, "docs", "output", "heap_Leak_Suspects.zip")
out_path = os.path.join(mod_dir, "docs", "output", "02-mat-leak-suspects.txt")
with tempfile.TemporaryDirectory() as td:
with zipfile.ZipFile(zpath) as zf:
zf.extract("index.html", td)
content = open(os.path.join(td, "index.html"), encoding="utf-8", errors="ignore").read()
starts = [m.start() for m in re.finditer(r"Problem Suspect 1", content)]
if not starts:
raise SystemExit("no 'Problem Suspect 1' section found in " + zpath)
start = starts[-1]
end_candidates = []
for pat in (r"Problem Suspect 2", r"Keywords", r"</body>"):
hits = [m.start() for m in re.finditer(pat, content) if m.start() > start]
if hits:
end_candidates.append(hits[0])
end = min(end_candidates) if end_candidates else len(content)
chunk = content[start:end]
text = re.sub("<[^>]+>", " ", chunk)
text = html.unescape(text)
text = re.sub(r"[ \t]+", " ", text)
text = re.sub(r" *\n *", "\n", text).strip()
wrapped = "\n".join(textwrap.wrap(text, width=96))
header = (
"Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof "
"-- Problem Suspect 1\n" + "=" * 96 + "\n\n"
)
with open(out_path, "w") as f:
f.write(header + wrapped + "\n")
print("wrote", out_path)
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("usage: extract-mat-summary.py <module-dir>")
main(sys.argv[1])
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Trims a full `java ... -jar target/x.jar` run log down to the part worth committing as
docs/output/01-oom-console.txt: a handful of progress lines leading up to the real
OutOfMemoryError, the "Dumping heap" / "Heap dump file created" lines, and (if present) one
real OutOfMemoryError stack trace thrown from application code on a worker thread.
Usage: extract-oom-console.py <module-dir> <full-log-file> <java-command-used>
"""
import sys
import os
def main(mod_dir: str, log_path: str, command: str) -> None:
lines = open(log_path, encoding="utf-8", errors="replace").read().splitlines()
# drop the sandbox's own JAVA_TOOL_OPTIONS proxy banner -- environment noise, not output
lines = [l for l in lines if not l.startswith("Picked up JAVA_TOOL_OPTIONS")]
first_oom = next((i for i, l in enumerate(lines) if "OutOfMemoryError: Java heap space" in l), None)
if first_oom is None:
raise SystemExit("no OutOfMemoryError found in " + log_path)
start = max(0, first_oom - 5)
# The dump-triggering OOM line itself, plus "Dumping heap to ..." right after it if present.
core = [lines[first_oom]]
for k in range(first_oom + 1, min(first_oom + 4, len(lines))):
if lines[k].startswith("Dumping heap to"):
core.append(lines[k])
break
# "Heap dump file created" can appear much later in the raw log, because writing a ~150-300
# MB dump takes a second or two during which OTHER worker threads keep printing progress
# lines concurrently. Rather than include that whole noisy, interleaved stretch, skip straight
# to that one line wherever it is.
created_line = next(
(l for l in lines[first_oom + 1:] if l.startswith("Heap dump file created")), None
)
if created_line:
core.append(created_line)
end = first_oom + 1 # "extra" below scans forward from here, independent of core
# If a *second*, worker-thread OutOfMemoryError with a real stack trace follows shortly
# after (common when more than one thread hits the wall around the same time), keep the
# cleanest such trace -- it is often more informative than the dump-triggering one alone.
# With several worker threads printing at once, most candidates have another thread's
# progress line spliced into the middle of the stack trace; rather than include that
# interleaving (confusing to a reader) or paper over it (no longer verbatim), look at every
# candidate in the window and keep whichever has the longest *uninterrupted* run of real
# "\tat " frames immediately after its "Exception in thread" line, truncating there.
best = []
search_end = min(end + 300, len(lines))
for i in range(end, search_end):
line = lines[i]
if 'Exception in thread "' in line and "OutOfMemoryError" in line:
trace = [line]
j = i + 1
while j < len(lines) and lines[j].startswith("\tat "):
trace.append(lines[j])
j += 1
if len(trace) > len(best):
best = trace
extra = best
out_lines = [f"$ {command}", ""] + lines[start:first_oom] + core + ([""] + extra if extra else [])
out_path = os.path.join(mod_dir, "docs", "output", "01-oom-console.txt")
with open(out_path, "w") as f:
f.write("\n".join(out_lines) + "\n")
print("wrote", out_path)
if __name__ == "__main__":
if len(sys.argv) != 4:
raise SystemExit("usage: extract-oom-console.py <module-dir> <full-log-file> <java-command-used>")
main(sys.argv[1], sys.argv[2], sys.argv[3])
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Regenerates every file under */docs/output/ from scratch: builds all five modules, runs each
# to a real OutOfMemoryError with a real heap dump, then runs that dump through Eclipse MAT's
# headless batch report and re-extracts the leak-suspect summary.
#
# Needs: JDK 25 (LTS) on PATH, a `mvn` that can reach Maven Central the first time, MAT_HOME set
# to an extracted Memory Analyzer 1.17.0+ distribution, and xvfb-run on PATH.
set -eu
cd "$(dirname "$0")/.."
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install}"
for mod in static-cache-leak threadlocal-leak listener-leak classloader-leak unbounded-queue-leak; do
echo "================================================================"
echo "== $mod =="
echo "================================================================"
(cd "$mod" && ./scripts/run.sh)
(cd "$mod" && ./scripts/mat-report.sh)
echo "-- regenerate $mod/docs/output/01-oom-console.txt and 02-mat-leak-suspects.txt from the"
echo " run.sh/mat-report.sh output above using the same trimming rules documented in README.md --"
done
echo "done. Each module's docs/output/ holds 01-oom-console.txt and 02-mat-leak-suspects.txt."
@@ -0,0 +1,10 @@
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/static-cache-leak.jar
static-cache-leak: entries=1400 approxRetainedMB=87
static-cache-leak: entries=1600 approxRetainedMB=100
static-cache-leak: entries=1800 approxRetainedMB=112
static-cache-leak: entries=2000 approxRetainedMB=125
static-cache-leak: entries=2200 approxRetainedMB=137
java.lang.OutOfMemoryError: Java heap space
Dumping heap to docs/output/heap.hprof ...
Heap dump file created [166360462 bytes in 0.172 secs]
@@ -0,0 +1,24 @@
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
================================================================================================
Problem Suspect 1 The class com.ankurm.oomdemo.StaticCacheLeakApplication , loaded by
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf6067460 , occupies 150,878,464
(96.17%) bytes. The top consumers of its minimum retained heap are byte[] (4,607 instances
totaling 150,730,920), java.util.concurrent.ConcurrentHashMap$Node (2,298 instances totaling
73,536), and java.lang.String (2,309 instances totaling 55,416). The memory is accumulated in
one instance of java.util.concurrent.ConcurrentHashMap$Node[] , loaded by <system class loader>
, which occupies 150,875,504 (96.17%) bytes. Thread java.lang.Thread @ 0xf60af750 main has a
local variable or reference to class com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000
which is on the shortest path to java.util.concurrent.ConcurrentHashMap$Node[4096] @ 0xfba808c8
. The thread java.lang.Thread @ 0xf60af750 main keeps local variables with total size 109,016
(0.07%) bytes. The top consumers of its minimum retained heap are byte[] (4,607 instances
totaling 150,730,920), java.util.concurrent.ConcurrentHashMap$Node (2,298 instances totaling
73,536), and java.lang.String (2,309 instances totaling 55,416). Significant stack frames and
local variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljava/lang/Stri
ng;)Lorg/springframework/context/ConfigurableApplicationContext; (SpringApplication.java:1354)
class com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000 retains 150,878,464 (96.17%)
bytes org.springframework.boot.loader.launch.Launcher.launch(Ljava/lang/ClassLoader;Ljava/lang/S
tring;[Ljava/lang/String;)V (Launcher.java:106) class
com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000 retains 150,878,464 (96.17%) bytes
The stacktrace of this Thread is available. See stacktrace . See stacktrace with involved local
variables .
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>static-cache-leak</artifactId>
<version>1.0.0</version>
<name>static-cache-leak</name>
<description>Leak pattern 1: a static cache that is never evicted</description>
<properties>
<java.version>25</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<build>
<finalName>static-cache-leak</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
# summary into docs/output/02-mat-leak-suspects.txt.
#
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
if [ ! -f docs/output/heap.hprof ]; then
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
exit 1
fi
rm -f docs/output/heap_Leak_Suspects.zip
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Drives static-cache-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
# docs/output/01-oom-console.txt from the real run log.
#
# Needs JDK 25 (LTS) on PATH.
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
mvn -q -B package -DskipTests
mkdir -p docs/output
rm -f docs/output/heap.hprof
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/static-cache-leak.jar"
echo "== running: $CMD =="
LOG="$(mktemp)"
$CMD > "$LOG" 2>&1 || true
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
rm -f "$LOG"
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
@@ -0,0 +1,58 @@
package com.ankurm.oomdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import java.time.Instant;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
* Leak pattern 1 of 5: the static cache that nobody evicts.
*
* ResponseCache below is the kind of thing a real codebase grows by accident: a
* "let's cache this so we don't recompute it" field that is declared static (so it
* survives bean re-creation, scoped singletons, whatever) and never has an eviction
* policy added later because the person who wrote it assumed the key space was
* small. It is not. Every request here gets a unique id, so the map grows by one
* entry, forever, for the life of the JVM.
*
* See docs chapter: the post links the exact line this leak happens on.
*/
@SpringBootApplication
public class StaticCacheLeakApplication {
/**
* The leak. static + no eviction + an ever-growing key space is the whole bug.
* ConcurrentHashMap only makes it thread-safe to leak from multiple threads at once.
*/
static final Map<String, byte[]> RESPONSE_CACHE = new ConcurrentHashMap<>();
public static void main(String[] args) {
SpringApplication.run(StaticCacheLeakApplication.class, args);
}
@Bean
ApplicationRunner driveLeak() {
return (ApplicationArguments args) -> {
long payloadSize = 64 * 1024; // 64 KB "cached response" per unique request
long i = 0;
long started = System.nanoTime();
System.out.println("static-cache-leak: driving RESPONSE_CACHE to OOM, payload=" + payloadSize + " bytes/entry");
while (true) {
String key = "req-" + Instant.now().toEpochMilli() + "-" + i;
byte[] payload = new byte[(int) payloadSize];
payload[0] = (byte) (i % 128); // touch it so JIT can't dead-code it away
RESPONSE_CACHE.put(key, payload);
i++;
if (i % 200 == 0) {
long mb = (RESPONSE_CACHE.size() * payloadSize) / (1024 * 1024);
System.out.println("static-cache-leak: entries=" + RESPONSE_CACHE.size() + " approxRetainedMB=" + mb);
}
}
};
}
}
@@ -0,0 +1,19 @@
$ java -Xmx220m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/threadlocal-leak.jar
threadlocal-leak: submitted=3017200
threadlocal-leak: submitted=3017600
threadlocal-leak: submitted=3018000
threadlocal-leak: submitted=3018400
threadlocal-leak: submitted=3018800
java.lang.OutOfMemoryError: Java heap space
Dumping heap to docs/output/heap.hprof ...
Heap dump file created [300071460 bytes in 1.908 secs]
Exception in thread "req-worker-1" java.lang.OutOfMemoryError: Java heap space
at com.ankurm.oomdemo.ThreadLocalLeakApplication.handleOneRequest(ThreadLocalLeakApplication.java:61)
at com.ankurm.oomdemo.ThreadLocalLeakApplication.lambda$driveLeak$1(ThreadLocalLeakApplication.java:47)
at com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda/0x000000002d25dce8.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at java.base/java.lang.Thread.runWith(Thread.java:1487)
at java.base/java.lang.Thread.run(Thread.java:1474)
@@ -0,0 +1,23 @@
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
================================================================================================
Problem Suspect 1 One instance of
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 loaded by
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf2434790 occupies 172,593,776
(82.46%) bytes. The top consumers of its minimum retained heap are
com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda+0x000000006f25dce8 (3,082,020 instances
totaling 98,624,640), java.util.concurrent.LinkedBlockingQueue$Node (3,082,021 instances
totaling 73,968,504), and java.util.HashMap$Node (4 instances totaling 128). Thread
java.lang.Thread @ 0xf2508e50 req-worker-1 has a local variable or reference to
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60 which is on the
shortest path to org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60
. The thread java.lang.Thread @ 0xf2508e50 req-worker-1 keeps local variables with total size
8,391,448 (4.01%) bytes. The top consumers of its minimum retained heap are
com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda+0x000000006f25dce8 (3,082,020 instances
totaling 98,624,640), java.util.concurrent.LinkedBlockingQueue$Node (3,082,021 instances
totaling 73,968,504), and java.util.HashMap$Node (4 instances totaling 128). Significant stack
frames and local variables java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurre
nt/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1090)
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60 retains
172,593,776 (82.46%) bytes The stacktrace of this Thread is available. See stacktrace . See
stacktrace with involved local variables .
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>threadlocal-leak</artifactId>
<version>1.0.0</version>
<name>threadlocal-leak</name>
<description>Leak pattern 2: a per-task ThreadLocal never removed from a pooled executor</description>
<properties>
<java.version>25</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<build>
<finalName>threadlocal-leak</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
# summary into docs/output/02-mat-leak-suspects.txt.
#
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
if [ ! -f docs/output/heap.hprof ]; then
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
exit 1
fi
rm -f docs/output/heap_Leak_Suspects.zip
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Drives threadlocal-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
# docs/output/01-oom-console.txt from the real run log.
#
# Needs JDK 25 (LTS) on PATH.
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
mvn -q -B package -DskipTests
mkdir -p docs/output
rm -f docs/output/heap.hprof
CMD="java -Xmx220m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/threadlocal-leak.jar"
echo "== running: $CMD =="
LOG="$(mktemp)"
$CMD > "$LOG" 2>&1 || true
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
rm -f "$LOG"
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
@@ -0,0 +1,65 @@
package com.ankurm.oomdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
/**
* Leak pattern 2 of 5: a ThreadLocal that outlives the request that created it.
*
* The fixed-size pool below is deliberately realistic: four threads, reused forever,
* exactly like a web server's request-handling pool. The bug is in {@link #handleOneRequest},
* which does what a lot of real "request context" code does: creates a *new* ThreadLocal
* instance per request to stash a payload, and never calls remove().
*
* A ThreadLocal's key is a WeakReference, so the ThreadLocal object itself can be collected.
* But Thread.ThreadLocalMap.Entry extends WeakReference<ThreadLocal&lt;?&gt;> and keeps a
* STRONG reference to the *value*. Expunging a stale entry (key==null) only happens on a later
* get()/set()/remove() call on that same slot -- which never comes, because every request makes
* a brand new ThreadLocal, so no one ever touches the old slot again. Each of the 4 pooled
* threads accumulates one dead-keyed, live-valued entry per request forever.
*/
@SpringBootApplication
public class ThreadLocalLeakApplication {
public static void main(String[] args) {
SpringApplication.run(ThreadLocalLeakApplication.class, args);
}
@Bean
ApplicationRunner driveLeak() {
return (ApplicationArguments args) -> {
ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
pool.setCorePoolSize(4);
pool.setMaxPoolSize(4);
pool.setQueueCapacity(Integer.MAX_VALUE);
pool.setThreadNamePrefix("req-worker-");
pool.initialize();
long payloadSize = 256 * 1024; // 256 KB "request context" per call
long i = 0;
System.out.println("threadlocal-leak: driving a 4-thread pool to OOM via leaked ThreadLocal entries");
while (true) {
final long reqId = i++;
pool.execute(() -> handleOneRequest(reqId, payloadSize));
if (reqId % 400 == 0) {
System.out.println("threadlocal-leak: submitted=" + reqId);
}
}
};
}
/**
* Every call creates a fresh ThreadLocal (the bug -- it should be a single static field)
* and never removes it. The whole point: this looks like ordinary per-request code.
*/
private static void handleOneRequest(long requestId, long payloadSize) {
ThreadLocal<byte[]> requestContext = new ThreadLocal<>();
requestContext.set(new byte[(int) payloadSize]);
requestContext.get()[0] = (byte) requestId; // "use" the context
// Missing on purpose: requestContext.remove();
}
}
@@ -0,0 +1,15 @@
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/unbounded-queue-leak.jar
unbounded-queue-leak: driving an unbounded queue to OOM, consumer sleeps 50ms/item
java.lang.OutOfMemoryError: Java heap space
Dumping heap to docs/output/heap.hprof ...
Heap dump file created [175828534 bytes in 0.250 secs]
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:329)
at com.ankurm.oomdemo.UnboundedQueueLeakApplication.lambda$driveLeak$0(UnboundedQueueLeakApplication.java:53)
at com.ankurm.oomdemo.UnboundedQueueLeakApplication$$Lambda/0x000000000b247688.run(Unknown Source)
at org.springframework.boot.SpringApplication.lambda$callRunner$0(SpringApplication.java:788)
at org.springframework.boot.SpringApplication$$Lambda/0x000000000b25ccd0.acceptWithException(Unknown Source)
at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82)
at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
@@ -0,0 +1,23 @@
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
================================================================================================
Problem Suspect 1 One instance of java.util.concurrent.LinkedBlockingQueue loaded by <system
class loader> occupies 79,698,088 (92.41%) bytes. The top consumers of its minimum retained heap
are byte[] (152 instances totaling 79,694,208), java.util.concurrent.LinkedBlockingQueue$Node
(153 instances totaling 3,672), and java.util.concurrent.locks.ReentrantLock$NonfairSync (2
instances totaling 64). The instance is referenced by
com.ankurm.oomdemo.UnboundedQueueLeakApplication$$Lambda+0x000000000b25d428 @ 0xf63bb570 ,
loaded by org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf631bab8 . The memory
is accumulated in one instance of java.util.concurrent.LinkedBlockingQueue$Node , loaded by
<system class loader> , which occupies 1,572,984 (1.82%) bytes. Thread java.lang.Thread @
0xfcc19e38 main has a local variable or reference to java.util.concurrent.LinkedBlockingQueue @
0xff7386e8 which is on the shortest path to java.util.concurrent.LinkedBlockingQueue$Node @
0xfccce5c0 . The thread java.lang.Thread @ 0xfcc19e38 main keeps local variables with total size
632,832 (0.73%) bytes. The top consumers of its minimum retained heap are byte[] (152 instances
totaling 79,694,208), java.util.concurrent.LinkedBlockingQueue$Node (153 instances totaling
3,672), and java.util.concurrent.locks.ReentrantLock$NonfairSync (2 instances totaling 64).
Significant stack frames and local variables com.ankurm.oomdemo.UnboundedQueueLeakApplication.la
mbda$driveLeak$0(Lorg/springframework/boot/ApplicationArguments;)V
(UnboundedQueueLeakApplication.java:53) java.util.concurrent.LinkedBlockingQueue @ 0xff7386e8
retains 79,698,088 (92.41%) bytes The stacktrace of this Thread is available. See stacktrace .
See stacktrace with involved local variables .
+40
View File
@@ -0,0 +1,40 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
<relativePath/>
</parent>
<groupId>com.ankurm</groupId>
<artifactId>unbounded-queue-leak</artifactId>
<version>1.0.0</version>
<name>unbounded-queue-leak</name>
<description>Leak pattern 5: a producer that outruns a consumer into an unbounded queue</description>
<properties>
<java.version>25</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
</dependencies>
<build>
<finalName>unbounded-queue-leak</finalName>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
# summary into docs/output/02-mat-leak-suspects.txt.
#
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
if [ ! -f docs/output/heap.hprof ]; then
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
exit 1
fi
rm -f docs/output/heap_Leak_Suspects.zip
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Drives unbounded-queue-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
# docs/output/01-oom-console.txt from the real run log.
#
# Needs JDK 25 (LTS) on PATH.
set -eu
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
cd "$MODULE_DIR"
mvn -q -B package -DskipTests
mkdir -p docs/output
rm -f docs/output/heap.hprof
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/unbounded-queue-leak.jar"
echo "== running: $CMD =="
LOG="$(mktemp)"
$CMD > "$LOG" 2>&1 || true
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
rm -f "$LOG"
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
@@ -0,0 +1,61 @@
package com.ankurm.oomdemo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import java.util.concurrent.LinkedBlockingQueue;
/**
* Leak pattern 5 of 5: a producer that outruns a consumer into a queue with no capacity bound.
*
* {@code new LinkedBlockingQueue<>()} with no capacity argument is unbounded -- it will accept
* elements until the heap is exhausted, full stop. That is the entire bug. It is also the most
* common of the five in real systems, because it usually arrives disguised as "a queue between
* a fast producer thread and a slow consumer thread," which sounds like ordinary
* producer/consumer code until the consumer falls behind (a slow downstream call, a paused
* consumer during a deploy, a poison message that makes every poll() take longer) and the queue
* becomes the de facto unbounded heap allocator for the whole payload stream.
*/
@SpringBootApplication
public class UnboundedQueueLeakApplication {
public static void main(String[] args) {
SpringApplication.run(UnboundedQueueLeakApplication.class, args);
}
@Bean
ApplicationRunner driveLeak() {
return (ApplicationArguments args) -> {
// The bug: no capacity bound. new LinkedBlockingQueue<>(10_000) would turn this
// whole failure mode into a put() that blocks instead of an OutOfMemoryError.
LinkedBlockingQueue<byte[]> queue = new LinkedBlockingQueue<>();
Thread consumer = new Thread(() -> {
while (true) {
try {
queue.take();
Thread.sleep(50); // the slow downstream call
} catch (InterruptedException e) {
return;
}
}
}, "slow-consumer");
consumer.setDaemon(true);
consumer.start();
long payloadSize = 512 * 1024; // 512 KB per message
long i = 0;
System.out.println("unbounded-queue-leak: driving an unbounded queue to OOM, consumer sleeps 50ms/item");
while (true) {
queue.put(new byte[(int) payloadSize]);
i++;
if (i % 200 == 0) {
System.out.println("unbounded-queue-leak: produced=" + i + " queueDepth=" + queue.size());
}
}
};
}
}