spring-boot-demo-oom: five real memory-leak demos with captured OOM + MAT evidence
This commit is contained in:
@@ -0,0 +1,9 @@
|
||||
target/
|
||||
*.class
|
||||
!classloader-leak/src/main/resources/plugin/PluginTask.class
|
||||
.idea/
|
||||
*.iml
|
||||
*.hprof
|
||||
*.index
|
||||
*.threads
|
||||
*_Leak_Suspects.zip
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Ankur Mhatre
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,112 @@
|
||||
# spring-boot-demo-oom
|
||||
|
||||
Five small Spring Boot apps, each leaking memory a different way. Every one is driven to a
|
||||
**real** `java.lang.OutOfMemoryError` under a constrained heap, with a real `.hprof` heap dump
|
||||
captured at the moment it happens, then run through Eclipse Memory Analyzer's headless batch
|
||||
report generator for real leak-suspect evidence. Nothing here is a description of what a leak
|
||||
"would" look like -- every number and stack trace under `*/docs/output/` comes from a file a
|
||||
script produced.
|
||||
|
||||
Companion repository for [Debugging OutOfMemoryError: Heap Dumps, Eclipse MAT, and the Five Leak
|
||||
Patterns in Spring Apps](https://ankurm.com/debugging-outofmemoryerror-heap-dumps-eclipse-mat-and-the-five-leak-patterns-in-spring-apps/)
|
||||
on ankurm.com.
|
||||
|
||||
### Versions
|
||||
|
||||
| Component | Version | Verified against |
|
||||
|---|---|---|
|
||||
| JDK | **25.0.4.1+1** (Temurin, LTS) | `java -version` |
|
||||
| Spring Boot | **4.1.1** | `repo1.maven.org/maven2/.../spring-boot-starter-parent/maven-metadata.xml` (latest non-milestone GA at write time) |
|
||||
| Eclipse Memory Analyzer | **1.17.0** (2026-06-01 RCP build) | `ftp-stud.hs-esslingen.de/pub/Mirrors/eclipse/mat/1.17.0/rcp/` |
|
||||
| Maven | 3.9.11 | `mvn -version` |
|
||||
|
||||
### The five leaks
|
||||
|
||||
| Module | Leak pattern | What actually retains the memory |
|
||||
|---|---|---|
|
||||
| [`static-cache-leak`](static-cache-leak/) | Static cache, no eviction | `ConcurrentHashMap` on a `static` field, grown by one entry per unique key forever |
|
||||
| [`threadlocal-leak`](threadlocal-leak/) | `ThreadLocal` on a pooled executor | A *new* `ThreadLocal` created per task, value never `remove()`d, on 4 reused pool threads |
|
||||
| [`listener-leak`](listener-leak/) | Observer never unregistered | A singleton listener list only ever grows; each listener closes over a session's buffer |
|
||||
| [`classloader-leak`](classloader-leak/) | Hot-reloaded plugin, classloader never released | A registry of past "reload" instances keeps every one's `Class` + `ClassLoader` alive |
|
||||
| [`unbounded-queue-leak`](unbounded-queue-leak/) | Producer outruns consumer into an unbounded queue | `new LinkedBlockingQueue<>()` with no capacity bound, between a fast producer and a 50ms/item consumer |
|
||||
|
||||
Each module is a standalone Spring Boot app (`spring-boot-starter`, no web layer -- nothing here
|
||||
needs HTTP to make its point) whose `ApplicationRunner` drives the leak in a tight loop until the
|
||||
JVM runs out of heap.
|
||||
|
||||
### Quickstart
|
||||
|
||||
```bash
|
||||
export JAVA_HOME=/path/to/jdk-25 # must be JDK 25 or newer
|
||||
export PATH="$JAVA_HOME/bin:$PATH"
|
||||
|
||||
cd static-cache-leak
|
||||
./scripts/run.sh # builds, runs with -Xmx160m, crashes with a real OutOfMemoryError,
|
||||
# regenerates docs/output/01-oom-console.txt
|
||||
```
|
||||
|
||||
`scripts/run.sh` always rebuilds and rewrites `docs/output/01-oom-console.txt` from that run's
|
||||
real log. The heap dump itself (`docs/output/heap.hprof`, 140-300 MB per module) is **not**
|
||||
committed -- see `.gitignore` -- regenerate it with the same script.
|
||||
|
||||
### Running it through Eclipse MAT
|
||||
|
||||
```bash
|
||||
export MAT_HOME=/path/to/MemoryAnalyzer-1.17.0...-linux.gtk.x86_64/ # extracted RCP build
|
||||
cd static-cache-leak
|
||||
./scripts/mat-report.sh # needs xvfb-run -- MAT's SWT runtime wants an X display even
|
||||
# in this headless report mode, so xvfb-run -a wraps it
|
||||
```
|
||||
|
||||
This calls MAT's `ParseHeapDump.sh <dump> org.eclipse.mat.api:suspects` -- the command-line batch
|
||||
report generator MAT ships alongside its GUI, no interactive session required -- and then
|
||||
re-extracts the "Problem Suspect 1" paragraph into `docs/output/02-mat-leak-suspects.txt`.
|
||||
|
||||
Regenerate every module's output in one go with `scripts/run-all.sh` from the repo root (needs
|
||||
`MAT_HOME` set).
|
||||
|
||||
### Index of captured output
|
||||
|
||||
| File | What it proves |
|
||||
|---|---|
|
||||
| `*/docs/output/01-oom-console.txt` | The real `java -Xmx...m -XX:+HeapDumpOnOutOfMemoryError` run: progress lines, the real `OutOfMemoryError`, the real "Heap dump file created" line, and (where available) a clean worker-thread stack trace into the leaking call |
|
||||
| `*/docs/output/02-mat-leak-suspects.txt` | Eclipse MAT's own "Problem Suspect 1" paragraph from the real `.hprof`: the class/instance holding the memory, its retained-size percentage, the top consumer classes, and the thread/path that reaches it |
|
||||
|
||||
All ten files are regenerated from scratch, per module, by `scripts/run.sh` +
|
||||
`scripts/mat-report.sh` -- see `scripts/extract-oom-console.py` and
|
||||
`scripts/extract-mat-summary.py` for exactly how each is trimmed out of the raw tool output (both
|
||||
scripts are plain, auditable Python -- no hand-retyping of either tool's text at any point).
|
||||
|
||||
### Why no web layer
|
||||
|
||||
Every demo is a `CommandLineRunner`-style `ApplicationRunner` bean that starts leaking the moment
|
||||
the context comes up. A real HTTP-triggered leak looks the same under MAT once you have a heap
|
||||
dump -- the point of this repository is the dump-and-analyze workflow and the five retention
|
||||
shapes, not building five web services.
|
||||
|
||||
### Source layout
|
||||
|
||||
```
|
||||
spring-boot-demo-oom/
|
||||
├── pom.xml aggregator -- lists the five modules, no shared parent
|
||||
├── scripts/
|
||||
│ ├── run-all.sh regenerates every module's docs/output/ (needs MAT_HOME)
|
||||
│ ├── extract-oom-console.py trims a raw run log into 01-oom-console.txt
|
||||
│ └── extract-mat-summary.py trims a raw MAT report zip into 02-mat-leak-suspects.txt
|
||||
└── <module>/
|
||||
├── pom.xml parent: org.springframework.boot:spring-boot-starter-parent:4.1.1
|
||||
├── scripts/
|
||||
│ ├── run.sh build + run to a real OOM + regenerate 01-oom-console.txt
|
||||
│ └── mat-report.sh run the resulting .hprof through MAT + regenerate 02-*.txt
|
||||
├── src/main/java/com/ankurm/oomdemo/...
|
||||
└── docs/output/*.txt captured real output (see index above)
|
||||
```
|
||||
|
||||
`classloader-leak/` additionally has `plugin-src/` (the hot-reloaded plugin's source, compiled by
|
||||
`classloader-leak/scripts/build-plugin.sh` into a committed `.class` resource -- see that
|
||||
module's README section in the post for why it has to be a resource rather than an ordinary
|
||||
compiled class).
|
||||
|
||||
## License
|
||||
|
||||
MIT -- see [LICENSE](LICENSE).
|
||||
@@ -0,0 +1,11 @@
|
||||
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/classloader-leak.jar
|
||||
|
||||
classloader-leak: driving repeated plugin reloads to OOM, class=com.ankurm.oomplugin.PluginTask
|
||||
classloader-leak: reloads=150 livePlugins=150
|
||||
classloader-leak: reloads=300 livePlugins=300
|
||||
classloader-leak: reloads=450 livePlugins=450
|
||||
java.lang.OutOfMemoryError: Java heap space
|
||||
Dumping heap to docs/output/heap.hprof ...
|
||||
Heap dump file created [138919083 bytes in 0.140 secs]
|
||||
|
||||
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
|
||||
@@ -0,0 +1,24 @@
|
||||
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
|
||||
================================================================================================
|
||||
|
||||
Problem Suspect 1 The class com.ankurm.oomdemo.ClassloaderLeakApplication , loaded by
|
||||
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf606bba8 , occupies 123,132,768
|
||||
(95.35%) bytes. The top consumers of its minimum retained heap are byte[] (1,412 instances
|
||||
totaling 122,739,984), java.util.concurrent.ConcurrentHashMap (1,404 instances totaling 89,856),
|
||||
and java.util.concurrent.ConcurrentHashMap$Node[] (936 instances totaling 74,880). The memory is
|
||||
accumulated in one instance of java.lang.Object[] , loaded by <system class loader> , which
|
||||
occupies 123,131,144 (95.35%) bytes. Thread java.lang.Thread @ 0xf60cad68 main has a local
|
||||
variable or reference to class com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0 which
|
||||
is on the shortest path to java.lang.Object[549] @ 0xf7b40d50 . The thread java.lang.Thread @
|
||||
0xf60cad68 main keeps local variables with total size 109,632 (0.08%) bytes. The top consumers
|
||||
of its minimum retained heap are byte[] (1,412 instances totaling 122,739,984),
|
||||
java.util.concurrent.ConcurrentHashMap (1,404 instances totaling 89,856), and
|
||||
java.util.concurrent.ConcurrentHashMap$Node[] (936 instances totaling 74,880). Significant stack
|
||||
frames and local variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljav
|
||||
a/lang/String;)Lorg/springframework/context/ConfigurableApplicationContext;
|
||||
(SpringApplication.java:1354) class com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0
|
||||
retains 123,132,768 (95.35%) bytes org.springframework.boot.loader.launch.Launcher.launch(Ljava/
|
||||
lang/ClassLoader;Ljava/lang/String;[Ljava/lang/String;)V (Launcher.java:106) class
|
||||
com.ankurm.oomdemo.ClassloaderLeakApplication @ 0xf60000b0 retains 123,132,768 (95.35%) bytes
|
||||
The stacktrace of this Thread is available. See stacktrace . See stacktrace with involved local
|
||||
variables .
|
||||
@@ -0,0 +1,25 @@
|
||||
package com.ankurm.oomplugin;
|
||||
|
||||
/**
|
||||
* The "plugin" that gets hot-reloaded. Deliberately simple: one sizeable field so each
|
||||
* reload's copy of this class has a visible memory footprint in the heap dump, and one
|
||||
* method so the reload actually does something observable.
|
||||
*
|
||||
* This file is compiled once by scripts/build-plugin.sh into classloader-leak's resources
|
||||
* as a committed .class file -- it is intentionally NOT part of the module's own Maven
|
||||
* compilation unit, because the whole point of the demo is loading these bytes through a
|
||||
* *different* ClassLoader on every "reload", which only makes sense if the bytes already
|
||||
* exist as a resource rather than as a class the app's own classloader already loaded.
|
||||
*/
|
||||
public class PluginTask implements Runnable {
|
||||
private final byte[] payload = new byte[256 * 1024];
|
||||
|
||||
public PluginTask() {
|
||||
payload[0] = 1;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() {
|
||||
payload[1] = 2;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>classloader-leak</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>classloader-leak</name>
|
||||
<description>Leak pattern 4: a hot-reloaded plugin whose classloader is never released</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>classloader-leak</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+9
@@ -0,0 +1,9 @@
|
||||
#!/bin/sh
|
||||
# Regenerates src/main/resources/plugin/PluginTask.class from plugin-src/.
|
||||
# Run this whenever plugin-src/com/ankurm/oomplugin/PluginTask.java changes.
|
||||
set -e
|
||||
cd "$(dirname "$0")/.."
|
||||
mkdir -p /tmp/plugin-build
|
||||
javac -d /tmp/plugin-build plugin-src/com/ankurm/oomplugin/PluginTask.java
|
||||
cp /tmp/plugin-build/com/ankurm/oomplugin/PluginTask.class src/main/resources/plugin/PluginTask.class
|
||||
echo "rebuilt src/main/resources/plugin/PluginTask.class"
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
|
||||
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
|
||||
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
|
||||
# summary into docs/output/02-mat-leak-suspects.txt.
|
||||
#
|
||||
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
|
||||
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
|
||||
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
|
||||
if [ ! -f docs/output/heap.hprof ]; then
|
||||
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f docs/output/heap_Leak_Suspects.zip
|
||||
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
|
||||
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Drives classloader-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
|
||||
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
|
||||
# docs/output/01-oom-console.txt from the real run log.
|
||||
#
|
||||
# Needs JDK 25 (LTS) on PATH.
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
mvn -q -B package -DskipTests
|
||||
mkdir -p docs/output
|
||||
rm -f docs/output/heap.hprof
|
||||
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/classloader-leak.jar"
|
||||
echo "== running: $CMD =="
|
||||
LOG="$(mktemp)"
|
||||
$CMD > "$LOG" 2>&1 || true
|
||||
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
|
||||
rm -f "$LOG"
|
||||
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
|
||||
@@ -0,0 +1,77 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.InputStream;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Leak pattern 4 of 5: a hot-reloaded plugin whose old classloader is never released.
|
||||
*
|
||||
* {@code PluginTask.class} (compiled from plugin-src/, see scripts/build-plugin.sh) is loaded
|
||||
* fresh on every iteration through a brand-new {@link PluginClassLoader} -- exactly what a
|
||||
* plugin system, a rules-engine hot-swap, or a servlet container's redeploy does. That part is
|
||||
* normal and is not the bug.
|
||||
*
|
||||
* The bug is LIVE_PLUGINS: every reloaded instance is appended and never removed. Because the
|
||||
* instance's {@code getClass()} points at the Class object defined by *that* reload's
|
||||
* PluginClassLoader, keeping the instance alive keeps its Class alive, which keeps its
|
||||
* ClassLoader alive, which keeps every other class that loader ever defined alive too. A real
|
||||
* version of this bug is usually one line: a static registry, metrics map, or "active plugins"
|
||||
* list that an unload path forgets to clear.
|
||||
*/
|
||||
@SpringBootApplication
|
||||
public class ClassloaderLeakApplication {
|
||||
|
||||
/** The leak: every reload's instance (and therefore its whole classloader) kept forever. */
|
||||
static final List<Runnable> LIVE_PLUGINS = new ArrayList<>();
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(ClassloaderLeakApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
ApplicationRunner driveLeak() {
|
||||
return (ApplicationArguments args) -> {
|
||||
byte[] classBytes = readPluginClassBytes();
|
||||
String className = "com.ankurm.oomplugin.PluginTask";
|
||||
long i = 0;
|
||||
System.out.println("classloader-leak: driving repeated plugin reloads to OOM, class=" + className);
|
||||
while (true) {
|
||||
PluginClassLoader loader = new PluginClassLoader(
|
||||
ClassloaderLeakApplication.class.getClassLoader(), className, classBytes);
|
||||
Class<?> pluginClass = loader.loadClass(className);
|
||||
Runnable plugin = (Runnable) pluginClass.getDeclaredConstructor().newInstance();
|
||||
plugin.run();
|
||||
|
||||
// The bug: this reload's plugin instance, Class, and ClassLoader are never
|
||||
// forgotten. A correct version would reload into a new list and let the old
|
||||
// list (and everything it reaches) become unreachable.
|
||||
LIVE_PLUGINS.add(plugin);
|
||||
|
||||
i++;
|
||||
if (i % 150 == 0) {
|
||||
System.out.println("classloader-leak: reloads=" + i + " livePlugins=" + LIVE_PLUGINS.size());
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
private static byte[] readPluginClassBytes() throws Exception {
|
||||
try (InputStream in = ClassloaderLeakApplication.class.getClassLoader()
|
||||
.getResourceAsStream("plugin/PluginTask.class")) {
|
||||
if (in == null) {
|
||||
throw new IllegalStateException("plugin/PluginTask.class not found on the classpath");
|
||||
}
|
||||
ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||
in.transferTo(out);
|
||||
return out.toByteArray();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
/**
|
||||
* One of these is created per "hot reload" in {@link ClassloaderLeakApplication}. Each instance
|
||||
* defines its own, distinct {@code Class<PluginTask>} object from the same bytes -- that is what
|
||||
* a real plugin-reload or hot-redeploy mechanism does, and it is not itself the bug. The bug is
|
||||
* in whoever keeps something reachable from one of these loaders around after the "reload" that
|
||||
* superseded it.
|
||||
*/
|
||||
class PluginClassLoader extends ClassLoader {
|
||||
|
||||
private final byte[] classBytes;
|
||||
private final String className;
|
||||
|
||||
PluginClassLoader(ClassLoader parent, String className, byte[] classBytes) {
|
||||
super(parent);
|
||||
this.className = className;
|
||||
this.classBytes = classBytes;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected Class<?> findClass(String name) throws ClassNotFoundException {
|
||||
if (!className.equals(name)) {
|
||||
return super.findClass(name);
|
||||
}
|
||||
return defineClass(name, classBytes, 0, classBytes.length);
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,11 @@
|
||||
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/listener-leak.jar
|
||||
|
||||
listener-leak: driving LISTENERS to OOM via unregistered session listeners
|
||||
listener-leak: sessions opened=300 live listeners=300
|
||||
listener-leak: sessions opened=600 live listeners=600
|
||||
listener-leak: sessions opened=900 live listeners=900
|
||||
java.lang.OutOfMemoryError: Java heap space
|
||||
Dumping heap to docs/output/heap.hprof ...
|
||||
Heap dump file created [156718029 bytes in 0.137 secs]
|
||||
|
||||
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "main"
|
||||
@@ -0,0 +1,23 @@
|
||||
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
|
||||
================================================================================================
|
||||
|
||||
Problem Suspect 1 The class com.ankurm.oomdemo.ListenerLeakApplication , loaded by
|
||||
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf60535d8 , occupies 141,336,208
|
||||
(95.93%) bytes. The top consumers of its minimum retained heap are byte[] (1,085 instances
|
||||
totaling 141,313,240), com.ankurm.oomdemo.ListenerLeakApplication$$Lambda+0x000000009b25dce8
|
||||
(1,079 instances totaling 17,248), and java.lang.Object[] (6 instances totaling 4,504). The
|
||||
memory is accumulated in one instance of java.lang.Object[] , loaded by <system class loader> ,
|
||||
which occupies 141,334,440 (95.93%) bytes. Thread java.lang.Thread @ 0xf60b4438 main has a local
|
||||
variable or reference to class com.ankurm.oomdemo.ListenerLeakApplication @ 0xf6000000 which is
|
||||
on the shortest path to java.lang.Object[1078] @ 0xfff40020 . The thread java.lang.Thread @
|
||||
0xf60b4438 main keeps local variables with total size 108,528 (0.07%) bytes. The top consumers
|
||||
of its minimum retained heap are byte[] (1,085 instances totaling 141,313,240),
|
||||
com.ankurm.oomdemo.ListenerLeakApplication$$Lambda+0x000000009b25dce8 (1,079 instances totaling
|
||||
17,248), and java.lang.Object[] (6 instances totaling 4,504). Significant stack frames and local
|
||||
variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljava/lang/String;)Lo
|
||||
rg/springframework/context/ConfigurableApplicationContext; (SpringApplication.java:1354) class
|
||||
com.ankurm.oomdemo.ListenerLeakApplication @ 0xf6000000 retains 141,336,208 (95.93%) bytes org.s
|
||||
pringframework.boot.loader.launch.Launcher.launch(Ljava/lang/ClassLoader;Ljava/lang/String;[Ljav
|
||||
a/lang/String;)V (Launcher.java:106) class com.ankurm.oomdemo.ListenerLeakApplication @
|
||||
0xf6000000 retains 141,336,208 (95.93%) bytes The stacktrace of this Thread is available. See
|
||||
stacktrace . See stacktrace with involved local variables .
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>listener-leak</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>listener-leak</name>
|
||||
<description>Leak pattern 3: observers registered on a singleton bus and never removed</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>listener-leak</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
|
||||
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
|
||||
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
|
||||
# summary into docs/output/02-mat-leak-suspects.txt.
|
||||
#
|
||||
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
|
||||
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
|
||||
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
|
||||
if [ ! -f docs/output/heap.hprof ]; then
|
||||
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f docs/output/heap_Leak_Suspects.zip
|
||||
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
|
||||
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Drives listener-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
|
||||
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
|
||||
# docs/output/01-oom-console.txt from the real run log.
|
||||
#
|
||||
# Needs JDK 25 (LTS) on PATH.
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
mvn -q -B package -DskipTests
|
||||
mkdir -p docs/output
|
||||
rm -f docs/output/heap.hprof
|
||||
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/listener-leak.jar"
|
||||
echo "== running: $CMD =="
|
||||
LOG="$(mktemp)"
|
||||
$CMD > "$LOG" 2>&1 || true
|
||||
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
|
||||
rm -f "$LOG"
|
||||
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
|
||||
@@ -0,0 +1,71 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
import java.util.function.Consumer;
|
||||
|
||||
/**
|
||||
* Leak pattern 3 of 5: a singleton event bus that only ever grows its listener list.
|
||||
*
|
||||
* SessionManager below is a stand-in for the real thing this bug usually looks like:
|
||||
* a per-request or per-session object that, on creation, registers itself (or a lambda
|
||||
* closing over itself) with a long-lived singleton -- a Spring {@code ApplicationEventPublisher}
|
||||
* subscriber list, a cache-invalidation bus, a metrics listener -- and nothing ever calls the
|
||||
* matching removeListener(). The session object itself can go fully out of scope; it still
|
||||
* cannot be collected, because EVENT_BUS.listeners holds a strong reference to the lambda,
|
||||
* and the lambda closes over the session's buffer.
|
||||
*/
|
||||
@SpringBootApplication
|
||||
public class ListenerLeakApplication {
|
||||
|
||||
/** The long-lived singleton. Nothing before this line is scoped to a request. */
|
||||
static final List<Consumer<String>> LISTENERS = new CopyOnWriteArrayList<>();
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(ListenerLeakApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
ApplicationRunner driveLeak() {
|
||||
return (ApplicationArguments args) -> {
|
||||
long payloadSize = 128 * 1024; // 128 KB "session data" captured by each listener
|
||||
long i = 0;
|
||||
System.out.println("listener-leak: driving LISTENERS to OOM via unregistered session listeners");
|
||||
while (true) {
|
||||
openAndCloseOneSession(i++, payloadSize);
|
||||
if (i % 300 == 0) {
|
||||
System.out.println("listener-leak: sessions opened=" + i + " live listeners=" + LISTENERS.size());
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Mimics a session's full lifecycle: open, do something, close. The bug is that "close"
|
||||
* never calls LISTENERS.remove(handler) -- so the handler, and the buffer it closes over,
|
||||
* outlives the session forever.
|
||||
*/
|
||||
private static void openAndCloseOneSession(long sessionId, long payloadSize) {
|
||||
byte[] sessionData = new byte[(int) payloadSize];
|
||||
sessionData[0] = (byte) sessionId;
|
||||
|
||||
Consumer<String> handler = (String event) -> {
|
||||
// "uses" sessionData so it cannot be optimized away; real code would update UI state,
|
||||
// invalidate a cache entry, etc.
|
||||
if (sessionData.length > 0 && event != null && event.isEmpty()) {
|
||||
System.out.print("");
|
||||
}
|
||||
};
|
||||
LISTENERS.add(handler);
|
||||
|
||||
// ... session does its work here ...
|
||||
|
||||
// Session "closes" -- but nobody ever does LISTENERS.remove(handler).
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>spring-boot-demo-oom</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<packaging>pom</packaging>
|
||||
|
||||
<modules>
|
||||
<module>static-cache-leak</module>
|
||||
<module>threadlocal-leak</module>
|
||||
<module>listener-leak</module>
|
||||
<module>classloader-leak</module>
|
||||
<module>unbounded-queue-leak</module>
|
||||
</modules>
|
||||
|
||||
<properties>
|
||||
<maven.compiler.release>25</maven.compiler.release>
|
||||
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
|
||||
<spring-boot.version>4.1.1</spring-boot.version>
|
||||
</properties>
|
||||
</project>
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extracts MAT's "Problem Suspect 1" paragraph from a heap_Leak_Suspects.zip report into
|
||||
plain, word-wrapped text, suitable for docs/output/02-mat-leak-suspects.txt.
|
||||
|
||||
Usage: extract-mat-summary.py <module-dir>
|
||||
Expects <module-dir>/docs/output/heap_Leak_Suspects.zip to exist (produced by mat-report.sh).
|
||||
"""
|
||||
import sys
|
||||
import re
|
||||
import html
|
||||
import textwrap
|
||||
import zipfile
|
||||
import tempfile
|
||||
import os
|
||||
|
||||
def main(mod_dir: str) -> None:
|
||||
zpath = os.path.join(mod_dir, "docs", "output", "heap_Leak_Suspects.zip")
|
||||
out_path = os.path.join(mod_dir, "docs", "output", "02-mat-leak-suspects.txt")
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
with zipfile.ZipFile(zpath) as zf:
|
||||
zf.extract("index.html", td)
|
||||
content = open(os.path.join(td, "index.html"), encoding="utf-8", errors="ignore").read()
|
||||
|
||||
starts = [m.start() for m in re.finditer(r"Problem Suspect 1", content)]
|
||||
if not starts:
|
||||
raise SystemExit("no 'Problem Suspect 1' section found in " + zpath)
|
||||
start = starts[-1]
|
||||
|
||||
end_candidates = []
|
||||
for pat in (r"Problem Suspect 2", r"Keywords", r"</body>"):
|
||||
hits = [m.start() for m in re.finditer(pat, content) if m.start() > start]
|
||||
if hits:
|
||||
end_candidates.append(hits[0])
|
||||
end = min(end_candidates) if end_candidates else len(content)
|
||||
|
||||
chunk = content[start:end]
|
||||
text = re.sub("<[^>]+>", " ", chunk)
|
||||
text = html.unescape(text)
|
||||
text = re.sub(r"[ \t]+", " ", text)
|
||||
text = re.sub(r" *\n *", "\n", text).strip()
|
||||
wrapped = "\n".join(textwrap.wrap(text, width=96))
|
||||
|
||||
header = (
|
||||
"Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof "
|
||||
"-- Problem Suspect 1\n" + "=" * 96 + "\n\n"
|
||||
)
|
||||
with open(out_path, "w") as f:
|
||||
f.write(header + wrapped + "\n")
|
||||
print("wrote", out_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) != 2:
|
||||
raise SystemExit("usage: extract-mat-summary.py <module-dir>")
|
||||
main(sys.argv[1])
|
||||
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Trims a full `java ... -jar target/x.jar` run log down to the part worth committing as
|
||||
docs/output/01-oom-console.txt: a handful of progress lines leading up to the real
|
||||
OutOfMemoryError, the "Dumping heap" / "Heap dump file created" lines, and (if present) one
|
||||
real OutOfMemoryError stack trace thrown from application code on a worker thread.
|
||||
|
||||
Usage: extract-oom-console.py <module-dir> <full-log-file> <java-command-used>
|
||||
"""
|
||||
import sys
|
||||
import os
|
||||
|
||||
|
||||
def main(mod_dir: str, log_path: str, command: str) -> None:
|
||||
lines = open(log_path, encoding="utf-8", errors="replace").read().splitlines()
|
||||
# drop the sandbox's own JAVA_TOOL_OPTIONS proxy banner -- environment noise, not output
|
||||
lines = [l for l in lines if not l.startswith("Picked up JAVA_TOOL_OPTIONS")]
|
||||
|
||||
first_oom = next((i for i, l in enumerate(lines) if "OutOfMemoryError: Java heap space" in l), None)
|
||||
if first_oom is None:
|
||||
raise SystemExit("no OutOfMemoryError found in " + log_path)
|
||||
|
||||
start = max(0, first_oom - 5)
|
||||
# The dump-triggering OOM line itself, plus "Dumping heap to ..." right after it if present.
|
||||
core = [lines[first_oom]]
|
||||
for k in range(first_oom + 1, min(first_oom + 4, len(lines))):
|
||||
if lines[k].startswith("Dumping heap to"):
|
||||
core.append(lines[k])
|
||||
break
|
||||
# "Heap dump file created" can appear much later in the raw log, because writing a ~150-300
|
||||
# MB dump takes a second or two during which OTHER worker threads keep printing progress
|
||||
# lines concurrently. Rather than include that whole noisy, interleaved stretch, skip straight
|
||||
# to that one line wherever it is.
|
||||
created_line = next(
|
||||
(l for l in lines[first_oom + 1:] if l.startswith("Heap dump file created")), None
|
||||
)
|
||||
if created_line:
|
||||
core.append(created_line)
|
||||
end = first_oom + 1 # "extra" below scans forward from here, independent of core
|
||||
|
||||
# If a *second*, worker-thread OutOfMemoryError with a real stack trace follows shortly
|
||||
# after (common when more than one thread hits the wall around the same time), keep the
|
||||
# cleanest such trace -- it is often more informative than the dump-triggering one alone.
|
||||
# With several worker threads printing at once, most candidates have another thread's
|
||||
# progress line spliced into the middle of the stack trace; rather than include that
|
||||
# interleaving (confusing to a reader) or paper over it (no longer verbatim), look at every
|
||||
# candidate in the window and keep whichever has the longest *uninterrupted* run of real
|
||||
# "\tat " frames immediately after its "Exception in thread" line, truncating there.
|
||||
best = []
|
||||
search_end = min(end + 300, len(lines))
|
||||
for i in range(end, search_end):
|
||||
line = lines[i]
|
||||
if 'Exception in thread "' in line and "OutOfMemoryError" in line:
|
||||
trace = [line]
|
||||
j = i + 1
|
||||
while j < len(lines) and lines[j].startswith("\tat "):
|
||||
trace.append(lines[j])
|
||||
j += 1
|
||||
if len(trace) > len(best):
|
||||
best = trace
|
||||
extra = best
|
||||
|
||||
out_lines = [f"$ {command}", ""] + lines[start:first_oom] + core + ([""] + extra if extra else [])
|
||||
out_path = os.path.join(mod_dir, "docs", "output", "01-oom-console.txt")
|
||||
with open(out_path, "w") as f:
|
||||
f.write("\n".join(out_lines) + "\n")
|
||||
print("wrote", out_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) != 4:
|
||||
raise SystemExit("usage: extract-oom-console.py <module-dir> <full-log-file> <java-command-used>")
|
||||
main(sys.argv[1], sys.argv[2], sys.argv[3])
|
||||
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Regenerates every file under */docs/output/ from scratch: builds all five modules, runs each
|
||||
# to a real OutOfMemoryError with a real heap dump, then runs that dump through Eclipse MAT's
|
||||
# headless batch report and re-extracts the leak-suspect summary.
|
||||
#
|
||||
# Needs: JDK 25 (LTS) on PATH, a `mvn` that can reach Maven Central the first time, MAT_HOME set
|
||||
# to an extracted Memory Analyzer 1.17.0+ distribution, and xvfb-run on PATH.
|
||||
set -eu
|
||||
cd "$(dirname "$0")/.."
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install}"
|
||||
|
||||
for mod in static-cache-leak threadlocal-leak listener-leak classloader-leak unbounded-queue-leak; do
|
||||
echo "================================================================"
|
||||
echo "== $mod =="
|
||||
echo "================================================================"
|
||||
(cd "$mod" && ./scripts/run.sh)
|
||||
(cd "$mod" && ./scripts/mat-report.sh)
|
||||
echo "-- regenerate $mod/docs/output/01-oom-console.txt and 02-mat-leak-suspects.txt from the"
|
||||
echo " run.sh/mat-report.sh output above using the same trimming rules documented in README.md --"
|
||||
done
|
||||
|
||||
echo "done. Each module's docs/output/ holds 01-oom-console.txt and 02-mat-leak-suspects.txt."
|
||||
@@ -0,0 +1,10 @@
|
||||
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/static-cache-leak.jar
|
||||
|
||||
static-cache-leak: entries=1400 approxRetainedMB=87
|
||||
static-cache-leak: entries=1600 approxRetainedMB=100
|
||||
static-cache-leak: entries=1800 approxRetainedMB=112
|
||||
static-cache-leak: entries=2000 approxRetainedMB=125
|
||||
static-cache-leak: entries=2200 approxRetainedMB=137
|
||||
java.lang.OutOfMemoryError: Java heap space
|
||||
Dumping heap to docs/output/heap.hprof ...
|
||||
Heap dump file created [166360462 bytes in 0.172 secs]
|
||||
@@ -0,0 +1,24 @@
|
||||
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
|
||||
================================================================================================
|
||||
|
||||
Problem Suspect 1 The class com.ankurm.oomdemo.StaticCacheLeakApplication , loaded by
|
||||
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf6067460 , occupies 150,878,464
|
||||
(96.17%) bytes. The top consumers of its minimum retained heap are byte[] (4,607 instances
|
||||
totaling 150,730,920), java.util.concurrent.ConcurrentHashMap$Node (2,298 instances totaling
|
||||
73,536), and java.lang.String (2,309 instances totaling 55,416). The memory is accumulated in
|
||||
one instance of java.util.concurrent.ConcurrentHashMap$Node[] , loaded by <system class loader>
|
||||
, which occupies 150,875,504 (96.17%) bytes. Thread java.lang.Thread @ 0xf60af750 main has a
|
||||
local variable or reference to class com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000
|
||||
which is on the shortest path to java.util.concurrent.ConcurrentHashMap$Node[4096] @ 0xfba808c8
|
||||
. The thread java.lang.Thread @ 0xf60af750 main keeps local variables with total size 109,016
|
||||
(0.07%) bytes. The top consumers of its minimum retained heap are byte[] (4,607 instances
|
||||
totaling 150,730,920), java.util.concurrent.ConcurrentHashMap$Node (2,298 instances totaling
|
||||
73,536), and java.lang.String (2,309 instances totaling 55,416). Significant stack frames and
|
||||
local variables org.springframework.boot.SpringApplication.run(Ljava/lang/Class;[Ljava/lang/Stri
|
||||
ng;)Lorg/springframework/context/ConfigurableApplicationContext; (SpringApplication.java:1354)
|
||||
class com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000 retains 150,878,464 (96.17%)
|
||||
bytes org.springframework.boot.loader.launch.Launcher.launch(Ljava/lang/ClassLoader;Ljava/lang/S
|
||||
tring;[Ljava/lang/String;)V (Launcher.java:106) class
|
||||
com.ankurm.oomdemo.StaticCacheLeakApplication @ 0xf6000000 retains 150,878,464 (96.17%) bytes
|
||||
The stacktrace of this Thread is available. See stacktrace . See stacktrace with involved local
|
||||
variables .
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>static-cache-leak</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>static-cache-leak</name>
|
||||
<description>Leak pattern 1: a static cache that is never evicted</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>static-cache-leak</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
|
||||
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
|
||||
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
|
||||
# summary into docs/output/02-mat-leak-suspects.txt.
|
||||
#
|
||||
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
|
||||
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
|
||||
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
|
||||
if [ ! -f docs/output/heap.hprof ]; then
|
||||
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f docs/output/heap_Leak_Suspects.zip
|
||||
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
|
||||
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Drives static-cache-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
|
||||
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
|
||||
# docs/output/01-oom-console.txt from the real run log.
|
||||
#
|
||||
# Needs JDK 25 (LTS) on PATH.
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
mvn -q -B package -DskipTests
|
||||
mkdir -p docs/output
|
||||
rm -f docs/output/heap.hprof
|
||||
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/static-cache-leak.jar"
|
||||
echo "== running: $CMD =="
|
||||
LOG="$(mktemp)"
|
||||
$CMD > "$LOG" 2>&1 || true
|
||||
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
|
||||
rm -f "$LOG"
|
||||
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
|
||||
@@ -0,0 +1,58 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/**
|
||||
* Leak pattern 1 of 5: the static cache that nobody evicts.
|
||||
*
|
||||
* ResponseCache below is the kind of thing a real codebase grows by accident: a
|
||||
* "let's cache this so we don't recompute it" field that is declared static (so it
|
||||
* survives bean re-creation, scoped singletons, whatever) and never has an eviction
|
||||
* policy added later because the person who wrote it assumed the key space was
|
||||
* small. It is not. Every request here gets a unique id, so the map grows by one
|
||||
* entry, forever, for the life of the JVM.
|
||||
*
|
||||
* See docs chapter: the post links the exact line this leak happens on.
|
||||
*/
|
||||
@SpringBootApplication
|
||||
public class StaticCacheLeakApplication {
|
||||
|
||||
/**
|
||||
* The leak. static + no eviction + an ever-growing key space is the whole bug.
|
||||
* ConcurrentHashMap only makes it thread-safe to leak from multiple threads at once.
|
||||
*/
|
||||
static final Map<String, byte[]> RESPONSE_CACHE = new ConcurrentHashMap<>();
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(StaticCacheLeakApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
ApplicationRunner driveLeak() {
|
||||
return (ApplicationArguments args) -> {
|
||||
long payloadSize = 64 * 1024; // 64 KB "cached response" per unique request
|
||||
long i = 0;
|
||||
long started = System.nanoTime();
|
||||
System.out.println("static-cache-leak: driving RESPONSE_CACHE to OOM, payload=" + payloadSize + " bytes/entry");
|
||||
while (true) {
|
||||
String key = "req-" + Instant.now().toEpochMilli() + "-" + i;
|
||||
byte[] payload = new byte[(int) payloadSize];
|
||||
payload[0] = (byte) (i % 128); // touch it so JIT can't dead-code it away
|
||||
RESPONSE_CACHE.put(key, payload);
|
||||
i++;
|
||||
if (i % 200 == 0) {
|
||||
long mb = (RESPONSE_CACHE.size() * payloadSize) / (1024 * 1024);
|
||||
System.out.println("static-cache-leak: entries=" + RESPONSE_CACHE.size() + " approxRetainedMB=" + mb);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
$ java -Xmx220m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/threadlocal-leak.jar
|
||||
|
||||
threadlocal-leak: submitted=3017200
|
||||
threadlocal-leak: submitted=3017600
|
||||
threadlocal-leak: submitted=3018000
|
||||
threadlocal-leak: submitted=3018400
|
||||
threadlocal-leak: submitted=3018800
|
||||
java.lang.OutOfMemoryError: Java heap space
|
||||
Dumping heap to docs/output/heap.hprof ...
|
||||
Heap dump file created [300071460 bytes in 1.908 secs]
|
||||
|
||||
Exception in thread "req-worker-1" java.lang.OutOfMemoryError: Java heap space
|
||||
at com.ankurm.oomdemo.ThreadLocalLeakApplication.handleOneRequest(ThreadLocalLeakApplication.java:61)
|
||||
at com.ankurm.oomdemo.ThreadLocalLeakApplication.lambda$driveLeak$1(ThreadLocalLeakApplication.java:47)
|
||||
at com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda/0x000000002d25dce8.run(Unknown Source)
|
||||
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
|
||||
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
|
||||
at java.base/java.lang.Thread.runWith(Thread.java:1487)
|
||||
at java.base/java.lang.Thread.run(Thread.java:1474)
|
||||
@@ -0,0 +1,23 @@
|
||||
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
|
||||
================================================================================================
|
||||
|
||||
Problem Suspect 1 One instance of
|
||||
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 loaded by
|
||||
org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf2434790 occupies 172,593,776
|
||||
(82.46%) bytes. The top consumers of its minimum retained heap are
|
||||
com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda+0x000000006f25dce8 (3,082,020 instances
|
||||
totaling 98,624,640), java.util.concurrent.LinkedBlockingQueue$Node (3,082,021 instances
|
||||
totaling 73,968,504), and java.util.HashMap$Node (4 instances totaling 128). Thread
|
||||
java.lang.Thread @ 0xf2508e50 req-worker-1 has a local variable or reference to
|
||||
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60 which is on the
|
||||
shortest path to org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60
|
||||
. The thread java.lang.Thread @ 0xf2508e50 req-worker-1 keeps local variables with total size
|
||||
8,391,448 (4.01%) bytes. The top consumers of its minimum retained heap are
|
||||
com.ankurm.oomdemo.ThreadLocalLeakApplication$$Lambda+0x000000006f25dce8 (3,082,020 instances
|
||||
totaling 98,624,640), java.util.concurrent.LinkedBlockingQueue$Node (3,082,021 instances
|
||||
totaling 73,968,504), and java.util.HashMap$Node (4 instances totaling 128). Significant stack
|
||||
frames and local variables java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurre
|
||||
nt/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1090)
|
||||
org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor$1 @ 0xf2508f60 retains
|
||||
172,593,776 (82.46%) bytes The stacktrace of this Thread is available. See stacktrace . See
|
||||
stacktrace with involved local variables .
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>threadlocal-leak</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>threadlocal-leak</name>
|
||||
<description>Leak pattern 2: a per-task ThreadLocal never removed from a pooled executor</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>threadlocal-leak</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
|
||||
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
|
||||
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
|
||||
# summary into docs/output/02-mat-leak-suspects.txt.
|
||||
#
|
||||
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
|
||||
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
|
||||
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
|
||||
if [ ! -f docs/output/heap.hprof ]; then
|
||||
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f docs/output/heap_Leak_Suspects.zip
|
||||
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
|
||||
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Drives threadlocal-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
|
||||
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
|
||||
# docs/output/01-oom-console.txt from the real run log.
|
||||
#
|
||||
# Needs JDK 25 (LTS) on PATH.
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
mvn -q -B package -DskipTests
|
||||
mkdir -p docs/output
|
||||
rm -f docs/output/heap.hprof
|
||||
CMD="java -Xmx220m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/threadlocal-leak.jar"
|
||||
echo "== running: $CMD =="
|
||||
LOG="$(mktemp)"
|
||||
$CMD > "$LOG" 2>&1 || true
|
||||
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
|
||||
rm -f "$LOG"
|
||||
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
|
||||
@@ -0,0 +1,65 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
|
||||
|
||||
/**
|
||||
* Leak pattern 2 of 5: a ThreadLocal that outlives the request that created it.
|
||||
*
|
||||
* The fixed-size pool below is deliberately realistic: four threads, reused forever,
|
||||
* exactly like a web server's request-handling pool. The bug is in {@link #handleOneRequest},
|
||||
* which does what a lot of real "request context" code does: creates a *new* ThreadLocal
|
||||
* instance per request to stash a payload, and never calls remove().
|
||||
*
|
||||
* A ThreadLocal's key is a WeakReference, so the ThreadLocal object itself can be collected.
|
||||
* But Thread.ThreadLocalMap.Entry extends WeakReference<ThreadLocal<?>> and keeps a
|
||||
* STRONG reference to the *value*. Expunging a stale entry (key==null) only happens on a later
|
||||
* get()/set()/remove() call on that same slot -- which never comes, because every request makes
|
||||
* a brand new ThreadLocal, so no one ever touches the old slot again. Each of the 4 pooled
|
||||
* threads accumulates one dead-keyed, live-valued entry per request forever.
|
||||
*/
|
||||
@SpringBootApplication
|
||||
public class ThreadLocalLeakApplication {
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(ThreadLocalLeakApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
ApplicationRunner driveLeak() {
|
||||
return (ApplicationArguments args) -> {
|
||||
ThreadPoolTaskExecutor pool = new ThreadPoolTaskExecutor();
|
||||
pool.setCorePoolSize(4);
|
||||
pool.setMaxPoolSize(4);
|
||||
pool.setQueueCapacity(Integer.MAX_VALUE);
|
||||
pool.setThreadNamePrefix("req-worker-");
|
||||
pool.initialize();
|
||||
|
||||
long payloadSize = 256 * 1024; // 256 KB "request context" per call
|
||||
long i = 0;
|
||||
System.out.println("threadlocal-leak: driving a 4-thread pool to OOM via leaked ThreadLocal entries");
|
||||
while (true) {
|
||||
final long reqId = i++;
|
||||
pool.execute(() -> handleOneRequest(reqId, payloadSize));
|
||||
if (reqId % 400 == 0) {
|
||||
System.out.println("threadlocal-leak: submitted=" + reqId);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Every call creates a fresh ThreadLocal (the bug -- it should be a single static field)
|
||||
* and never removes it. The whole point: this looks like ordinary per-request code.
|
||||
*/
|
||||
private static void handleOneRequest(long requestId, long payloadSize) {
|
||||
ThreadLocal<byte[]> requestContext = new ThreadLocal<>();
|
||||
requestContext.set(new byte[(int) payloadSize]);
|
||||
requestContext.get()[0] = (byte) requestId; // "use" the context
|
||||
// Missing on purpose: requestContext.remove();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
$ java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/unbounded-queue-leak.jar
|
||||
|
||||
unbounded-queue-leak: driving an unbounded queue to OOM, consumer sleeps 50ms/item
|
||||
java.lang.OutOfMemoryError: Java heap space
|
||||
Dumping heap to docs/output/heap.hprof ...
|
||||
Heap dump file created [175828534 bytes in 0.250 secs]
|
||||
|
||||
Caused by: java.lang.OutOfMemoryError: Java heap space
|
||||
at java.base/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:329)
|
||||
at com.ankurm.oomdemo.UnboundedQueueLeakApplication.lambda$driveLeak$0(UnboundedQueueLeakApplication.java:53)
|
||||
at com.ankurm.oomdemo.UnboundedQueueLeakApplication$$Lambda/0x000000000b247688.run(Unknown Source)
|
||||
at org.springframework.boot.SpringApplication.lambda$callRunner$0(SpringApplication.java:788)
|
||||
at org.springframework.boot.SpringApplication$$Lambda/0x000000000b25ccd0.acceptWithException(Unknown Source)
|
||||
at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82)
|
||||
at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
|
||||
@@ -0,0 +1,23 @@
|
||||
Eclipse MAT 1.17.0 batch report (org.eclipse.mat.api:suspects) on heap.hprof -- Problem Suspect 1
|
||||
================================================================================================
|
||||
|
||||
Problem Suspect 1 One instance of java.util.concurrent.LinkedBlockingQueue loaded by <system
|
||||
class loader> occupies 79,698,088 (92.41%) bytes. The top consumers of its minimum retained heap
|
||||
are byte[] (152 instances totaling 79,694,208), java.util.concurrent.LinkedBlockingQueue$Node
|
||||
(153 instances totaling 3,672), and java.util.concurrent.locks.ReentrantLock$NonfairSync (2
|
||||
instances totaling 64). The instance is referenced by
|
||||
com.ankurm.oomdemo.UnboundedQueueLeakApplication$$Lambda+0x000000000b25d428 @ 0xf63bb570 ,
|
||||
loaded by org.springframework.boot.loader.launch.LaunchedClassLoader @ 0xf631bab8 . The memory
|
||||
is accumulated in one instance of java.util.concurrent.LinkedBlockingQueue$Node , loaded by
|
||||
<system class loader> , which occupies 1,572,984 (1.82%) bytes. Thread java.lang.Thread @
|
||||
0xfcc19e38 main has a local variable or reference to java.util.concurrent.LinkedBlockingQueue @
|
||||
0xff7386e8 which is on the shortest path to java.util.concurrent.LinkedBlockingQueue$Node @
|
||||
0xfccce5c0 . The thread java.lang.Thread @ 0xfcc19e38 main keeps local variables with total size
|
||||
632,832 (0.73%) bytes. The top consumers of its minimum retained heap are byte[] (152 instances
|
||||
totaling 79,694,208), java.util.concurrent.LinkedBlockingQueue$Node (153 instances totaling
|
||||
3,672), and java.util.concurrent.locks.ReentrantLock$NonfairSync (2 instances totaling 64).
|
||||
Significant stack frames and local variables com.ankurm.oomdemo.UnboundedQueueLeakApplication.la
|
||||
mbda$driveLeak$0(Lorg/springframework/boot/ApplicationArguments;)V
|
||||
(UnboundedQueueLeakApplication.java:53) java.util.concurrent.LinkedBlockingQueue @ 0xff7386e8
|
||||
retains 79,698,088 (92.41%) bytes The stacktrace of this Thread is available. See stacktrace .
|
||||
See stacktrace with involved local variables .
|
||||
@@ -0,0 +1,40 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>unbounded-queue-leak</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>unbounded-queue-leak</name>
|
||||
<description>Leak pattern 5: a producer that outruns a consumer into an unbounded queue</description>
|
||||
|
||||
<properties>
|
||||
<java.version>25</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<finalName>unbounded-queue-leak</finalName>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+23
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs docs/output/heap.hprof (produced by scripts/run.sh) through Eclipse Memory Analyzer's
|
||||
# headless batch report generator -- MemoryAnalyzer's -application org.eclipse.mat.api.parse,
|
||||
# invoked via the ParseHeapDump.sh wrapper MAT ships -- then re-extracts the Problem Suspect
|
||||
# summary into docs/output/02-mat-leak-suspects.txt.
|
||||
#
|
||||
# No GUI is shown, but MAT's SWT runtime still needs an X display even in this mode, hence
|
||||
# xvfb-run. Needs: MAT_HOME pointing at an extracted Memory Analyzer 1.17.0+ "rcp" distribution,
|
||||
# and xvfb-run on PATH (package "xvfb" on Debian/Ubuntu).
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
: "${MAT_HOME:?Set MAT_HOME to your Memory Analyzer install (e.g. /opt/mat)}"
|
||||
if [ ! -f docs/output/heap.hprof ]; then
|
||||
echo "docs/output/heap.hprof not found -- run scripts/run.sh first" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f docs/output/heap_Leak_Suspects.zip
|
||||
xvfb-run -a "$MAT_HOME/ParseHeapDump.sh" "$MODULE_DIR/docs/output/heap.hprof" org.eclipse.mat.api:suspects
|
||||
python3 "$ROOT_DIR/scripts/extract-mat-summary.py" "$MODULE_DIR"
|
||||
Executable
+22
@@ -0,0 +1,22 @@
|
||||
#!/usr/bin/env bash
|
||||
# Drives unbounded-queue-leak to a REAL java.lang.OutOfMemoryError under a constrained heap, with
|
||||
# -XX:+HeapDumpOnOutOfMemoryError so the crash leaves a real .hprof behind. Also regenerates
|
||||
# docs/output/01-oom-console.txt from the real run log.
|
||||
#
|
||||
# Needs JDK 25 (LTS) on PATH.
|
||||
set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
ROOT_DIR="$(cd "$MODULE_DIR/.." && pwd)"
|
||||
cd "$MODULE_DIR"
|
||||
|
||||
mvn -q -B package -DskipTests
|
||||
mkdir -p docs/output
|
||||
rm -f docs/output/heap.hprof
|
||||
CMD="java -Xmx160m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=docs/output/heap.hprof -jar target/unbounded-queue-leak.jar"
|
||||
echo "== running: $CMD =="
|
||||
LOG="$(mktemp)"
|
||||
$CMD > "$LOG" 2>&1 || true
|
||||
python3 "$ROOT_DIR/scripts/extract-oom-console.py" "$MODULE_DIR" "$LOG" "$CMD"
|
||||
rm -f "$LOG"
|
||||
echo "heap dump: docs/output/heap.hprof (not committed -- see .gitignore; regenerate it with this script)"
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
package com.ankurm.oomdemo;
|
||||
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.ApplicationArguments;
|
||||
import org.springframework.boot.ApplicationRunner;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
import java.util.concurrent.LinkedBlockingQueue;
|
||||
|
||||
/**
|
||||
* Leak pattern 5 of 5: a producer that outruns a consumer into a queue with no capacity bound.
|
||||
*
|
||||
* {@code new LinkedBlockingQueue<>()} with no capacity argument is unbounded -- it will accept
|
||||
* elements until the heap is exhausted, full stop. That is the entire bug. It is also the most
|
||||
* common of the five in real systems, because it usually arrives disguised as "a queue between
|
||||
* a fast producer thread and a slow consumer thread," which sounds like ordinary
|
||||
* producer/consumer code until the consumer falls behind (a slow downstream call, a paused
|
||||
* consumer during a deploy, a poison message that makes every poll() take longer) and the queue
|
||||
* becomes the de facto unbounded heap allocator for the whole payload stream.
|
||||
*/
|
||||
@SpringBootApplication
|
||||
public class UnboundedQueueLeakApplication {
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(UnboundedQueueLeakApplication.class, args);
|
||||
}
|
||||
|
||||
@Bean
|
||||
ApplicationRunner driveLeak() {
|
||||
return (ApplicationArguments args) -> {
|
||||
// The bug: no capacity bound. new LinkedBlockingQueue<>(10_000) would turn this
|
||||
// whole failure mode into a put() that blocks instead of an OutOfMemoryError.
|
||||
LinkedBlockingQueue<byte[]> queue = new LinkedBlockingQueue<>();
|
||||
|
||||
Thread consumer = new Thread(() -> {
|
||||
while (true) {
|
||||
try {
|
||||
queue.take();
|
||||
Thread.sleep(50); // the slow downstream call
|
||||
} catch (InterruptedException e) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
}, "slow-consumer");
|
||||
consumer.setDaemon(true);
|
||||
consumer.start();
|
||||
|
||||
long payloadSize = 512 * 1024; // 512 KB per message
|
||||
long i = 0;
|
||||
System.out.println("unbounded-queue-leak: driving an unbounded queue to OOM, consumer sleeps 50ms/item");
|
||||
while (true) {
|
||||
queue.put(new byte[(int) payloadSize]);
|
||||
i++;
|
||||
if (i % 200 == 0) {
|
||||
System.out.println("unbounded-queue-leak: produced=" + i + " queueDepth=" + queue.size());
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user