bufferedreader: read a file line by line module (BufferedReader vs Scanner vs Files.lines vs readAllLines on a 1 GB file; heap at -Xmx64m, timings, read-call counts, one-huge-line OOM, charset/BOM/line-terminator pitfalls, Scanner swallowing IOException), 10 contract tests

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017ic4b3pDoGLiWk6LLBeHjW
This commit is contained in:
Claude
2026-11-15 01:20:37 +05:30
parent 5faa57bb2d
commit f8c9e244a3
25 changed files with 809 additions and 0 deletions
@@ -0,0 +1,22 @@
$ LC_ALL=C java -Dfile.encoding=COMPAT CharsetDemo
Charset.defaultCharset() = US-ASCII
native.encoding = ANSI_X3.4-1968
file.encoding (property) = ANSI_X3.4-1968
latin1.txt bytes: 63 61 66 e9 0a (the text is "café" saved as ISO-8859-1)
new FileReader(path) -> "caf\uFFFD"
new FileReader(path, UTF_8) -> "caf\uFFFD"
Files.newBufferedReader(path) -> MalformedInputException: Input length = 1
Files.readAllLines(path) -> MalformedInputException: Input length = 1
Files.lines(path).toList() -> UncheckedIOException: java.nio.charset.MalformedInputException: Input length = 1 [cause: MalformedInputException: Input length = 1]
new Scanner(path).nextLine() -> "caf\uFFFD"
readAllLines(path, ISO_8859_1) <- fix -> [café]
utf8.txt bytes: 63 61 66 c3 a9 20 e2 82 b9 31 30 30 0a ("café ₹100" saved as UTF-8)
new FileReader(path) -> "caf\uFFFD\uFFFD \uFFFD\uFFFD\uFFFD100"
new FileReader(path, UTF_8) <- fix -> "caf\u00E9 \u20B9100"
bom.csv bytes: ef bb bf 69 64 2c 6e 61 6d 65 0a 31 2c 41 ...
header line = "\uFEFFid,name" length=8 (the text is 7 characters)
header.equals("id,name") = false
after stripping U+FEFF = "id,name" equals=true