bufferedreader: read a file line by line module (BufferedReader vs Scanner vs Files.lines vs readAllLines on a 1 GB file; heap at -Xmx64m, timings, read-call counts, one-huge-line OOM, charset/BOM/line-terminator pitfalls, Scanner swallowing IOException), 10 contract tests
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b3pDoGLiWk6LLBeHjW
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
$ java CharsetDemo
|
||||
Charset.defaultCharset() = UTF-8
|
||||
native.encoding = ANSI_X3.4-1968
|
||||
file.encoding (property) = UTF-8
|
||||
|
||||
latin1.txt bytes: 63 61 66 e9 0a (the text is "café" saved as ISO-8859-1)
|
||||
new FileReader(path) -> "caf\uFFFD"
|
||||
new FileReader(path, UTF_8) -> "caf\uFFFD"
|
||||
Files.newBufferedReader(path) -> MalformedInputException: Input length = 1
|
||||
Files.readAllLines(path) -> MalformedInputException: Input length = 1
|
||||
Files.lines(path).toList() -> UncheckedIOException: java.nio.charset.MalformedInputException: Input length = 1 [cause: MalformedInputException: Input length = 1]
|
||||
new Scanner(path).nextLine() -> "caf\uFFFD"
|
||||
readAllLines(path, ISO_8859_1) <- fix -> [café]
|
||||
|
||||
utf8.txt bytes: 63 61 66 c3 a9 20 e2 82 b9 31 30 30 0a ("café ₹100" saved as UTF-8)
|
||||
new FileReader(path) -> "caf\u00E9 \u20B9100"
|
||||
new FileReader(path, UTF_8) <- fix -> "caf\u00E9 \u20B9100"
|
||||
|
||||
bom.csv bytes: ef bb bf 69 64 2c 6e 61 6d 65 0a 31 2c 41 ...
|
||||
header line = "\uFEFFid,name" length=8 (the text is 7 characters)
|
||||
header.equals("id,name") = false
|
||||
after stripping U+FEFF = "id,name" equals=true
|
||||
Reference in New Issue
Block a user