Add excel-export module: Apache POI 6.0.0 XSSF vs SXSSF under a fixed heap, 1,000,000 rows streamed with StreamingResponseBody, OpenCSV, sheet rolling, temp-file cleanup
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_017ic4b4VQVEJWetTT1dYzyJ
This commit is contained in:
Claude Sonnet 5.5
@@ -0,0 +1,25 @@
|
||||
sheetLimit=1000 (header included), 2500 data rows
|
||||
sheet 1 "Sales": last row index 999, first data cell 1
|
||||
sheet 2 "Sales 2": last row index 999, first data cell 1000
|
||||
sheet 3 "Sales 3": last row index 502, first data cell 1999
|
||||
XSSF and SXSSF files: same sheets, same formatted cell values on every 97th row
|
||||
|
||||
styling of a 5,000-row SXSSF file, read back with XSSF
|
||||
header cell A1 text=Order id bold=true
|
||||
freeze pane: row split at 1
|
||||
auto filter: A1:I1
|
||||
cell styles in the workbook: 5 (for 5,000 rows x 9 cells)
|
||||
B2 as Excel shows it: 30-May-2025
|
||||
amount number format: [>=10000000]##\,##\,##\,##0.00;[>=100000]##\,##\,##0.00;##,##0.00
|
||||
|
||||
the sheet limit
|
||||
createRow(1048575) ok
|
||||
createRow(1048576) -> IllegalArgumentException: Invalid row number (1048576) outside allowable range (0..1048575)
|
||||
|
||||
lakh/crore number format, rendered by POI's DataFormatter (not by Excel)
|
||||
999.50 -> 999.50
|
||||
99,999.50 -> 99,999.50
|
||||
123,456.78 -> 1,23,456.78
|
||||
1,234,567.89 -> 12,34,567.89
|
||||
12,345,678.90 -> 1,23,45,678.90
|
||||
123,456,789.10 -> 12,34,56,789.10
|
||||
@@ -0,0 +1,7 @@
|
||||
after write(), no dispose(): 1 temp file(s) left: [poi-sxssf-sheet2787588911680201252.xml]
|
||||
after close() only: 0 temp file(s) left
|
||||
after dispose(): 0 temp file(s) left
|
||||
|
||||
temp file size after writing 50,000 rows (9 columns), before dispose():
|
||||
compressTempFiles=false: 24,339,585 bytes
|
||||
compressTempFiles=true: 2,974,461 bytes
|
||||
@@ -0,0 +1,18 @@
|
||||
first bytes: EF BB BF (UTF-8 BOM so Excel reads UTF-8)
|
||||
rows written: 3
|
||||
Order id,Order date,Region,Product,Qty,Unit price,Amount,Status,Note
|
||||
1,2025-05-30,Chennai,Product-384,20,464.83,9296.60,RETURNED,INV/2025-26/0000001 dispatched via hub 71
|
||||
2,2025-07-29,Pune,Product-121,20,596.20,11924.00,RETURNED,INV/2025-26/0000002 dispatched via hub 46
|
||||
3,2025-11-22,Kolkata,Product-6,11,61.05,671.55,SHIPPED,INV/2025-26/0000003 dispatched via hub 51
|
||||
|
||||
comma, quote and newline inside fields: "Pune, MH","said ""hi""","line1\nline2"\n
|
||||
|
||||
=HYPERLINK("http://x","pay") -> '=HYPERLINK("http://x","pay")
|
||||
+91 98200 00000 -> '+91 98200 00000
|
||||
-2+3 -> '-2+3
|
||||
@SUM(A1) -> '@SUM(A1)
|
||||
Mumbai -> Mumbai
|
||||
|
||||
raw CSVWriter into a broken stream: writeNext called 10,000 times, none threw
|
||||
checkError() = true, getException() = java.io.IOException: Broken pipe
|
||||
CsvExporter into the same stream: IOException: CSV write failed (cause: java.io.IOException: Broken pipe)
|
||||
@@ -0,0 +1,3 @@
|
||||
GET /export/xssf?rows=1500 -> 200, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, attachment; filename="sales.xlsx", content-length header: none (chunked: chunked), 91,345 bytes
|
||||
GET /export/sxssf?rows=1500 -> 200, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, attachment; filename="sales.xlsx", content-length header: none (chunked: chunked), 93,481 bytes
|
||||
GET /export/csv?rows=1500 -> 200, text/csv;charset=UTF-8, attachment; filename="sales.csv", content-length header: none (chunked: chunked), 152,833 bytes
|
||||
@@ -0,0 +1,7 @@
|
||||
# XSSF under a fixed 512 MB heap: rows until it dies
|
||||
# openjdk version "25.0.5" 2026-10-20 LTS | 2 vCPU | 8027 MB RAM
|
||||
# indicative numbers from a shared sandbox, one run per line; see README
|
||||
XSSF /export/xssf?rows=10000 Xmx=512m http=200 curl=0 bytes=576961 10.1s | peak used heap 135 MB | max after GC 112 MB | 18 GC pauses, longest 495 ms | peak RSS 397 MB | temp dir peak 0 MB, files left 0 | OOM=no
|
||||
XSSF /export/xssf?rows=25000 Xmx=512m http=200 curl=0 bytes=1430086 13.8s | peak used heap 307 MB | max after GC 255 MB | 18 GC pauses, longest 327 ms | peak RSS 671 MB | temp dir peak 0 MB, files left 0 | OOM=no
|
||||
XSSF /export/xssf?rows=50000 Xmx=512m http=200 curl=0 bytes=2851566 14.3s | peak used heap 494 MB | max after GC 474 MB | 26 GC pauses, longest 315 ms | peak RSS 674 MB | temp dir peak 0 MB, files left 0 | OOM=no
|
||||
XSSF /export/xssf?rows=100000 Xmx=512m http=500 curl=0 bytes=107 68.3s | peak used heap - MB | max after GC 512 MB | 106 GC pauses, longest 2412 ms | peak RSS 667 MB | temp dir peak 0 MB, files left 0 | OOM=YES | body: {"timestamp":"2026-10-29T06:03:25.226Z","status":500,"error":"Internal Server Error","path":"/export/xssf"}
|
||||
@@ -0,0 +1,8 @@
|
||||
# SXSSF under the same 512 MB heap, same rows, plus 1,000,000
|
||||
# openjdk version "25.0.5" 2026-10-20 LTS | 2 vCPU | 8027 MB RAM
|
||||
# indicative numbers from a shared sandbox, one run per line; see README
|
||||
SXSSF /export/sxssf?rows=10000 Xmx=512m http=200 curl=0 bytes=598239 10.6s | peak used heap 165 MB | max after GC 18 MB | 10 GC pauses, longest 57 ms | peak RSS 333 MB | temp dir peak 4 MB, files left 0 | OOM=no
|
||||
SXSSF /export/sxssf?rows=25000 Xmx=512m http=200 curl=0 bytes=1490887 3.0s | peak used heap 211 MB | max after GC 18 MB | 11 GC pauses, longest 42 ms | peak RSS 366 MB | temp dir peak 11 MB, files left 0 | OOM=no
|
||||
SXSSF /export/sxssf?rows=50000 Xmx=512m http=200 curl=0 bytes=2978433 6.5s | peak used heap 307 MB | max after GC 17 MB | 11 GC pauses, longest 82 ms | peak RSS 477 MB | temp dir peak 23 MB, files left 0 | OOM=no
|
||||
SXSSF /export/sxssf?rows=100000 Xmx=512m http=200 curl=0 bytes=5954167 9.9s | peak used heap 306 MB | max after GC 22 MB | 12 GC pauses, longest 78 ms | peak RSS 484 MB | temp dir peak 46 MB, files left 0 | OOM=no
|
||||
SXSSF /export/sxssf?rows=1000000 Xmx=512m http=200 curl=0 bytes=59408766 36.0s | peak used heap 321 MB | max after GC 19 MB | 29 GC pauses, longest 79 ms | peak RSS 504 MB | temp dir peak 475 MB, files left 0 | OOM=no
|
||||
@@ -0,0 +1,6 @@
|
||||
# 1,000,000 rows: SXSSF and CSV under a 64 MB heap, XSSF given a large heap
|
||||
# openjdk version "25.0.5" 2026-10-20 LTS | 2 vCPU | 8027 MB RAM
|
||||
# indicative numbers from a shared sandbox, one run per line; see README
|
||||
SXSSF /export/sxssf?rows=1000000 Xmx=64m http=200 curl=0 bytes=59408766 37.5s | peak used heap 56 MB | max after GC 20 MB | 163 GC pauses, longest 43 ms | peak RSS 208 MB | temp dir peak 475 MB, files left 0 | OOM=no
|
||||
CSV /export/csv?rows=1000000 Xmx=64m http=200 curl=0 bytes=104476584 4.3s | peak used heap 52 MB | max after GC 15 MB | 82 GC pauses, longest 40 ms | peak RSS 175 MB | temp dir peak 0 MB, files left 0 | OOM=no
|
||||
XSSF /export/xssf?rows=1000000 Xmx=3g http=500 curl=0 bytes=107 273.6s | peak used heap - MB | max after GC 3072 MB | 132 GC pauses, longest 12794 ms | peak RSS 3295 MB | temp dir peak 0 MB, files left 0 | OOM=YES | body: {"timestamp":"2026-10-29T06:11:02.454Z","status":500,"error":"Internal Server Error","path":"/export/xssf"}
|
||||
@@ -0,0 +1,9 @@
|
||||
# Sheet rolling: 1,100,000 rows, SXSSF, 64 MB heap, default sheet limit 1,048,576
|
||||
# openjdk version "25.0.5" 2026-10-20 LTS | 2 vCPU | 8027 MB RAM
|
||||
# indicative numbers from a shared sandbox, one run per line; see README
|
||||
SXSSF /export/sxssf?rows=1100000 Xmx=64m http=200 curl=0 bytes=65376880 39.0s | peak used heap 56 MB | max after GC 25 MB | 178 GC pauses, longest 49 ms | peak RSS 209 MB | temp dir peak 523 MB, files left 0 | OOM=no
|
||||
file size: 63 MB
|
||||
<sheet name="Sales"
|
||||
<sheet name="Sales 2"
|
||||
sheet1.xml: 1048576 rows (header included)
|
||||
sheet2.xml: 51426 rows (header included)
|
||||
@@ -0,0 +1,5 @@
|
||||
# compressTempFiles on a 1,000,000-row SXSSF export, 64 MB heap
|
||||
# openjdk version "25.0.5" 2026-10-20 LTS | 2 vCPU | 8027 MB RAM
|
||||
# indicative numbers from a shared sandbox, one run per line; see README
|
||||
SXSSF /export/sxssf?rows=1000000&compress=false Xmx=64m http=200 curl=0 bytes=59408766 37.1s | peak used heap 55 MB | max after GC 19 MB | 169 GC pauses, longest 54 ms | peak RSS 209 MB | temp dir peak 475 MB, files left 0 | OOM=no
|
||||
SXSSF /export/sxssf?rows=1000000&compress=true Xmx=64m http=200 curl=0 bytes=59408766 57.4s | peak used heap 56 MB | max after GC 22 MB | 168 GC pauses, longest 37 ms | peak RSS 208 MB | temp dir peak 56 MB, files left 0 | OOM=no
|
||||
@@ -0,0 +1,6 @@
|
||||
xssf.xlsx ZIP entry timestamps: [(2026, 10, 29, 11, 46, 52)]
|
||||
sxssf.xlsx ZIP entry timestamps: [(1980, 0, 0, 0, 0, 0)]
|
||||
LibreOffice 24.2.7.2 420(Build:2)
|
||||
xssf.xlsx: LibreOffice loads it; row 2 -> 1,05/30/2025,Chennai,Product-384,20,464.83,9296.6,RETURNED,INV/2025-26/0000001 dispatched via hub 71
|
||||
sxssf.xlsx: LibreOffice: source file could not be loaded
|
||||
rezipped.xlsx: LibreOffice loads it; row 2 -> 1,05/30/2025,Chennai,Product-384,20,464.83,9296.6,RETURNED,INV/2025-26/0000001 dispatched via hub 71
|
||||
Reference in New Issue
Block a user