asmhatre/java-ai-agents
1
0
Fork 0
Code Activity

Add onnx-djl module: DistilBERT sentiment with ONNX Runtime and DJL, batching and a Spring Boot endpoint with measured latency

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
Claude
2026-10-11 09:33:36 +00:00
parent fdff2b7388
commit 280cbe7d56
20 changed files with 805 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
onnx-djl/output/04-batching.txt
+9
View File
@@ -0,0 +1,9 @@
Padding does not change the answers: 32 texts one by one versus one batch of 32.
same labels: true, largest score difference 0.00e+00
Time to classify 32 texts, median of 5 runs after a warm-up, by batch size:
batch size 1: 230 ms in total, 7.2 ms per text
batch size 4: 204 ms in total, 6.4 ms per text
batch size 8: 166 ms in total, 5.2 ms per text
batch size 16: 142 ms in total, 4.4 ms per text
batch size 32: 134 ms in total, 4.2 ms per text