asmhatre/java-ai-agents
1
0
Fork 0
Code Activity

Add jlama module: run a 4-bit TinyLlama inside the JVM with Jlama on the Vector API, with tokens per second, streaming and an Ollama baseline

Browse Source 
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01G8ikz8xdWuTP5yun8DZ1hk
This commit is contained in:
Claude
2026-10-11 09:24:56 +00:00
parent 10545299f8
commit fdff2b7388
17 changed files with 564 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
jlama/output/01-load-and-ask.txt
+9
View File
@@ -0,0 +1,9 @@
Machine: 2 CPUs, JDK 25.0.4.1+1-LTS, Vector API preferred float width 512 bits
Model: tjake/TinyLlama-1.1B-Chat-v1.0-Jlama-Q4, 1132 MB on disk
Load time: 665 ms; process resident memory 119 MB before, 175 MB after
Question: In one sentence, what is a Java record?
Answer: A Java record is a data structure that allows for the creation of immutable, named, and indexed collections of objects.
Prompt tokens: 26, generated tokens: 24
Prompt time: 4838 ms, generation time: 2762 ms