Files
java-ai-agents/jlama/output/02-tokens-per-second.txt

12 lines
705 B
Plaintext

Same question, 5 measured runs after 1 warm-up run, temperature 0, limit 64 tokens.
run 1: prompt 26 tokens in 4902 ms (5.3 tok/s), generated 24 tokens in 3040 ms (7.9 tok/s)
run 2: prompt 26 tokens in 4762 ms (5.5 tok/s), generated 24 tokens in 3061 ms (7.8 tok/s)
run 3: prompt 26 tokens in 4527 ms (5.7 tok/s), generated 24 tokens in 2856 ms (8.4 tok/s)
run 4: prompt 26 tokens in 4892 ms (5.3 tok/s), generated 24 tokens in 3007 ms (8.0 tok/s)
run 5: prompt 26 tokens in 5203 ms (5.0 tok/s), generated 24 tokens in 2894 ms (8.3 tok/s)
Generation tokens/s: min 7.8, median 8.0, max 8.4
Prompt tokens/s: min 5.0, median 5.3, max 5.7
Prompt + generated tokens in the longest run: 50 (limit was 64)