#59 · LLM engineering and evals · 4.6 minStream LLM Replies in Python: First Word in 0.2s
The same prompt showed its first word after 0.2 seconds instead of eight.
Python 3.12.15 | ollama==0.6.33 videos tagged "time to first token".
#59 · LLM engineering and evals · 4.6 minThe same prompt showed its first word after 0.2 seconds instead of eight.
Python 3.12.15 | ollama==0.6.3
#83 · Local and open models · 5.7 minReading a 2,542 token prompt took my laptop 155 seconds.
Python 3.12.15 | ollama==0.6.3
#87 · Local and open models · 5.5 minOne model, one laptop: 1.6, 6.6 and 9.2 tokens per second. All three were real.
Python 3.12.15 | ollama==0.6.3