#83 · Local and open models · 5.7 minContext Length vs Memory: Why Local LLMs Slow Down
Reading a 2,542 token prompt took my laptop 155 seconds.
Python 3.12.15 | ollama==0.6.32 videos tagged "local llm speed".
#83 · Local and open models · 5.7 minReading a 2,542 token prompt took my laptop 155 seconds.
Python 3.12.15 | ollama==0.6.3
#87 · Local and open models · 5.5 minOne model, one laptop: 1.6, 6.6 and 9.2 tokens per second. All three were real.
Python 3.12.15 | ollama==0.6.3