Kimi 3.1 running locally on a MacBook Pro at 2 tok/s

#17
by konstantinnikol - opened

Kimi 3.1 running locally on a MacBook Pro M5 Max 128 Gb

2 tok/s decode
8.4 tok/s prefill

Its a source build with weights downloaded separately.

Code: https://github.com/argonautlabsai/argodrive-ds4-kimi-k3.1
Full results and setup: https://argonautlabs.ai/research/kimi-k31-single-gpu.html#apple-silicon

nice speed

Latest results:

  • Decode, 512 tokens of raw text 2.49 tok/s
  • Prefill, 512-token prompt 9.0 tok/s

Sign up or log in to comment