产品
jitllm
One binary. No dependencies. Drop it in and run. Up to 4.62x faster prefill than llama.cpp and 5.53x faster batched decoding than vLLM. jitllm detects your hardware and optimizes kernels and model placement across CPUs a…
关于
One binary. No dependencies. Drop it in and run. Up to 4.62x faster prefill than llama.cpp and 5.53x faster batched decoding than vLLM. jitllm detects your hardware and optimizes kernels and model placement across CPUs and GPUs. Run open-weight flagships, even larger than memory. OpenAI- and Anthropic-compatible APIs, desktop and terminal apps. Linux, macOS and Windows.
收录记录
- Product Hunt 新品
讨论这个产品