zizq

关于

I'm Chris, and this is Zizq, a centralised job queue with a heap of awesome features I won't list off here. https://github.com/zizq-labs/zizq It gets through the queue fast under heavy throughput and requires no setup out of the box. The server is built in Rust as a single static binary, and it provides its own durable storage via an LSM tree storage engine (fjall), and aims to make searching and mutability of jobs first-class considerations. For example you can update jobs that are sitting in the middle of a queue without draining them and re-queueing them. The server interacts with clients through a HTTP/1 and HTTP/2 API, using a persistent streaming connection with heartbeats and automated CI-tested disconnect or network partition/split detection, which acts as the liveness probe on both client and server. Because it's HTTP, clients are easy to write without re-inventing the queue machinery. Do it with curl and shell scripts on the command line if you really want to (I've actually done this but that is not released as I would strongly advise against doing such a thing in production). As of now there are four official clients, for Node, Rust, Elixir and Ruby. More to follow in due course and/or on request. Jobs are always written to the LSM tree to ensure durability, but LSM trees are well optimised for write-heavy workloads like job queues (they always write sequentially) and internally Zizq employs some techniques like write coalescing and some lightweight use of in-memory skip list structures to get the most bang for buck out of the LSM tree storage. Feedback and questions appreciated.

收录记录

1. Show HN

   2026年10月7日 · 1 次收录