Highlights

⛰️ Stability. This release improves stability and performance of the Petals DHT in presence of many servers joined via NAT traversal & relays. Now, the DHT prefers to store the keys on directly reachable peers, so that all peers can access them faster and with less failures. Also, this release contains a minor fix to the block reassignment algorithm that decreases excess reassignments that were leading to the swarm downtime in the past.

🌎 Basic routing. We have improved the routing algorithm for inference, so that clients weakly prefer servers holding more blocks to minimize latency and increase inference speed. This is only a basic algorithm, and we are working on smarter routing (taking into account latency, throughput, etc.) for both inference and fine-tuning in future releases. This release also makes the servers share more technical information about themselves (their version, free cache, etc.), so it can be used by the smarter routing algorithms in future and shown at http://health.petals.ml for debugging purposes.

What's Changed

• Fix fine-tuning notebooks intros by @borzunov in https://github.com/bigscience-workshop/petals/pull/194
• Ignore network RPS if we failed to measure it by @borzunov in https://github.com/bigscience-workshop/petals/pull/198
• Make client ignore blacklist if all servers holding a block are blacklisted by @borzunov in https://github.com/bigscience-workshop/petals/pull/197
• Increase tolerances in test_tp_block by @justheuristic in https://github.com/bigscience-workshop/petals/pull/196
• Fix --no_auto_relay help by @borzunov in https://github.com/bigscience-workshop/petals/pull/199
• Use length-weighted sampling in routing for inference by @justheuristic in https://github.com/bigscience-workshop/petals/pull/204
• Return available cache size in rpc_info() by @justheuristic in https://github.com/bigscience-workshop/petals/pull/191
• Add service checking direct reachability from peers by @justheuristic in https://github.com/bigscience-workshop/petals/pull/195
• Report server version and dht.client_mode in rpc_info(), check for updates on startup by @borzunov in https://github.com/bigscience-workshop/petals/pull/209
• Don't switch blocks if it makes swarm disjoint by @borzunov in https://github.com/bigscience-workshop/petals/pull/210
• Fix output shape when resuming generation by @borzunov in https://github.com/bigscience-workshop/petals/pull/211
• Improve errors in case of missing blocks, suggest to join your own server by @borzunov in https://github.com/bigscience-workshop/petals/pull/212
• CI: Convert model only when convert_model.py or setup.cfg change by @borzunov in https://github.com/bigscience-workshop/petals/pull/213
• CI: Update deprecated actions, don't measure network RPS by @borzunov in https://github.com/bigscience-workshop/petals/pull/215
• Bump version to 1.1.1 by @borzunov in https://github.com/bigscience-workshop/petals/pull/214

Full Changelog: https://github.com/bigscience-workshop/petals/compare/v1.1.0...v1.1.1