Add shimmy - Python-free Rust inference server

[Michael-A-Kuykendall]

Adds shimmy to the ML serving platforms list.

Project Details:

• Repository: https://github.com/Michael-A-Kuykendall/shimmy
• Stars: 2,918 ⭐
• Language: Rust
• License: MIT

Why Shimmy belongs in awesome-ml-serving:

Production ML Serving Focus:

• OpenAI-compatible API for seamless integration with existing ML workflows
• High-performance Rust implementation optimized for production inference
• Single binary deployment eliminates Python dependency hell
• Hot model swapping for zero-downtime model updates
• Auto-discovery of available models

Key Technical Features:

• Modern Model Formats: GGUF and SafeTensors support
• API Compatibility: Drop-in replacement for OpenAI endpoints
• Resource Efficiency: Rust performance for production workloads
• Deployment Simplicity: Single binary, no complex dependencies
• Production Ready: 2,918 stars with active development and community

Serving Use Cases:

• LLM inference serving for production applications
• OpenAI API proxy for cost savings and privacy
• Edge AI deployment with minimal resource footprint
• Multi-model serving with hot-swapping capabilities
• Enterprise on-premises AI inference

Positioning: Shimmy fills the gap for developers who need OpenAI-compatible inference serving without Python dependencies, providing a Rust-native solution that's both performant and easy to deploy in production environments.

This addition aligns with the repository's goal of curating "platforms for serving models in production" by providing a lightweight, efficient alternative for LLM inference serving.