High-speed Large Language Model Serving for Local Deployment

[ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration

Tool for test diferents large language models without code.

Local LLM Inference Library

LLM chatbot example using OpenVINO with RAG (Retrieval Augmented Generation).

Study Buddy is a desktop application that provides AI tutoring without requiring internet access or accounts

Lightweight 6GB VRAM Gradio web app with auto-installer for running AuraFlow locally — no cloud, no clutter.

Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI models that runs completely locally on your computer. No subscriptions, no data sent to the internet — just you and your personal AI assistant

Verify claims using AI agents that debate using scraped evidence and local language models.

Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

script which performs RAG and use a local LLM for Q&A

Script which takes a .wav audio file, performs speech-to-text using OpenAI/Whisper, and then, using Llama3, summarization and action point from the transcript generated