All projectsPROJECT/06
GarudaAI
Local LLM Setup and Remote Access CLI
Linux CLI tool that profiles GPU specs, determines which local LLMs can run efficiently on the hardware, installs Ollama with the optimal model, and exposes a remote-access endpoint reachable from any phone on the network.
View SourceJanuary 2026
GarudaAI · SchematicFIG/06
The Problem
Running a local LLM requires knowing your GPU specs, picking quantization levels, and configuring remote access. GarudaAI collapses the entire setup into one command.
How It Works
Python CLI→
GPU Hardware Profiler (VRAM + compute detection)→
Ollama Auto-Installer→
Model Pull→
Network Tunnel (mobile remote access)
Key Features
- Profiles GPU VRAM and compute against quantization tables (Q4_K_M, Q5, Q8, fp16)
- Recommends the highest-quality model your hardware can actually run
- Automatically installs Ollama and pulls the matched model without manual steps
- Exposes a mobile-accessible inference endpoint via network tunnel
- Supports LLaMA 3, Mistral, Phi-3, and all Ollama-compatible model families
Technical Highlights
- GPU profiling pipeline that matches VRAM and compute against model quantization requirements to recommend the best-fit local LLM.
- One-command setup: installs Ollama, pulls the right model, and tunnels a mobile-accessible endpoint for on-the-go inference.