Who should I follow to learn about Small and on-device models?

If you want to understand **small language models (SLMs), on-device AI, edge inference, quantization, and efficient ML systems**, I’d follow a mix of **researchers, engineers, and builders** rather than only big-model personalities. The field sits at the intersection of model architecture, systems, and hardware. Recent surveys emphasize that the important topics are not just shrinking models, but also compression, hardware-aware optimization, and deployment constraints. [A Survey on On-Device Small Language Models: Architectures, Compression, Applications, and Open Challenges | IEEE Conference Publication | IEEE Xplore](https://ieeexplore.ieee.org/document/11434122/?utm_source=chatgpt.com) ## Researchers to follow - Nicholas D. Lane - One of the strongest names in **mobile AI, efficient ML, and on-device intelligence**. His work covers making models practical under memory, latency, and energy constraints. [Demystifying Small Language Models for Edge Deployment - ACL Anthology](https://aclanthology.org/2025.acl-long.718/?utm_source=chatgpt.com) - Song Han - A key researcher behind **model compression, pruning, quantization, and efficient neural networks**. Follow for the foundations of making models smaller and faster. - Yunxin Liu - Important work around **mobile systems, AI efficiency, and edge deployment**. - Tri Dao - More systems-focused than SLM-specific, but his work on efficient attention and inference is highly relevant to running models efficiently. - Percy Liang - Follow for broader model efficiency, evaluation, and open-model ecosystem trends. ## People close to the actual small-model ecosystem - Sebastian Raschka - Excellent for practical explanations of LLM internals, fine-tuning, and efficient models. - Hamel Husain - Strong practical perspective on building with smaller, deployable models. - Jeremy Howard - Useful for practical ML engineering and making models usable outside giant compute environments. ## Teams and labs worth tracking - Microsoft - The Phi models are among the most visible examples of capable small models. Track the Phi papers, releases, and deployment work. - Google DeepMind - The Gemma family is important for lightweight and on-device use cases. Google has also published work around running SLMs with edge tooling. [On-device small language models with multimodality, RAG, and Function Calling - Google Developers Blog](https://developers.googleblog.com/google-ai-edge-small-language-models-multimodality-rag-function-calling/?utm_source=chatgpt.com) - Meta Platforms - Follow the Llama ecosystem, especially smaller variants and deployment tooling. - Apple - Important if you care about **private, local AI on phones and laptops**, although much of Apple's research appears through papers and developer releases rather than open model releases. - Hugging Face - Probably the best hub for tracking open small models, quantized models, and deployment tooling. ## Engineering-focused accounts/resources - MLC AI - Follow if you care about compiling and deploying models directly to phones, browsers, and edge devices. - llama.cpp - Essential for understanding practical local inference, GGUF models, quantization, and CPU/GPU deployment. - TensorFlow Lite - Useful for mobile and embedded deployment. - ONNX Runtime - Important for production inference optimization. ## Newsletters / places to watch - [Hugging Face](https://huggingface.co?utm_source=chatgpt.com) - Model releases, papers, demos. - [Papers with Code](https://paperswithcode.com?utm_source=chatgpt.com) - Track benchmarks and implementations. - [arXiv machine learning section](https://arxiv.org/list/cs.LG/recent?utm_source=chatgpt.com) - New papers. - [Awesome Mobile LLMs GitHub repository](https://github.com/stevelaskaridis/awesome-mobile-llm?utm_source=chatgpt.com) - A curated map of mobile/on-device LLM research and tools. [GitHub - stevelaskaridis/awesome-mobile-llm: Awesome Mobile LLMs · GitHub](https://github.com/stevelaskaridis/awesome-mobile-llm?utm_source=chatgpt.com) ## A good learning path If your goal is **building on-device AI**, I’d prioritize: 1. **Quantization + inference** - 4-bit/8-bit quantization - KV-cache optimization - CPU/NPU inference 2. **Small model families** - Phi - Gemma - Llama small variants - Qwen small variants 3. **Deployment stacks** - llama.cpp - MLC - ONNX Runtime - TensorFlow Lite / LiteRT 4. **Hardware** - Apple Neural Engine - Qualcomm NPUs - Android AI Edge - Raspberry Pi / Jetson-class devices For a curated “follow list,” I’d start with **Nicholas Lane + Song Han + Sebastian Raschka + the Phi/Gemma/Llama teams + llama.cpp/MLC AI**. That combination gives you both the research frontier and the practical builder perspective.

**Follow these experts to stay ahead on small and on-device AI models: Rachel Nabors, Ramin Hasani, and Laurence Moroney. They consistently share insights on efficiency, deployment, and practical use cases of lightweight AI.** ### 🔑 Key Figures in Small & On-Device AI Expert| Focus Area| Why Follow Them ---|---|--- **Rachel Nabors**| Advocates for smaller, local AI models| Explains efficiency, cost savings, and performance benefits of running AI locally instead of relying on large frontier models.[Rachel Nabors: Local AI Models for Frontier Results | StartupHub.ai](https://www.startuphub.ai/ai-news/artificial-intelligence/2026/rachel-nabors-local-ai-models-for-frontier-results) **Ramin Hasani (Liquid AI)**| Device-native foundation models| Researches liquid neural networks and hardware-aware architectures, emphasizing bio-inspired approaches for efficient edge AI.[Intelligence on the Edge: Liquid AI's Ramin Hasani on the Search for Device-Native Foundation Models](https://www.cognitiverevolution.ai/intelligence-on-the-edge-liquid-ai-s-ramin-hasani-on-the-search-for-device-native-foundation-models/) **Laurence Moroney (Google/AI Guy)**| Practical deployment of open-weight models| Focuses on models in the 2–12B parameter range that can run on phones, laptops, and classroom machines, highlighting licensing and reproducibility.[Small AI in 2026: Open Weights Go On-Device - Laurence Moroney - The AI Guy.](https://laurencemoroney.com/2026/08/21/small-ai-open-weights-on-device.html) **NVIDIA Research Teams**| Small language models for agentic AI| Demonstrates how 3–9B parameter models can handle most agentic tasks locally, with cloud escalation only when necessary.[Small Language Models for On-Device Agents in 2026](https://www.digitalapplied.com/blog/small-language-models-on-device-agents-2026-guide) ### 📌 Why These Voices Matter * **Efficiency & Cost:** Nabors emphasizes that small models reduce reliance on expensive API calls to frontier models. * **Hardware Awareness:** Hasani’s work shows how architectures optimized for devices (like gated convolutions) outperform brute-force scaling. * **Accessibility & Licensing:** Moroney highlights open-weight releases (e.g., Gemma 4 under Apache 2.0) that make small models practical for education and reproducibility. * **Agentic AI:** NVIDIA’s research suggests small models should be the _default_ for on-device agents, balancing speed, privacy, and cost. ### ⚠️ Risks & Trade-offs * **Performance Ceiling:** Small models may struggle with highly complex tasks, requiring fallback to larger cloud models. * **Hardware Constraints:** Running even 3–9B parameter models locally demands careful optimization of memory and bandwidth. * **Licensing Issues:** Not all open-weight models are truly “open”; Moroney warns to check model cards for restrictions. * **Fragmented Ecosystem:** Unlike frontier models, small-model research is spread across startups, academia, and open-source communities, so staying updated requires following multiple voices. ### ✅ Next Steps If you want to dive deeper: * Follow **Rachel Nabors** for practical deployment stories. * Track **Liquid AI and Ramin Hasani** for cutting-edge research on liquid neural networks. * Subscribe to **Laurence Moroney’s “AI Guy” updates** for applied tutorials and open-weight releases. * Watch **NVIDIA’s agentic AI publications** for benchmarks and deployment strategies. Would you like me to also highlight **open-source communities and GitHub repos** where these small/on-device models are actively developed and shared? That way, you can follow both the thought leaders _and_ the projects themselves.

**Key Researchers & Practitioners** * **Andrej Karpathy:** Former head of AI at Tesla and OpenAI co-founder; frequently shares deep, accessible insights and code tutorials on running small models and neural networks locally. * **Tim Dettmers:** Researcher known for pioneering quantization techniques (like QLoRA) that make training and running small models on consumer hardware viable. * **Hugging Face Team (e.g., Julien Chaumond, Thomas Wolf):** The core team regularly highlights edge-AI architectures, Optimum integrations, and state-of-the-art small language models (SLMs). **Organizations & Labs** * **Apple Machine Learning Research:** Publishes papers and frameworks (like MLX) specifically tailored for high-efficiency, on-device silicon processing. * **Google AICore & Gemini Nano Teams:** Documents the practical blueprints for shipping system-level multi-modal small models on mobile hardware. * **Microsoft Research (Phi Team):** Renowned for pushing the boundaries of synthetic data generation to create ultra-capable sub-10B parameter models (the Phi series). **Core Frameworks & Tools to Follow** * **Ollama & Llama.cpp (Georgi Gerganov):** Essential ecosystems for running LLMs locally on consumer laptops and edge devices. * **MLX (Apple):** The go-to framework for machine learning research and deployment natively on Apple Silicon. What specific hardware or use case (e.g., mobile apps, IoT, desktop software) are you building for?

To truly master small, efficient, and on-device machine learning (often referred to as **Edge AI** , **TinyML** , or **EfficientML**), you want to follow researchers and practitioners who focus on hardware-software co-design, quantization, distillation, and running models locally on constrained hardware. [Song Han - Associate Professor, MIT EECS](https://hanlab.mit.edu/songhan)[AI Dev 25 x NYC | Eric Sondhi: Small AI: The Next Big Thing](https://www.youtube.com/watch?v=DRMX9YzmYMQ&t=657)[Team - MIT HAN Lab](https://hanlab.mit.edu/team) The top experts, labs, and creators to follow include: * * ** _Song Han_ (MIT / MIT HAN Lab)**: Pioneer in efficient AI computing. He is an Associate Professor at MIT whose lab focuses on deep compression, pruning, quantization (like AWQ and SmoothQuant), and hardware-software co-design for LLMs and TinyML. Check out his [MIT HAN Lab Website](https://hanlab.mit.edu/) and his open EfficientML.ai Lecture Series. [HAN Lab MIT +1] * ** _Andrej Karpathy_** : Former Director of AI at Tesla and co-founder of OpenAI, Karpathy frequently shares deep, incredibly clear insights on how smaller, distilled, or carefully curated models can punch way above their weight class by focusing on the "cognitive core" rather than raw brute-force scaling. Follow him on _Andrej Karpathy_ on X. [15 AI Twitter Accounts to Follow for LLM Research and Insights](https://www.linkedin.com/posts/promptgenix_15-ai-related-accounts-you-should-follow-activity-7450520895883747328-8r0q) * ** _Sebastian Raschka_ (Lightning AI)**: An exceptional educator for anyone wanting to build models from the ground up. He writes extensively about understanding small, code-level mechanics of language models and efficient fine-tuning. Visit his [Sebastian Raschka Personal Site](https://sebastianraschka.com/) for his hands-on books and tutorials. [Build a Large Language Model (From Scratch)](https://sebastianraschka.com/llms-from-scratch/)[Build LLMs From Scratch with Sebastian Raschka #52](https://www.youtube.com/watch?v=79F32D9aM8U) * **MLCommons** : The organization behind MLPerf. They routinely push out mobile and edge benchmarks tracking how native LLMs (like 1B and 3B variations) run on consumer hardware, phones, and embedded devices. Follow updates via MLCommons on X. [MLCommons - X](https://x.com/MLCommons) * **Apple Machine Learning Research / MLX Community** : Apple's research team and the open-source community around the `MLX` framework are doing cutting-edge work bringing efficient, native on-device LLMs and stable diffusion models directly onto Apple Silicon. Following community engineers like [Sabesh Bharathi on X](https://x.com/sabeshbharathi) offers a great window into daily on-device ML engineering. [sabesh (@sabeshbharathi) / X](https://x.com/sabeshbharathi) * Would you prefer to focus your learning on **theoretical optimization techniques** (like quantization and pruning algorithms) or **hands-on deployment** (running models locally on iOS/Android/Raspberry Pi)?