Who should I follow to learn about small and on-device models?

If you want to get really good at small/on-device models, I’d follow a mix of researchers, systems engineers, and builders. My shortlist:

My top 10

1. Awni Hannun - probably my #1 for on-device AI. Co-created Apple’s MLX and focuses heavily on efficient ML, Apple Silicon, speech, and local inference. Ep. 1 - Awni Hannun on ASR, ML... - localhost - Apple Podcasts

2. Georgi Gerganov - creator of llama.cpp. Essential if you care about what actually makes models run fast on ordinary CPUs/GPUs, quantization, GGUF, and local inference. Llama.cpp - Run LLM Inference in C/C++

3. Tianqi Chen - follow for the systems side: model compilation, hardware-aware optimization, MLC-LLM, TVM, and deployment across different hardware. Tianqi Chen

4. Nicholas D. Lane - excellent researcher to follow for mobile/edge AI and the intersection of models, hardware, and resource constraints. He's also a co-author of a particularly useful recent survey of SLMs for edge deployment. Demystifying Small Language Models for Edge Deployment - ACL Anthology

5. Mengwei Xu - especially relevant if you want small models actually doing useful things on phones. His work includes Android agents and benchmarking small language models on-device. Mengwei Xu - ACL Anthology

6. Prince Canuma - very practical open-source perspective around MLX, vision-language models, and audio models. A great person to watch if you want to build rather than just read papers. Tiny-LLM – a course of serving LLM on Apple Silicon for systems engineers | Hacker News

7. Simon Willison - not primarily an SLM researcher, but one of the best people for understanding how local/small models are becoming useful in real applications. He also actively experiments with MLX and local models. Tiny-LLM – a course of serving LLM on Apple Silicon for systems engineers | Hacker News

8. The Google AI Edge / LiteRT team - more useful as a team/account than an individual. They're doing serious work on Gemma-sized models, Android/iOS/Web deployment, quantization, RAG and function calling. Google AI Edge | Google for Developers

9. Apple ML Research - another account/team I'd follow rather than a single person. Their recent work is unusually relevant: Apple's ~3B on-device model uses KV-cache sharing and 2-bit quantization-aware training. Updates to Apple’s On-Device and Server Foundation Language Models - Apple Machine Learning Research

10. Zhenyan Lu - worth following specifically for SLM research. The SLM survey from his group covers roughly 100M–5B models and evaluates their actual latency and memory footprint; the follow-up work looks at 60+ publicly available SLMs for edge deployment. Small Language Models: Survey, Measurements, and Insights

If you only follow 5

I'd make it:

Awni Hannun → Georgi Gerganov → Tianqi Chen → Nicholas Lane → Simon Willison

That gives you five complementary perspectives:

• Awni: efficient ML + Apple Silicon
• Georgi: low-level inference + quantization
• Tianqi: compilers + hardware
• Lane: academic edge-AI research
• Simon: practical local-model applications

And there's a particularly important distinction to keep in mind: “small model” and “model that runs well on a small device” aren't necessarily the same thing. Current research is increasingly focused on memory bandwidth, KV cache, quantization, sparsity, and hardware/model co-design - not simply reducing parameter count. On-device large language models: a survey of model compression and system optimization | Artificial Intelligence Review | Springer Nature Link

If your goal is building on-device AI products, rather than doing research, I'd change the list substantially and give you a “20 people/accounts to follow on X + GitHub, ranked by signal-to-noise” list.

To learn about Small Language Models (SLMs) and on-device AI, the best approach is to follow the researchers, engineers, and developer advocates who build the architectures, quantization techniques, and execution runtimes (like Ollama, MLC-LLM, and ExecuTorch).

The key people, organizations, and channels to follow are organized below by category:

1. Edge AI & Optimization Researchers / Engineers

• Tim Dettmers (Professor / Researcher): Best known for pioneering quantization methods like QLoRA and 8-bit/4-bit matrix multiplication, which make running and fine-tuning SLMs on consumer hardware possible. Follow him for deep dives into memory efficiency and model sizing.

• Soumith Chintala & The PyTorch ExecuTorch Team: As co-creator of PyTorch, Soumith and the engineering teams at Meta/PyTorch heavily drive ExecuTorch , which is the primary framework for deploying on-device AI natively on mobile chips (iOS/Android).

• Guido Appenzeller & The Ollama Team: The creators and core maintainers behind Ollama have democratized running SLMs (like Llama 3.2, Phi, and Mistral) locally on laptops and desktops. Following them provides great insight into local developer workflows.

2. Key Labs and Research Teams (X / Twitter & Substack)

Rather than just individuals, tracking the core labs pushing out frontier small models yields constant insights into distillation, data curation, and architecture design:

• Hugging Face Research Team (Leandro von Werra, Clément Delangue, and the HF open-source team): Hugging Face constantly drops open SLMs (like the SmolLM series) and writes incredible technical breakdowns on edge optimization, pruning, and quantization. Small Language Models (SLM): A Comprehensive Overview

• Microsoft Research (The Phi Team): Microsoft's team behind the Phi series (Phi-3, Phi-3.5, Phi-4) pioneered "textbook-quality" synthetic data training to make sub-4B parameter models punch way above their weight class. Small Language Models (SLM): A Comprehensive Overview

• Google DeepMind (Gemma Team): The creators of Gemma and EmbeddingGemma regularly share research on squeezing multimodal capabilities into ultra-low footprint on-device models.

3. Practical Implementers & Edge Creators

• Machine Learning Street Talk (Andrey Kurenkov & co.): An exceptional podcast and community featuring deep technical interviews with researchers working on model compression, distillation, and efficient architectures.

• Hamel Husain & Shreya Shankar: Excellent data scientists and educators to follow for practical MLOps, focusing on how developers actually evaluate, fine-tune, and deploy smaller, cheaper models into production applications.

Where to look next:

• Hugging Face Daily Papers: Keep an eye on the cs.CL (Computation and Language) or cs.LG categories on Hugging Face Papers, filtering for keywords like quantization , pruning , knowledge distillation , and edge AI.

• GitHub Repositories: Star and watch repositories like Ollama , MLC-LLM (MLC AI) , and llama.cpp (led by Georgi Gerganov), where the bleeding-edge implementation of running models on iPhones, MacBooks, and edge hardware happens daily.

Are you looking to learn more about the theory behind training small models (like data pruning and distillation), or are you trying to figure out how to deploy them into a mobile or web app?

To stay on the cutting edge of small language models (SLMs) and on-device AI, you want to follow researchers, developers, and communities focusing heavily on model compression, quantization, edge-deployment, and local execution.

Here are the key people, organizations, and hubs to follow (primarily active across X/Twitter, Bluesky, and GitHub): The AI Researcher's Guide to a Non-Boring Bluesky Feed | Naomi SaphraWho to follow on Bluesky in ML/AI? : r/LocalLLaMA - Reddit

1. Independent & Practical AI Researchers / Builders

** Simon Willison :** Independent AI researcher and creator of Datasette. He constantly tests, writes about, and documents running small models locally and building practical tools around them. You can follow his work on his Simon Willison's Blog or via his Simon Willison Bluesky Profile.

Nathan Lambert (Interconnects) : Research scientist at Allen AI who writes extensively on fine-tuning, RLHF, and the practical mechanics behind open-weights and smaller models. Check out his Interconnects Substack or his Nathan Lambert Bluesky Profile. AI/ML People - StarterPack

2. Core Labs & Engineering Teams Driving On-Device Tech

• • Hugging Face: The epicenter of the open-source community. Following Hugging Face and key researchers there will keep you updated on the latest open SLMs (like Gemma, Phi, and Llama derivatives) and optimization pipelines. Check out the Hugging Face Bluesky Account. [Bluesky]

• Apple Machine Learning / Foundation Models Research: Apple's ML research group publishes crucial work on efficient on-device inference, memory management, and pruning techniques tailored for consumer hardware.

• Qualcomm AI Research: Qualcomm routinely publishes breakthroughs in running heavy quantization (like sub-4-bit models) directly on mobile and automotive chips.

3. Communities & Ecosystems

• • LocalLLaMA Community: While not a single person, the r/LocalLLaMA community on Reddit and associated chat spaces are where grassroots developers test out 1B–9B parameter models, share GGUF/EXL2 quantization benchmarks, and discuss running models locally on phones, Macs (MLX framework), and PCs. Small Language Models for On

• Ollama & MLX Teams: Following the core developers behind tools like Ollama or Apple's MLX framework gives you a direct window into how on-device runtimes are evolving day-to-day. [Digital Applied]

For a deep dive into why small language models are taking over edge computing and agentic workflows, watch this breakdown:

1m

Small Language Models are the Future of Agentic AI Reading ...

1 year ago

YouTube · AAIF Live

Would you like recommendations tailored more toward mobile deployment (iOS/Android) , hardware/quantization math , or fine-tuning your own small models?

Follow Microsoft’s Edge AI for Beginners materials for a practical introduction to deploying small language models on edge devices. For the research side, explore work on on-device inference and small/large model collaboration, such as Edgellm.(techcommunity.microsoft.com)(arxiv.org)

Follow these experts and organizations to stay ahead in small and on-device AI models: Vikas Chandra (Meta), Raghuraman Krishnamoorthi (Meta), NVIDIA’s applied AI research teams, and academic groups like MIT CSAIL’s Decentralized Information Group. They are actively shaping the field with practical engineering insights, compression techniques, and privacy-preserving methods.On-Device LLMs in 2026: What Changed, What Matters, What's Next - Edge AI and Vision AllianceOn-Device LLMs: State of the Union, 2026 – Vikas Chandra – Senior Director & Distinguished Scientist, AI @ MetaSmall Language Models for On-Device Agents in 2026Enabling privacy-preserving AI training on everyday devices | MIT News | Massachusetts Institute of TechnologyPocket Experts: MobileMoE and the Memory Math of On-Device AI | Cognaptus

🔑 Key People to Follow

• Vikas Chandra (Meta) Senior Director & Distinguished Scientist at Meta AI. He focuses on scaling down models for mobile and embedded devices, emphasizing quantization, compression, and deployment strategies. His talks at Embedded Vision Summit highlight why small models are becoming central to real-world applications.On-Device LLMs in 2026: What Changed, What Matters, What's Next - Edge AI and Vision AllianceOn-Device LLMs: State of the Union, 2026 – Vikas Chandra – Senior Director & Distinguished Scientist, AI @ Meta

• Raghuraman Krishnamoorthi (Meta) Works closely with Chandra on on-device LLMs. He provides practical insights into memory bandwidth bottlenecks and efficient deployment on consumer hardware.On-Device LLMs: State of the Union, 2026 – Vikas Chandra – Senior Director & Distinguished Scientist, AI @ Meta

• NVIDIA Research Teams Published the influential paper “Small Language Models are the Future of Agentic AI” (2025). They argue that 1–3B parameter models are the sweet spot for tool use and agentic loops, making them leaders in small-model advocacy.Small Language Models for On-Device Agents in 2026

• MIT CSAIL – Decentralized Information Group (DIG) Researchers like Irene Tenison, Lalana Kagal, and Anna Murphy are advancing privacy-preserving federated learning for small devices. Their work ensures AI can run securely on resource-constrained hardware like wearables and sensors.Enabling privacy-preserving AI training on everyday devices | MIT News | Massachusetts Institute of Technology

• Zelina Phones (Cognaptus) Author of MobileMoE , which explores how Mixture-of-Experts(MoE) architectures can be adapted for smartphones. She emphasizes that efficiency isn’t just about shrinking models but aligning architecture, quantization, and runtime with device constraints.Pocket Experts: MobileMoE and the Memory Math of On-Device AI | Cognaptus

📌 Organizations and Communities

• Meta AI – pushing boundaries in small-scale LLM deployment.

• NVIDIA Applied AI – strong research presence in agentic AI and efficient runtimes.

• MIT CSAIL – academic hub for federated learning and privacy-preserving AI.

• Edge AI and Vision Alliance – industry consortium where experts like Chandra share practical deployment strategies.On-Device LLMs in 2026: What Changed, What Matters, What's Next - Edge AI and Vision Alliance

⚠️ Challenges & Trade-offs

• Memory bandwidth bottlenecks: Phones have ~50–90 GB/s vs. GPUs with 2–3 TB/s, making compression critical.

• Battery & thermal limits: Small models must finish inference quickly to avoid draining devices.

• Capability vs. size: Sub-billion models can outperform larger ones if trained with high-quality synthetic data and distillation.On-Device LLMs in 2026: What Changed, What Matters, What's Next - Edge AI and Vision Alliance

👉 If you want to dive deeper, I’d suggest following these experts on LinkedIn, Twitter/X, and arXiv for their latest publications. Would you like me to curate a list of their most recent talks, papers, and social handles so you can follow them directly?