Machine Learning Systems

Community Spotlight: TinyML Innovation Emerging from Colombian Universities

Diego Méndez — Tue, 17 Mar 2026 00:00:00 GMT

Our TinyML Show&Tell sessions continue to highlight an encouraging trend: students around the world are not only learning about embedded AI, they are actively building real systems with it. A recent session featured projects from Colombia, where a growing community of students and researchers is exploring how TinyML and EdgeAI can support applications in healthcare, rehabilitation, and privacy-preserving sensing.

Many of these projects are connected to the work of Diego Méndez, a full professor at Pontificia Universidad Javeriana in Bogotá, whose students are developing embedded AI systems through research and experimentation. Their work reflects the impact of engaging with the TinyML/EdgeAI ecosystem and collaborating within its global community.

This momentum can be traced in part to the SciTinyML 2025 Workshop held in Bogotá last year, organized by the TinyMLedu initiative and partially supported by the EdgeAI Foundation, Edge Impulse and Seeed Studio. Workshops like this often act as catalysts, bringing together students, educators, and practitioners who continue collaborating and building long after the event ends.

Today, we highlight a few of the projects emerging from this growing TinyML/EdgeAI community in Colombia.

Postural Monitoring with TinyML and Federated Learning (Ángela Torres - Universidad del Rosario)

Ángela presented her work on the validation of an IoT system with federated learning and TinyML for postural monitoring in a simulated healthcare environment. The system combines embedded sensing with on-device machine learning to detect posture patterns while allowing models to improve collaboratively through federated learning.

By keeping sensitive health data on local devices and sharing only model updates, the approach enables privacy-preserving monitoring, an important requirement for many healthcare applications.

Embedded Movement Intention Detection for Post-Stroke Rehabilitation (Andrés Gómez - Pontificia Universidad Javeriana)

Andrés presented the results of his master’s thesis: an embedded movement intention detection system based on surface electromyography (sEMG) designed to support the smart rehabilitation of post-stroke patients.

The system processes muscle activity signals directly on an embedded device to infer a patient’s intended movement in real time. This capability could enable more responsive rehabilitation devices and assistive technologies that adapt to a patient’s intent. Andrés’ research has also resulted in peer-reviewed journal publications.

Albaricoque: A Privacy-Preserving Human Detection Node (Sergio Mesa - Pontificia Universidad Javeriana)

Sergio presented the preliminary results of his master’s thesis: Albaricoque, a TinyML-based human detection node designed with privacy in mind. By using ultrasonic and passive infrared (PIR) sensors and performing inference locally on the device, the system can detect human presence avoiding the usage of cameras that transmit sensitive visual data.

The project has already gained international recognition: Albaricoque was selected as one of the winning entries in the global Edge Impulse Hackathon in December 2025.

Together, these projects show how local initiatives can help grow a global ecosystem of TinyML and EdgeAI builders. The Bogotá workshop helped spark a community that is now producing research, prototypes, and student-led innovations.

This is exactly what we hope to see: students building real systems, sharing their work, and helping expand the global TinyML community. We welcome students to register to present at an upcoming Show&Tell session!

Written by Professor Diego Méndez
Pontificia Universidad Javeriana

The Model Is Not the Product

Vijay Janapa Reddi — Tue, 10 Mar 2026 00:00:00 GMT

Everyone talks about models. But in production AI systems, the model is only a small piece of the story.

The model is not the product.

Last month I argued that AI Engineering is the missing discipline. The response confirmed what I suspected: the gap is real, and people feel it. But defining a field is only step one. The harder question is: what does an AI engineer actually need to see what others miss?

The 5% Problem

In 2015, a team of Google engineers audited their production ML systems and published a finding that should have changed how everyone thinks about AI: the model, the neural network, the thing everyone obsesses over, accounted for roughly 5% of the total code (see Figure 1). The other 95% was data pipelines, feature extraction, serving infrastructure, configuration management, and monitoring.

Figure 1: The relative hidden technical debt in machine learning systems by Sculley et al. 2015.

That ratio has not changed. If anything, it has gotten worse. The models have gotten larger, but the systems around them have grown faster.

This is the gap that most AI education ignores. We teach people to build the 5% and hope they figure out the other 95% on the job.

Data, Algorithm, Machine

So how do you reason about the other 95%? In the Machine Learning Systems textbook, I use a simple diagnostic framework I call the D-A-M taxonomy: every ML system is shaped by three forces. Data is the fuel, what you train on and how you move it. Algorithm is the blueprint, the math that turns data into predictions. Machine is the engine, the silicon, memory, and power budget you actually have to work with (see Figure 2).

The key insight is that these three forces are interdependent. Compressing a model to fit on a phone changes its accuracy. Doubling the training data demands more compute. Switching from a CPU to a GPU reshapes which algorithms are even practical. Change any one axis, and the others must adapt.

Figure 2: The D-A-M Taxonomy: Data, Algorithm, and Machine are interdependent axes. The intersections (what to learn from, how to move information, how to execute efficiently) are where ML systems engineering lives.

This sounds abstract until you see it in the real world.

In 2012, Krizhevsky split AlexNet across two GPUs because neither had enough memory to hold the whole network. A Machine constraint shaped the Algorithm that launched the deep learning revolution.

That was two GPUs and 6 GB of memory. What happens when the constraint is an entire country's hardware supply?

DeepSeek: When Constraints Become Advantages

In late 2024, DeepSeek released V3, a model that matched GPT-4 on most benchmarks. What stunned the field was not the performance. It was the cost: $5.576 million (see Figure 3) for the final training run, against an estimated $100 million or more for GPT-4.

How did they overcome this issue? Not by finding a shortcut. But by being forced into better engineering.

US export controls barred China from buying NVIDIA's top-tier H100 GPUs. DeepSeek trained on H800s, the same silicon but with NVLink bandwidth cut by 56%. That single Machine constraint cascaded across every axis of the D-A-M taxonomy.

Figure 3: How one hardware constraint cascaded into three architectural innovations that produced a frontier model at a fraction of the cost.

Machine → Algorithm: the hardware forced a new architecture. With half the interconnect bandwidth, the standard way of splitting a model across GPUs was prohibitively expensive. So DeepSeek engineered around it. Instead, they built a 671-billion-parameter Mixture-of-Experts model where only 37 billion parameters activate per token. Think of it like a hospital with 100 specialists but only 5 in the room for any given patient. Less data moves through the system on every forward pass, by design.

Machine → Data: the hardware forced a new number format. They trained in FP8, 8-bit precision instead of the standard 16, cutting memory usage in half and doubling effective bandwidth. The tradeoff is that lower precision can destabilize training, which required new data engineering to manage.

Algorithm → Data: the algorithm forced a new memory strategy. Their Multi-head Latent Attention compresses the model's working memory by over 98%, fundamentally changing how much data moves during inference. Fewer bytes per token means faster serving and lower cost.

The result: a frontier model trained for a fraction of what competitors spent. Not because DeepSeek had better researchers (though they very well might). Because the constraint forced a better architecture.

Export controls meant to slow China's AI progress may have produced the most hardware-efficient training system anyone has built.

DeepSeek's team published their infrastructure work at ISCA 2025, a computer architecture conference, not an AI one, explicitly framing it as hardware-software co-design. That venue choice tells you everything about where the real innovation happened.

It Is Not Just the Model

DeepSeek shows how Machine constraints reshape Algorithms. But even when the architecture is right, the infrastructure can be the bottleneck.

When Meta trained Llama 3, they used 16,384 H100 GPUs for 54 days. During that run, they recorded 419 hardware failures, roughly one every three hours. GPUs crashed, network links dropped, storage nodes failed. The dominant engineering challenge was not the model architecture; it was keeping the cluster alive long enough to finish a checkpoint.

This is the other 95% in action. The model architecture worked fine. The engineering challenge was everything around it: detecting failures, routing around dead nodes, restarting from checkpoints, and coordinating 16,384 GPUs that would rather not cooperate.

This is exactly the kind of reasoning I want future AI engineers to develop. Not just "how do I train a model" but "where is my system actually breaking, and which axis do I fix first?"

Here is the practical takeaway: the next time you hit a wall in your ML system, before you reach for a bigger model or more data, ask which axis is actually the bottleneck. Is it Data (you are starving for quality inputs)? Algorithm (you are using the wrong architecture for your hardware)? Machine (you are memory-bound and no algorithm change will help)? That question is where DeepSeek started, and the constraint is what made the answer interesting.

Your Turn

The lesson is simple. Frontier AI is no longer just about better models. It is about better systems thinking.

When an ML system breaks, the cause is almost always on one of three axes. Data. Algorithm. Machine. AI engineering begins where those three forces collide.

DeepSeek turned a hardware handicap into an architectural advantage. Meta turned 419 failures into a resilience playbook.

What constraint shaped your system? And did you fight it, or did you let it guide you?

Next month: the full stack of AI engineering, from silicon to serving. What each layer does, where the bottlenecks hide, and why no one teaches it end-to-end.

What the Community Is Building

A discipline does not emerge from a single book or tool. It emerges from people building things together.

Visit MLSysBook.ai for curriculum updates, hardware recommendations, and TinyTorch learning materials.

If you would like to support our community outreach and global workshops, consider contributing at opencollective.com/mlsysbook.

Correction: Show & Tell Time Update (Please Note)

Vijay Janapa Reddi — Thu, 26 Feb 2026 00:00:00 GMT

Hello everyone,

We made a mistake in the newsletter regarding the time for Thursday’s Student Show & Tell — thank you for your patience!

The correct time is:

📅 Thursday, February 26 at 9:00am EST / 2:00pm GMT

Please tune in via the Google Meet link here.

🇨🇴 Featuring presentations from students in Colombia
🛠️ Each presentation is 10–15 minutes and focused on real, applied systems work

We’re looking forward to seeing you there and learning together.

Warmly,
Kari

Community Update: Kits and This Month’s Student Show and Tell

Vijay Janapa Reddi — Tue, 24 Feb 2026 00:00:00 GMT

Hello Everyone,

Welcome to this week’s MLSysBook community update, where we share progress, tools, and opportunities to help build the discipline of AI engineering together. This week, we’re spotlighting the hardware kits that power hands-on learning and inviting you to see student systems in action at our upcoming TinyML Show & Tell.

Tune in to this month’s Show & Tell via the Google Meet link

📅 Thursday, February 26 at 9am EST, 2pm GMT

🇨🇴 Featuring presentations from students in Colombia

🛠️ Each presentation is 10-15 minutes and focused on real, applied systems work

For students: If you want to share your TinyML project, we would love to see you at a future Show and Tell. These sessions are designed to give you a pedestal to present your work, explain your systems decisions, and engage with a global audience.(Register here)

Many educators and students ask the same question: where do we begin if we want to build real TinyML and Edge AI systems?

We have curated a small set of hardware kits that we recommend for hands on coursework, research, and student projects. You can find them here:

TinyML Kits: https://mlsysbook.ai/kits/

These kits are designed to expose learners to the realities that define AI engineering in practice including memory limits, latency constraints, power considerations, and deployment tradeoffs. They are well suited for TinyML and Edge AI courses, capstone projects, and research prototypes.

For educators: If you are teaching and would like guidance on getting started, simply reply. For those in the global south, you may apply for kits here.

If you would like to sponsor a kit for a student or institution that cannot afford one, your support will fund under resourced educators and learners and help expand access to practical AI engineering education worldwide.

Working with a SEEED XIAO ML Kit

Thank you for being part of this growing global AI Engineering community.

Kari Janapareddi

Director of Partnerships and Strategic Operations
MLSysbook.ai Community

🔧 Building TinyTorch Together — Live Kickoff

Vijay Janapa Reddi — Wed, 04 Feb 2026 00:00:00 GMT

Dear Tiny Torch Community,

We’re excited to invite you to the first Tiny Torch kickoff session—a live Zoom tutorial that marks the beginning of something we’ve been building toward for a while.

Tiny Torch is meant to be learned by doing, refined through real use, and shaped by the people who care about making it genuinely teachable and useful. This first session brings together a small group of early community members who are excited to learn alongside us and help us make the material stronger as we go.

If you enjoy rolling up your sleeves, asking questions, trying things out, and sharing what you notice along the way, we’d love to have you join us.

Those who participate in this initial group will be recognized as Founding Contributors to Tiny Torch. Your feedback, questions, and observations will directly influence how this material evolves—for future learners and educators around the world.

What we’ll do together in this first session

Walk through the Tiny Torch setup and core tutorial, step by step
Highlight areas we’re actively refining and would love input on
Explain how this contributor group will collaborate with a TA and the core team over the coming weeks

You don’t need to have everything figured out to participate. Curiosity, enthusiasm, and a willingness to engage are what matter most.

Details

Date: Tuesday February 10th
Time: 9am EST
Location: Zoom Registration Link

If this sounds like something you’d enjoy being part of, please register using the link above. We’ll follow up with next steps and a bit more context before we meet.

I’m grateful for this community and for the energy so many of you bring to learning and building together. If this resonates with you, I’m really glad you’re here.

See you soon,
Vijay

Calling for Kit Applications - XIAOML for Applied AI Engineering

Vijay Janapa Reddi — Tue, 03 Feb 2026 00:00:00 GMT

Dear Educator,

We’re reaching out to invite university educators teaching (or planning to teach) TinyML and edge AI to apply to receive TinyML hardware kits for use in their courses.

These kits are designed to support underfunded or resource-constrained institutions that want to offer students meaningful, hands-on experience building AI systems that run in the real world—not just in theory.

If this sounds relevant for your classroom, we invite you to apply using this short form:
👉 Apply here by March 15th

The application helps us understand your teaching context, how the kits will be used, and the potential for long-term student impact. Selected educators will receive kits at no cost, along with access to open-source TinyML learning resources and a global educator community.

This initiative is made possible through a generous collaboration: Seeed Studio developed the kits and offered them at a reduced cost, the Edge AI Foundation funded them, and ICTP will support distribution to universities across the Global South. We’re deeply grateful for partners who share a commitment to expanding access to hands-on AI education.

XIAOML Kit & Components

Thank you for the work you do to support and inspire students. We’d love to learn more about what you’re building.

Kari Janapareddi

Director of Partnerships and Strategic Operations
MLSysbook.ai Community

The Shift Toward AI Engineering

Vijay Janapa Reddi — Tue, 27 Jan 2026 00:00:00 GMT

My dad told me if I didn’t study computer science in college, he’d throw me out of the house. So I did. In hindsight, he was right. CS gave me the foundation for everything I’ve done over the past 25 years, the way of thinking and problem-solving instincts that still matter every day.

My career didn’t stop at CS. I studied computer engineering and electrical engineering, and over time worked across the stack, from hardware and systems to software and machine learning. Seeing the same constraints resurface at every layer is what made the systems gap impossible to ignore.

That perspective is what makes the current moment in AI stand out to me. Signals from education and industry suggest the landscape has expanded more than most people realize.

The Shift You Can See in the Data

Computer science enrollment is down at 62% of computing programs this year. CS, software engineering, and information systems are all declining. Meanwhile, 66% of programs report that recent graduates are struggling to find jobs. The “learn to code” promise that defined a generation of career advice isn’t landing the way it used to. That makes sense. Coding was never the destination, it was the foundation.

But this isn’t a retreat from technology, and it’s not a sign that CS doesn’t matter. It’s a redirection. Students are asking a more nuanced question: what kind of tech career will last?

The fields that are growing? Computer engineering. Cybersecurity. AI. Engineering overall is up 7.5%. Students aren’t abandoning tech. They’re rethinking what kind of tech career will last.

The CRA report captures it well: students are gravitating toward majors that feel “more physical and less susceptible to the impact of AI.” They want to build real-world devices, not just write code that might get automated. They’re looking for skills that feel durable.

I think this signals something important. And at the same time, there’s a bigger picture that’s easy to miss.

The Gap That’s Quietly Widening

Here’s what I’ve observed after years of teaching computer and machine learning systems, and working with industry partners: everyone wants to use AI, but very few people understand how to build the systems that make it work at scale.

Who optimizes models to run on phones? Who designs the chips, builds the frameworks, creates the deployment pipelines? Who debugs why a model that worked perfectly in a notebook crashes in production? Who figures out how to serve millions of users without the infrastructure costs exploding?

These questions don’t fit neatly into computer science alone, though CS fundamentals remain essential. They’re not pure software engineering either. And they’re not just electrical engineering.

They require a different kind of thinking—one that spans the full stack from silicon to cloud, from training to deployment, from algorithms to real-world constraints.

What Is AI Engineering?

I’ve started calling this AI Engineering. Not as a replacement for existing fields. CS, EE, and software engineering remain foundational. But building production AI systems requires a synthesis of skills that don’t live neatly in any one discipline.

Think about what it actually takes to deploy an AI system that works:

The ML theory: Optimization, statistical learning, how models generalize and when they fail. This is the ML research foundation—understanding why things work, not just that they work.

The ML systems: Distributed training, hardware acceleration, memory hierarchies, serving infrastructure. This is what makes AI run efficiently at scale, the difference between a demo and a product.

The ML applications: Problem framing, data collection, domain constraints, what “good enough” actually means in a specific context. This is what connects AI to real-world impact.

Most education gives you one of these. Maybe two if you’re lucky. Almost no one gets all three.

AI Engineering sits at the intersection:

CleanShot 2026-01-22 at 21.11.00@2x.png

The ML Systems Researcher lives in the overlap of theory and systems. The Data Scientist bridges theory and application. The MLOps Engineer connects systems and application. The AI Engineer needs to move fluidly across all three. Not as an expert in everything, but with enough literacy across the stack to see how the pieces connect, and enough depth in at least one area to actually build things.

Why This Matters Right Now

Andrew Ng famously said that AI is the new electricity. He was right. But here’s what’s easy to miss: if AI is electricity, we’re training everyone to use the appliances while almost no one learns how to build the power plants.

Look at what’s actually happening. OpenAI has committed to spending $1.15 trillion on infrastructure over the next decade. Microsoft just opened Fairwater, the world’s most powerful AI datacenter. It covers 315 acres, required 26.5 million pounds of structural steel, and runs 120 miles of underground cable. Analysts predict that building AI infrastructure globally will exceed $1 trillion by 2030, including data centers, power generation, cooling systems, and semiconductor supply chains.

These aren’t software problems. They’re engineering problems. Physical problems. Power, cooling, hardware, optimization, efficiency.

And that’s only half the picture. AI isn’t just running in massive data centers. It’s also moving to the edge.

AI Is Going Everywhere

The TinyML market is growing at 34% annually, driven by the need to run AI on devices with severe constraints: phones, wearables, medical devices, agricultural sensors, industrial equipment. These systems can’t send data to the cloud and wait for a response. They need to process locally, in real time, with minimal power.

NVIDIA calls this “Physical AI”. Jensen Huang recently said that “the ChatGPT moment for robotics is here.” Companies like Boston Dynamics, Caterpillar, and LG are building robots that can reason, plan, and act in the real world. Robotics is now the fastest-growing category on Hugging Face.

This is where the “physical computing” instinct that students have makes sense. AI that interacts with the real world requires understanding hardware constraints, power budgets, latency requirements, and sensor systems. You can’t just train a model and throw it over the wall. You have to engineer the entire system.

The students shifting toward computer engineering and hardware? They’re sensing this. The companies scrambling to hire people who understand deployment and optimization? They’re responding to it.

But we do not yet have the educational infrastructure to train AI Engineers at scale. The knowledge exists. It is just scattered across research groups, companies, and a handful of courses. Most people never see the full picture.

That is the gap the MLSys Book exists to fill. Not by replacing existing disciplines, but by making the connective tissue visible. And that is what this community is here to build together.

And if you want to get hands on, TinyTorch lets you build your own ML framework from scratch. Because you cannot debug what you did not build.

One last thing that matters to me. The reason this book and these materials are freely available is simple. The ability to build and reason about real AI systems should not depend on access to a specific lab, company, or country.

A lot of the most important engineering knowledge today is still learned through proximity. This project is an attempt to make that knowledge explicit, so more people can participate in building AI systems that are reliable, accountable, and actually work in the real world.

What’s Coming

Over the coming months, I will share what we are learning as this work evolves. New chapters as they are ready. Updates in the field. Hands on labs that surface real system tradeoffs. Insights from practitioners building and deploying production AI. Resources for educators teaching this emerging field.

This is a work in progress, shaped by the people who engage with it. If there is a topic you want to go deep on, efficient inference, on device learning, ML operations, or something else entirely, tell me. I read the messages that come in, and they shape what we prioritize next.

If this resonated, forward it to someone who should see it. A student trying to make sense of their path. A colleague watching how the work is changing. Someone who wants to understand what it actually takes to build AI systems in the real world.

Vijay

Supporting the work:

The MLSys Book, TinyTorch, and the TinyML Kit materials are freely available. Their ongoing development is supported by Harvard, community donations, industry sponsorship, and partnerships.

Many people support the project simply by giving it GitHub ⭐ stars and providing feedback.

Sources:

MLSysBook.ai Livestream Resources + 2 Days Left for the SEEED Discount

Vijay Janapa Reddi — Tue, 23 Dec 2025 00:00:00 GMT

Hello Everyone,

We wanted to share a quick follow-up and make sure you have easy access to the resources from our recent MLSys community livestream hosted by Edge AI Foundation.

Here are the key links:

📄 Slides (PPT)
▶️ YouTube recording

We’re also excited to share that TinyTorch.ai has officially launched. TinyTorch is a lightweight, educational framework designed to help learners understand how deep learning systems work from the ground up, without the abstraction of large production libraries. If you haven’t explored it yet, the holiday break can be a great time to tinker and experiment at your own pace.

As a reminder, there are just two days left to take advantage of the SEEED 20% discount code (VWP6YDXH) mentioned during the livestream. If you’ve been considering a kit, now’s a great time as the code expires on 12/24.

As we wrap up the year, we want to thank you again for being part of this growing community. We’re excited about what’s ahead in the new year and look forward to sharing more workshops, resources, and opportunities to learn together.

Wishing you a happy, healthy start to the new year.

All my best,
Kari Janapareddi

Director of Partnerships and Strategic Operations
Machine Learning Systems Education

Thank You for Joining the MLSysBook.ai Community Celebration

Vijay Janapa Reddi — Wed, 17 Dec 2025 00:00:00 GMT

Hello everyone,

Thanks to those of you who joined us today for the MLSysBook.ai 10K Stars Community Celebration—and a very warm welcome to those of you who subscribed today. We’re grateful you’re here and excited to have you as part of this growing global community.

If you missed it, here are the links…

To the livestream

To the slides

RECAP: During the livestream, we shared how MLSysBook.ai is evolving into an open, living resource for machine learning systems education and how this work is being amplified worldwide through community-led outreach. We highlighted global workshops, online show & tell spaces, and the open sharing of student projects, and real-world deployments. All of this is made possible by a remarkable group of volunteers, educators, and partners who believe in making ML systems education accessible everywhere.

If you’d like to support this work (click here!), donations directly fund workshops, kit donations, and the continued development of open educational materials. As a small thank-you:

Every donation of $100 or more today receives a free Edge AI mug
All new subscribers & donators today are entered to win a free SEEED XIAOML kit

And a reminder—this week only, you can get 20% off SEEED XIAOML kits using our special community discount code (VWP6YDXH) during checkout.

We’d also love to hear from you. If there are topics, workshops, tools, or community activities you’d like to see in the future, please reply to this email and let us know! Your feedback helps shape where we go next.

Thank you again for being part of this milestone moment. Your curiosity, participation, and support help us continue building an inclusive, global ML systems community.

All my best,
Kari Janapareddi
On behalf of the MLSysBook Community

MLSysBook.ai Live Event: Global Community Updates + AMA

Vijay Janapa Reddi — Tue, 16 Dec 2025 00:00:00 GMT

Hello and welcome to the MLSysBook community!

We’re excited to invite you to an upcoming live MLSysBook.ai community event tomorrow (Wednesday Dec 17th, 10EST) . You can find the full event details and RSVP via our LinkedIn event page here:
https://www.linkedin.com/events/7402387002232520704/

This event is a snapshot of what our community has been building together. We’ll share highlights from our global outreach efforts, including how educators and volunteers around the world are coordinating hands-on workshops, running online community spaces, and openly sharing learning materials, tools, and real-world ML systems projects. At the heart of this work is a simple goal: making machine learning systems education accessible, practical, and truly global.

We’ll wrap things up with an open AMA with Professor Vijay Janapa Reddi. This is your chance to ask questions about the MLSysBook vision, the evolving ML systems discipline, community priorities, and what’s coming next.

We hope you’ll join us, bring your curiosity, and be part of the conversation.

Sincerely,

The MLSysBook Community Team