You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Welcome to the Edge Impulse open courseware for embedded machine learning! This repository houses a collection of slides, reading material, project prompts, and sample questions to get you started creating your own embedded machine learning course. You will also have access to videos that cover much of the material. You are welcome to share these videos with your class either in the classroom or let students watch them on their own time.
This repository is part of the Edge Impulse University Program. Please see this page for more information on how to join: edgeimpulse.com/university.
How to Use This Repository
Please note that the content in this repository is not intended to be a full semester-long course. Rather, you are encouraged to pull from the modules, rearrange the ordering, make modifications, and use as you see fit to integrate the content into your own curriculum.
For example, many of the lectures and examples from the TinyML Courseware (given by [3]) go into detail about how TensorFlow Lite works along with advanced topics like quantization. Feel free to skip those sections if you would just like an overview of embedded machine learning and how to use it with Edge Impulse.
In general, content from [3] cover theory and hands-on Python coding with Jupyter Notebooks to demonstrate these concepts. Content from [1] and [2] cover hands-on demonstrations and projects using Edge Impulse to deploy machine learning models to embedded systems.
Content is divided into separate modules. Each module is assumed to be about a week's worth of material, and each section within a module contains about 60 minutes of presentation material. Modules also contain example quiz/test questions, practice problems, and hands-on assignments.
If you would like to see more content than what is available in this repository, please refer to the Harvard TinyMLedu site for additional course material.
The YouTube videos in this repository are shared via the standard YouTube license. You are allowed to show them to your class or provide links for students (and others) to watch.
Professional Development
Much of the material found in this repository is curated from a collection of online courses with permission from the original creators. You are welcome to take the courses (as professional development) to learn the material in a guided fashion or refer students to the courses for additional learning opportunities.
Introduction to Embedded Machine Learning - Coursera course by Edge Impulse that introduces neural networks and deep learning concepts and applies them to embedded systems. Hands-on projects rely on training and deploying machine learning models with Edge Impulse. Free with optional paid certificate.
Computer Vision with Embedded Machine Learning - Follow-on Coursera course that covers image classification and object detection using convolutional neural networks. Hands-on projects rely on training and deploying models with Edge Impulse. Free with optional paid certificate.
Optional prerequisites: many machine learning concepts can be quite advanced. While these advanced topics are briefly discussed in the slides and videos, they are not required for quiz questions and hands-on projects. If you would like to dig deeper into such concepts in your course, students may need to be familiar with the following:
Linear algebra
Matrix addition, subtraction, and multiplication
Dot product
Matrix transposition and inversion
Calculus
The derivative and chain rule are important for backpropagation (a part of model training)
Integrals and summation are used to find the area under a curve (AUC) for some model evaluations
Digital signal processing (DSP)
Sampling rate
Nyquist–Shannon sampling theorem
Fourier transform and fast Fourier transform (FFT)
If you find errors or have suggestions about how to make this material better, please let us know! You may create an issue describing your feedback or create a pull request if you are familiar with Git.
This repo uses automatic link checking and spell checking. If continuous integration (CI) fails after a push, you may find the dead links or misspelled words, fix them, and push again to re-trigger CI. If dead links or misspelled words are false positives (i.e. purposely malformed link or proper noun), you may update .mlc_config.json for links to ignore or .wordlist.txt for words to ignore.
Required Hardware and Software
Students will need a computer and Internet access to perform machine learning model training and hands-on exercises with the Edge Impulse Studio and Google Colab. Students are encouraged to use the Arduino Tiny Machine Learning kit to practice performing inference on an embedded device.
Students will need to install the latest Arduino IDE.
Preexisting Datasets and Projects
This is a collection of preexisting datasets, Edge Impulse projects, or curation tools to help you get started with your own edge machine learning projects. With a public Edge Impulse project, note that you can clone the project to your account and/or download the dataset from the Dashboard.
This module provides an overview of machine learning and how it can be used to solve problems. It also introduces the idea of running machine learning algorithms on resource-constrained devices, such as microcontrollers. It covers some of the limitations and ethical concerns of machine learning. Finally, it demonstrates a few examples of Python in Google Colab, which will be used in early modules to showcase how programming is often performed for machine learning with TensorFlow and Keras.
Learning Objectives
Describe the differences between artificial intelligence, machine learning, and deep learning
Provide examples of how machine learning can be used to solve problems (that traditional deterministic programming cannot)
Provide examples of how embedded machine learning can be used to solve problems (where other forms of machine learning would be limited or inappropriate)
Describe the limitations of machine learning
Describe the ethical concerns of machine learning
Describe the differences between supervised and unsupervised machine learning
This module provides an overview of neural networks and how they can be used to make predictions. Simple examples are given in Python (Google Colab) for students to play with and learn from. If you do not wish to explore basic machine learning with Keras in Google Colab, you may skip this module to move on to using the Edge Impulse graphical environment. Note that some later exercises rely on Google Colab for curating data, visualizing neural networks, etc.
Learning Objectives
Provide examples of how machine learning can be used to solve problems (that traditional deterministic programming cannot)
Provide examples of how embedded machine learning can be used to solve problems (where other forms of machine learning would be limited or inappropriate)
Describe challenges associated with running machine learning algorithms on embedded systems
Describe broadly how a mathematical model can be used to generalize trends in data
Explain how the training process results from minimizing a loss function
Describe why datasets should be broken up into training, validation, and test sets
Explain how overfitting occurs and how to identify it
Demonstrate the ability to train a dense neural network using Keras and TensorFlow
In this module, students will get an understanding of how data is collected and used to train a machine learning model. They will have the opportunity to collect their own dataset, upload it to Edge Impulse, and train a model using the graphical interface. From there, they will learn how to evaluate a model using a confusion matrix to calculate precision, recall, accuracy, and F1 scores.
Learning Objectives
Provide examples of how embedded machine learning can be used to solve problems (where other forms of machine learning would be limited or inappropriate)
Describe challenges associated with running machine learning algorithms on embedded systems
Describe why datasets should be broken up into training, validation, and test sets
Explain how overfitting occurs and how to identify it
Describe broadly what happens during machine learning model training
Describe the difference between model training and inference
Describe why test and validation datasets are needed
Evaluate a model's performance by calculating accuracy, precision, recall, and F1 scores
Demonstrate the ability to train a machine learning model with a given dataset and evaluate its performance
This module covers why quantization is important for models running on embedded systems and some of the limitations. It also shows how to use a model for inference and set an appropriate threshold to minimize false positives or false negatives, depending on the system requirements. Finally, it covers the steps to deploy a model trained on Edge Impulse to an Arduino board.
Learning Objectives
Provide examples of how embedded machine learning can be used to solve problems (where other forms of machine learning would be limited or inappropriate)
Describe challenges associated with running machine learning algorithms on embedded systems
Describe broadly what happens during machine learning model training
Describe the difference between model training and inference
Demonstrate the ability to perform inference on an embedded system to solve a problem
This module describes several approaches to anomaly detection and why we might want to use it in embedded machine learning.
Learning Objectives
Provide examples of how embedded machine learning can be used to solve problems (where other forms of machine learning would be limited or inappropriate)
Describe challenges associated with running machine learning algorithms on embedded systems
Describe broadly what happens during machine learning model training
Describe the difference between model training and inference
Demonstrate the ability to perform inference on an embedded system to solve a problem
Describe how anomaly detection can be used to solve problems
This module introduces the concept of image classification, why it is important in machine learning, and how it can be used to solve problems. Convolution and pooling operations are covered, which form the building blocks for convolutional neural networks (CNNs). Saliency maps and Grad-CAM are offered as two techniques for visualizing the inner gs of CNNs. Data augmentation is introduced as a method for generating new data from existing data to train a more robust model. Finally, transfer learning is shown as a way of reusing pretrained models.
Learning Objectives
Describe the differences between image classification, object detection, and image segmentation
Describe how embedded computer vision can be used to solve problems
Describe how convolution and pooling operations are used to filter and downsample images
Describe how convolutional neural networks differ from dense neural networks and how they can be used to solve computer vision problems
In this module, we look at object detection, how it differs from image classification, and the unique set of problems it solves. We also briefly examine image segmentation and discuss constrained object detection. Finally, we look at responsible AI as it relates to computer vision and AI at large.
Learning Objectives
Describe the differences between image classification, object detection, and image segmentation
Describe how embedded computer vision can be used to solve problems
Describe how image segmentation can be used to solve problems
Describe how convolution and pooling operations are used to filter and downsample images
Describe how convolutional neural networks differ from dense neural networks and how they can be used to solve computer vision problems
Describe the limitations of machine learning
Describe the ethical concerns of machine learning
Describe the requirements for collecting a good dataset and what factors can create a biased dataset
In this module, we create a functioning keyword spotting (also known as "wake word detection") system. To do so, we must introduce several concepts unique to audio digital signal processing and combine it with image classification techniques.
Learning Objectives
Describe how machine learning can be used to classify sounds
Describe how sound classification can be used to solve problems
Describe the major components in a keyword spotting system
Demonstrate the ability to train and deploy a sound classification system
