Putting the "Machine" Back in Machine Learning for Engineering Students

Ting-Wu Chin, Dimitrios Stamoulis, Diana MarculescuTeaching ML 2021 · OpenReview

Abstract

Computer hardware architecture has played an important role in the recent advances made in deep learning and associated applications. However, effective teaching strategies for hardware architecture are often decoupled from the software-driven machine learning curriculum. This paper outlines the design and success of a graduate-level course that bridges the gap between machine learning (ML) algorithms and the hardware they run on. By moving beyond "black-box" software, the course teaches engineering students how to optimize neural networks for real-world constraints like energy efficiency, latency, and hardware area.

Introduction

The "Deep Learning Revolution" was fundamentally enabled by hardware advancements, specifically the rise of GPUs and specialized accelerators. However, computer science curricula often treat machine learning (ML) as a purely mathematical or software-driven discipline. As ML applications migrate toward "edge" devices—such as microcontrollers and mobile phones—engineers can no longer ignore hardware constraints. There is a significant educational gap: students often know how to build a high-accuracy model in PyTorch, but they may not understand why that model is too slow or power-hungry for a specific chip. This paper presents a pedagogical framework designed to bridge the gap between ML algorithms and their underlying hardware architectures.

Methodology

The authors developed a curriculum for Electrical and Computer Engineering (ECE) graduate students that emphasizes "Hardware Architectures for Machine Learning." Unlike traditional ML courses, this curriculum focuses on the physical implications of software choices through several key methods:

Course syllabus and schedule outlining the integration of lectures, homework, projects, and paper presentations.

As shown in the course schedule (Figure 1), the curriculum incorporates several unique elements:

  • Research-Driven Content: Due to the rapid pace of the field, the course utilizes recently published technical papers instead of a traditional textbook.
  • Implementation-Based Assignments: Students use hardware modeling tools, such as the "Tetris" accelerator simulator, to quantify how neural network configurations impact physical metrics like cycles and joules. Initial assignments utilize the MNIST dataset (LeNet-5), while advanced projects involve modern architectures like NasNet and MobileNetV2.
  • Hardware-Software Co-exploration: The curriculum teaches students to simultaneously optimize both the neural network architecture and the hardware design, rather than treating one as a fixed constraint.
  • Active Learning Components: Grading is heavily weighted toward a semester-long project (55%) and student-led paper presentations (15%), fostering deep engagement with state-of-the-art research.

Key Insights

A central theme of the course is the trade-off between computational intensity (FLOPs) and predictive accuracy. Students are encouraged to visualize these trade-offs—such as Accuracy vs. Runtime or Throughput vs. Design Area—to identify hardware-software bottlenecks. By utilizing Python-based hardware models, students can experiment with custom hardware designs without the need for expensive physical FPGA or ASIC equipment. Furthermore, the course successfully integrated students from diverse backgrounds (e.g., circuits/logic vs. systems/architecture), facilitating peer-to-peer learning.

Results and Evaluation

The course was highly successful based on student evaluations and qualitative feedback. It received high satisfaction ratings, ranging from 4.4 to 4.6 out of 5.0. Students reported that the workload was balanced and provided sufficient depth to understand the field's essence. The "active learning" format was particularly praised for fostering high-quality technical discussions often absent in standard lecture-heavy formats.

Discussion and Future Work

While successful, the course's reliance on small-group discussions and specialized TAs may present scaling challenges for larger undergraduate classes. Additionally, while simulators are effective for learning, a gap remains between Python models and the complexities of physical hardware deployment, such as thermal issues and memory bus contention. Future iterations of the course aim to expand coverage of distributed/federated learning and hardware for unsupervised learning, ensuring the curriculum keeps pace with the evolving landscape of machine learning hardware.

Summary updated

Abstract

Computer hardware architecture has played an important role in the recent advances made in deep learning and associated applications. However, effective teaching strategies for hardware architectur...
BibTeX
@inproceedings{chin2021putting,
  author = {Ting-Wu Chin and Dimitrios Stamoulis and Diana Marculescu},
  title = {{Putting the "Machine" Back in Machine Learning for Engineering Students}},
  booktitle = {Teaching ML},
  year = {2021},
  url = {https://openreview.net/forum?id=hOEl4OCg0rZO},
}
Figure