Design and Parametric Study of a MEMS-based Reservoir Computer for Reinforcement Learning

Author

• Andrew P. Carr, California Polytechnic State University, San Luis ObispoFollow

Available at: https://digitalcommons.calpoly.edu/theses/3304

Date of Award

6-2026

Degree Name

MS in Mechanical Engineering

Department/Program

Mechanical Engineering

College

College of Engineering

Advisor

Mohammad Hasan

Advisor Department

Mechanical Engineering

Advisor College

College of Engineering

Abstract

Single-node reservoir computing (RC) is a hardware-efficient approach to machine learning, leveraging the dynamics of physical systems. In this work, two reinforcement learning algorithms, Q-learning and Proximal Policy Optimization (PPO), are applied to a simulated micro-electro-mechanical system (MEMS)-based reservoir computer to solve both discrete and continuous control tasks. MEMS-based reservoirs are low-power, compact, and their natural frequencies (kHz to MHz) pair well with real-time control loops. To explore the relationship between reservoir dynamics and learning performance, a parametric study is conducted on two reservoir hyperparameters, reservoir size and neuron separation, using CartPole-v1 and MountainCar-v0. The RC successfully learns multiple tasks with both algorithms. Most notably, the system learned a successful gait for the 4-degree-of-freedom bipedal locomotion task, BipedalWalker-v3. In the parametric studies, increasing reservoir size led to increased robustness to random initialization while increasing typical performance. A strong sensitivity to reservoir initialization was observed, motivating reservoir initialization searches during tuning. Optimal neuron separation was shown to be task dependent; however, each task was solved with neuron separations near the fundamental period of the MEMS device. In contrast to ultra-fast optical RCs, MEMS-based reservoir computers can operate at frequencies compatible with real-time sensing and control, enabling integration into mechanical systems. Overall, this work demonstrates the successful application of reinforcement learning algorithms to MEMS-based reservoir computing on both discrete and continuous control tasks while highlighting the relationship between reservoir dynamics and reinforcement learning performance.