Abstract
Advancements in additive manufacturing have enabled design and fabrication of materials and structures not previously realizable. In particular, the design space of composite materials and structures has vastly expanded, and the resulting size and complexity has challenged traditional design methodologies, such as brute force exploration and one factor at a time (OFAT) exploration, to find optimum or tailored designs. To address this challenge, supervised machine learning approaches have emerged to model the design space using curated training data; however, the selection of the training data is often determined by the user. In this work, we develop and utilize a Reinforcement learning (RL)-based framework for the design of composite structures which avoids the need for user-selected training data. For a 5 $\times$ 5 composite design space comprised of soft and compliant blocks of constituent material, we find that using this approach, the model can be trained using 2.78% of the total design space consists of $2^{25}$ design possibilities. Additionally, the developed RL-based framework is capable of finding designs at a success rate exceeding 90%. The success of this approach motivates future learning frameworks to utilize RL for the design of composites and other material systems.
Introduction
Additive manufacturing has opened up a nearly infinite design space for composite materials. However, finding the exact arrangement of stiff and soft materials to achieve a specific mechanical property is a massive computational challenge. For a simple $5 \times 5$ grid of two materials, there are $2^{25}$ (over 33 million) possible configurations. Traditional "brute force" searches are too slow, and supervised machine learning models typically require massive, human-curated datasets that are expensive to generate via simulations or physical experiments. This research addresses the "curse of dimensionality" by investigating whether an AI can learn to design these materials through trial and error, without needing a pre-existing library of designs.
Methods
The design process is framed as a Reinforcement Learning (RL) problem. Instead of predicting a property from a design, an agent iteratively modifies a design to reach a user-specified goal ($E_{goal}$).
The framework utilizes Deep Q-Learning to estimate actions (flipping cells between stiff and soft) that bring the design closer to the target. A critical component is Hindsight Experience Replay (HER), which allows the agent to learn from "accidental" successes—designs that didn't meet the original goal but achieved a different modulus. This significantly improves data efficiency. A physics-based Finite Element Analysis (FEA) simulator serves as the environment, providing real-time feedback on the Young’s modulus of current designs.
To validate the agent, an exhaustive dataset of all $2^{25}$ designs was generated using a parallelized FEA solver. Furthermore, physical samples were 3D-printed using VeroWhitePlus (stiff) and TangoBlackPlus (soft) and tested to ensure simulation accuracy.
Results and Discussion
The RL agent achieved a success rate exceeding 90% while exploring only 2.78% of the total design space during training. This efficiency stems from the agent learning the underlying logic of material placement rather than simply memorizing configurations.
The RL approach significantly outperformed nearest-neighbor baselines, particularly in complex regions of the design space where many different configurations yield similar properties. Experimental validation confirmed the high accuracy of the FEA model, with a Root Mean Squared Error (RMSE) of only 98.7 MPa. The agent also demonstrates "design intent" preservation, making minimal strategic changes to reach a goal, which is valuable for practical engineering refinement.
Conclusion
While successful for $5 \times 5$ grids, scaling this framework to larger or 3D grids will require more advanced architectures to handle the exponentially larger state space. Future work will also explore multi-objective optimization to balance properties like strength, toughness, and thermal conductivity. This study demonstrates that RL is a powerful tool for the inverse design of complex composite structures, bypassing the need for massive pre-curated datasets.
Summary updated