Performance comparison of neural network training algorithms in modeling of bimodal drug delivery

Alireza Ghaffari, Hamid Abdollahi, Mohammad Reza Khoshayand, Iman Soltani, A. Dadgar, Morteza Rafiee‐TehraniInternational Journal of Pharmaceutics 2006 · DOI

TL;DR

This paper compares five different neural network (NN) training algorithms for modeling bimodal drug delivery, finding that gradient descent backpropagation algorithms (IBP and BBP) offer superior predictive ability and efficiency compared to Levenberg-Marquardt (LM) and genetic algorithms (GA). The study also reveals that increasing coating levels and decreasing pectin-chitosan content generally retard drug release, with pectin-chitosan playing a more dominant role.

What Problem Does This Paper Address?

Developing effective drug delivery systems often involves complex, non-linear relationships between formulation parameters and drug release profiles. Traditional modeling techniques can struggle with these complexities. Artificial Neural Networks (ANNs) offer a powerful alternative for modeling such systems, but their performance is highly dependent on the chosen training algorithm. This paper addresses the critical need to compare the effectiveness of various ANN training algorithms for modeling a specific pharmaceutical challenge: bimodal drug delivery. Bimodal release, characterized by an initial slow release followed by a faster rate, can be therapeutically beneficial but requires precise control over formulation. The authors aim to identify the most suitable training algorithm for accurately predicting the in vitro release profile of theophylline from coated pellets, considering factors like coating weight gain and pectin-chitosan content.

Methods

The study employed a multilayer perceptron (MLP) feedforward neural network architecture with a single hidden layer of four nodes and 14 output nodes (representing drug release at different time points). Two causal factors – coating level amount (X1) and amount of pectin-chitosan in the coating solution (X2) – served as inputs.

Five training algorithms were evaluated, categorized into three classes:

  • Gradient Descent:
    • Incremental Backpropagation (IBP): Weights are updated after each training pattern.
  • Batch Backpropagation (BBP): Weights are updated once per iteration after processing all training patterns.
  • Quick Propagation (QP): A heuristic modification of backpropagation designed for faster learning.
  • Quasi-Newton:
    • Levenberg-Marquardt (LM): An approximation to the Newton method, known for its fast convergence.
  • Genetic Algorithm (GA): A stochastic search method inspired by natural selection.

The performance of these algorithms was assessed using metrics like Root Mean Squared Error (RMSE) for both training and test sets, mean prediction error (bias), CPU time, and R² values. The training process was terminated when the RMSE on the test set reached a minimum to prevent overfitting.

Datasets

The study utilized experimental data generated from a two-factor, three-level experimental design with additional points. This design involved preparing 17 different formulations of theophylline pellets coated with varying amounts of pectin-chitosan and Eudragit® RS. The drug release profiles of these formulations were measured at 14 different time points.

For training and evaluation:

  • Training Set: Formulations F1, F2, F4, F5, F6, F7, F8, F9, F10, F11, F12, F14, F15, F17 were used.
  • Test Set: Formulations F3, F13, and F16 were reserved for testing the generalization ability of the trained models.

Key Insights and Ideas

  • Gradient Descent Dominance: Incremental and batch backpropagation (IBP and BBP) consistently outperformed other algorithms in terms of predictive accuracy (lower RMSE and higher R²). This suggests that for this specific pharmaceutical modeling task, the direct, iterative adjustment of weights by gradient descent is more effective than the more complex search strategies of LM or GA.
  • BBP's Efficiency Edge: While IBP and BBP showed comparable predictive abilities, BBP achieved convergence significantly faster (3-4 times quicker) and required less CPU time, making it a more practical choice for large datasets.
  • LM's Moderate Performance: The Levenberg-Marquardt algorithm performed moderately well, showing better precision than QP but not as good as IBP and BBP. Interestingly, LM achieved a lower RMSE on the training set than on the test set, highlighting the importance of evaluating generalization.
  • GA's Erratic Nature: The genetic algorithm exhibited erratic training behavior and the least predictive ability among the tested algorithms. This aligns with the understanding that GA is a global search method that can struggle with the fine-tuning required for precise local modeling.
  • Pectin-Chitosan's Dominance: The amount of pectin-chitosan in the coating solution was found to have a more significant impact on drug release than the coating level itself, particularly in the initial stages.
  • Coating Level and Pectin-Chitosan Interaction: Increasing coating levels and decreasing pectin-chitosan content generally retarded drug release. However, the relationship was non-linear and time-dependent, with initial stages showing less predictable behavior.

Key Results

  • Predictive Ability Ranking: The order of predictive ability for the training algorithms was: IBP, BBP > LM > QP > GA.
  • RMSE Comparison: BBP achieved the lowest test RMSE (3.0668), while QP had the highest (6.5918). LM's test RMSE was 4.0998.
  • CPU Time: BBP required significantly less CPU time (4.2 s) compared to IBP (14.8 s) and QP (3.6 s). GA took the longest (25 s).
  • R² Values: BBP achieved a high R² of 0.9815 for test set prediction, indicating excellent model fit. LM achieved an R² of 0.9517, and GA showed a lower R² of 0.8329.
  • Factor Importance: The analysis of relative importance showed that both coating level (X1) and pectin-chitosan amount (X2) were significant, with X2 playing a slightly more dominant role (49.15% for X1 and 50.85% for X2).

Limitations and Future Work

  • GA Limitations: The paper notes that GA, while good for global search, can be inefficient for fine-tuning local optima, which is crucial for precise modeling. The authors suggest that hybridizing GA with gradient-based algorithms could be a promising avenue for future work to leverage the strengths of both approaches.
  • Computational Time: While BBP was efficient, GA training required a considerable amount of computation time. Hybrid approaches could also address this.
  • Generalizability of Findings: The study focused on a specific drug (theophylline) and formulation system. Further research could explore the applicability of these findings to other drugs and delivery systems.
  • Overtraining Prevention: The study used early stopping based on test set RMSE to prevent overtraining. While effective, exploring other regularization techniques could be beneficial.
  • Model Complexity: The study used a single hidden layer with four nodes. Investigating the impact of different network architectures on performance could be a future direction.

Summary updated

BibTeX
@article{ghaffari2006performance,
  author = {Alireza Ghaffari and Hamid Abdollahi and Mohammad Reza Khoshayand and Iman Soltani and A. Dadgar and Morteza Rafiee‐Tehrani},
  title = {{Performance comparison of neural network training algorithms in modeling of bimodal drug delivery}},
  journal = {International Journal of Pharmaceutics},
  year = {2006},
  doi = {10.1016/j.ijpharm.2006.07.056},
}
Figure