Fractional-Order Deep Backpropagation Neural Network

Chunhui Bao, Yifei Pu, Yi Zhang

Abstract

In recent years, the research of artificial neural networks based on fractional calculus has attracted much attention. In this paper, we proposed a fractional-order deep backpropagation (BP) neural network model with <svg xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg" style="vertical-align:-3.1815pt" id="M1" height="11.8174pt" version="1.1" viewBox="-0.0498162 -8.6359 13.3712 11.8174" width="13.3712pt"><g transform="matrix(.013,0,0,-0.013,0,0)"><path id="g113-77" d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z"/></g><g transform="matrix(.0091,0,0,-0.0091,8.294,3.132)"><path id="g50-51" d="M414 144C384 79 371 75 317 75H135L276 221C367 316 408 376 408 465C408 570 327 635 237 635C179 635 131 609 100 575L42 494L67 471C94 510 138 565 205 565C277 565 321 517 321 435C321 348 258 270 195 195C146 137 88 81 33 26V0H411C423 44 433 88 446 135L414 144Z"/></g></svg> regularization. The proposed network was optimized by the fractional gradient descent method with Caputo derivative. We also illustrated the necessary conditions for the convergence of the proposed network. The influence of <svg xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg" style="vertical-align:-3.1815pt" id="M2" height="11.8174pt" version="1.1" viewBox="-0.0498162 -8.6359 13.3712 11.8174" width="13.3712pt"><g transform="matrix(.013,0,0,-0.013,0,0)"><use xlink:href="#g113-77"/></g><g transform="matrix(.0091,0,0,-0.0091,8.294,3.132)"><use xlink:href="#g50-51"/></g></svg> regularization on the convergence was analyzed with the fractional-order variational method. The experiments have been performed on the MNIST dataset to demonstrate that the proposed network was deterministically convergent and can effectively avoid overfitting.
BibTeX
@article{bao2018fractional,
  author = {Chunhui Bao and Yifei Pu and Yi Zhang},
  title = {{Fractional-Order Deep Backpropagation Neural Network}},
  journal = {Comput. Intell. Neurosci},
  year = {2018},
  doi = {10.1155/2018/7361628},
}

Loading related content...

Figure