Architecture vs Algorithm
There are many deep learning (DL) architectures, artificial neural networks (ANNs) in general, and some optimization algorithms (OAs) for training them. There hasn’t been much interest in inventing new OAs as there has been for ANNs. That is why there have been many DL-archs versus one or few optimizers, all of which are based on some form of gradient descent. But why is it…