Bayesian Statistics as an Alternative to Gradient Descent in Sequence Learning

Rainer Spiegel

Abstract


Recurrent neural networks are frequently applied to simulate sequence learning applications such as language processing, sensory-motor learning, etc. For this purpose, they often apply a truncated gradient descent (=error correcting) learning algorithm. In order to converge to a solution that is congruent with a target set of sequences, many iterations of sequence presentations and weight adjustments are typically needed. Moreover, there is no guarantee of finding the global minimum of error in a multidimensional error landscape resulting from the discrepancy between target values and the networkâ??s prediction. This paper presents a new approach of inferring the global error minimum right from the start. It further applies this information to reverse-engineer the weights. As a consequence, learning is speeded-up tremendously, whilst computationally-expensive iterative training trials can be skipped. Technology applications in established and emerging industries will be discussed.

Full Text:

PDF


Copyright (c) 2017 Rainer Spiegel


International Journal of Emerging Technologies in Learning (iJET) – eISSN: 1863-0383
Creative Commons License
Indexing:
Scopus logo Clarivate Analyatics ESCI logo EI Compendex logo IET Inspec logo DOAJ logo DBLP logo Learntechlib logo EBSCO logo Ulrich's logo Google Scholar logo MAS logo