I recently discussed how the nonparametric consistency of wide NN's proved underwhelming, which is partly why econometricians lost interest in NN's in the 1990s.
The other thing was the realization that NN objective surfaces are notoriously bumpy, so that arrival at a local optimum (e.g., by the stochastic gradient descent popular in NN circles) offered little comfort.
So econometricians' interest declined on both counts. But now both issues are being addressed. The new focus on NN depth as opposed to width is bearing much fruit. And recent advances in "reinforcement learning" methods effectively promote global as opposed to just local optimization, by experimenting (injecting randomness) in clever ways. (See, e.g., Taddy section 6, here.)
All told, it seems like quite an exciting new time for NN's. I've been away for 15 years. Time to start following again...
The other thing was the realization that NN objective surfaces are notoriously bumpy, so that arrival at a local optimum (e.g., by the stochastic gradient descent popular in NN circles) offered little comfort.
So econometricians' interest declined on both counts. But now both issues are being addressed. The new focus on NN depth as opposed to width is bearing much fruit. And recent advances in "reinforcement learning" methods effectively promote global as opposed to just local optimization, by experimenting (injecting randomness) in clever ways. (See, e.g., Taddy section 6, here.)
All told, it seems like quite an exciting new time for NN's. I've been away for 15 years. Time to start following again...
—
Monday, February 26, 2018
—
1 Comment
—
Forecasting,
Machine learning,
Networks,
Nonlinearity,
Nonparametrics
I am genuinely thankful to the holder of this web page who has shared this wonderful paragraph at at this place Visit Pocket Option here
ReplyDelete