梯度下降、牛顿法、拟牛顿法详细介绍

本文转载自：https://blog.csdn.net/qy20115549/article/details/79771480

本文作者：合肥工业大学 管理学院 钱洋 email：1563178220@qq.com 内容可能有不到之处，欢迎交流。

**未经本人允许禁止转载**。
<font face="楷体" size=5 color=#D2691E>
最近，在看论文的时候，优化方法使用的是L-BFGS算法，所以我花了几天时间，看了梯度下降、牛顿法和逆牛顿法的原理及相关源码。以下我个人笔记，仅供大家参考。
</font>
#内容
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173851443?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173905583?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173913102?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173921463?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173930749?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173942382?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173950069?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331173959689?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174016773?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174024081?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174031000?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174039509?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174047050?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174055154?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174103875?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>
<center>
![这里写图片描述](http://img.blog.csdn.net/20180331174145659?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcXkyMDExNTU0OQ==/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast)
</center>

梯度下降、牛顿法、拟牛顿法详细介绍

欢迎大家关注DataLearner官方微信，接受最新的AI技术推送

相关博客

最热博客