Optimized Deep learning

n mchn lrnng nd sttstcs, th lrnng rt s tnng prmtr n n ptmztn lgrthm tht dtrmns th stp sz t ch trtn whl mvng twrd mnmm f lss fnctn. Snc t nflncs t wht xtnt nwly cqrd nfrmtn vrrds ld nfrmtn, t mtphrclly rprsnts th spd t whch mchn lrnng mdl "lrns". n th dptv cntrl ltrtr, th lrnng rt s cmmnly rfrrd t s gn. n sttng lrnng rt, thr s trd-ff btwn th rt f cnvrgnc nd vrshtng. Whl th dscnt drctn s slly dtrmnd frm th grdnt f th lss fnctn, th lrnng rt dtrmns hw bg stp s tkn n tht drctn. t hgh lrnng rt wll mk th lrnng jmp vr mnm bt t lw lrnng rt wll thr tk t lng t cnvrg r gt stck n n ndsrbl lcl mnmm. n rdr t chv fstr cnvrgnc, prvnt sclltns nd gttng stck n ndsrbl lcl mnm th lrnng rt s ftn vrd drng trnng thr n ccrdnc t lrnng rt schdl r by sng n dptv lrnng rt. Th lrnng rt nd ts djstmnts my ls dffr pr prmtr, n whch cs t s dgnl mtrx tht cn b ntrprtd s n pprxmtn t th nvrs f th Hssn mtrx n Nwtn's mthd. Th lrnng rt s rltd t th stp lngth dtrmnd by nxct ln srch n qs-Nwtn mthds nd rltd ptmztn lgrthms.