[{"data":1,"prerenderedAt":8852},["ShallowReactive",2],{"post-\u002Fblog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation":3},{"id":4,"title":5,"body":6,"cardimage":8837,"description":8831,"draft":8838,"enableComment":8839,"extension":8840,"image":8831,"meta":8841,"navigation":8839,"onday":8842,"path":8843,"seo":8844,"stem":8845,"summary":8846,"tags":8847,"__hash__":8851},"blog\u002Fblog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation.md","Explanation of Neural Network From Maximum Likelihood Estimation",{"type":7,"value":8,"toc":8830},"minimark",[9,14,18,21,27,77,80,143,146,337,373,429,648,730,786,789,899,902,906,909,1161,1194,1201,1296,1302,1603,1714,1721,1973,1976,2248,2280,2283,2612,2615,2618,2732,2738,2852,2856,3032,3201,3204,3427,3430,3740,3743,4108,4111,4477,4509,4952,4955,5295,5331,5721,5927,6019,6022,6122,6157,6568,6571,6739,6742,7357,7360,7886,7918,7921,8321,8324,8782,8814,8817,8824],[10,11,13],"h2",{"id":12},"coin-tossing","Coin Tossing",[15,16,17],"p",{},"MLE (Maximum Likelihood Estimation) is one of the core ideas in statistics.",[15,19,20],{},"One-sentence summary:",[22,23,24],"blockquote",{},[15,25,26],{},"MLE selects, from all possible model parameters, the one that \"makes the observed data most likely.\"",[15,28,29,30,76],{},"Below, you find a damaged coin with uneven texture. So you don't know if it is fair. That is, when you toss this coin, you don't know the probability of heads or tails. You want to know: what is the probability ",[31,32,36,57],"span",{"className":33,"translate":35},[34],"katex","no",[31,37,40],{"className":38},[39],"katex-mathml",[41,42,44],"math",{"xmlns":43},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML",[45,46,47,53],"semantics",{},[48,49,50],"mrow",{},[51,52,15],"mi",{},[54,55,15],"annotation",{"encoding":56},"application\u002Fx-tex",[31,58,62],{"className":59,"ariaHidden":61},[60],"katex-html","true",[31,63,66,71],{"className":64},[65],"base",[31,67],{"className":68,"style":70},[69],"strut","height:0.625em;vertical-align:-0.1944em;",[31,72,15],{"className":73},[74,75],"mord","mathnormal"," of getting heads.",[15,78,79],{},"Thus, you decide to toss it 10 times consecutively and obtain the following result: heads 8 times, tails 2 times.",[15,81,82,83,142],{},"Suppose this were a fair coin. Then we set the prior hypothesis ",[31,84,86,108],{"className":85,"translate":35},[34],[31,87,89],{"className":88},[39],[41,90,91],{"xmlns":43},[45,92,93,105],{},[48,94,95,97,101],{},[51,96,15],{},[98,99,100],"mo",{},"=",[102,103,104],"mn",{},"0.5",[54,106,107],{"encoding":56},"p=0.5",[31,109,111,132],{"className":110,"ariaHidden":61},[60],[31,112,114,117,120,125,129],{"className":113},[65],[31,115],{"className":116,"style":70},[69],[31,118,15],{"className":119},[74,75],[31,121],{"className":122,"style":124},[123],"mspace","margin-right:0.2778em;",[31,126,100],{"className":127},[128],"mrel",[31,130],{"className":131,"style":124},[123],[31,133,135,139],{"className":134},[65],[31,136],{"className":137,"style":138},[69],"height:0.6444em;",[31,140,104],{"className":141},[74],".",[15,144,145],{},"Under this prior hypothesis, the probability of obtaining the above result is:",[31,147,150],{"className":148,"translate":35},[149],"katex-display",[31,151,153,205],{"className":152,"translate":35},[34],[31,154,156],{"className":155},[39],[41,157,159],{"xmlns":43,"display":158},"block",[45,160,161,202],{},[48,162,163,166,170,173,177,179,181,183,186,188,196,199],{},[51,164,165],{},"P",[98,167,169],{"stretchy":168},"false","(",[51,171,172],{},"D",[51,174,176],{"mathvariant":175},"normal","∣",[51,178,15],{},[98,180,100],{},[102,182,104],{},[98,184,185],{"stretchy":168},")",[98,187,100],{},[189,190,191,193],"msup",{},[102,192,104],{},[102,194,195],{},"10",[98,197,198],{},"≈",[102,200,201],{},"0.000976",[54,203,204],{"encoding":56},"P(D|p=0.5)\n=\n0.5^{10}\n\\approx 0.000976",[31,206,208,242,264,328],{"className":207,"ariaHidden":61},[60],[31,209,211,215,219,223,227,230,233,236,239],{"className":210},[65],[31,212],{"className":213,"style":214},[69],"height:1em;vertical-align:-0.25em;",[31,216,165],{"className":217,"style":218},[74,75],"margin-right:0.1389em;",[31,220,169],{"className":221},[222],"mopen",[31,224,172],{"className":225,"style":226},[74,75],"margin-right:0.0278em;",[31,228,176],{"className":229},[74],[31,231,15],{"className":232},[74,75],[31,234],{"className":235,"style":124},[123],[31,237,100],{"className":238},[128],[31,240],{"className":241,"style":124},[123],[31,243,245,248,251,255,258,261],{"className":244},[65],[31,246],{"className":247,"style":214},[69],[31,249,104],{"className":250},[74],[31,252,185],{"className":253},[254],"mclose",[31,256],{"className":257,"style":124},[123],[31,259,100],{"className":260},[128],[31,262],{"className":263,"style":124},[123],[31,265,267,271,275,319,322,325],{"className":266},[65],[31,268],{"className":269,"style":270},[69],"height:0.8641em;",[31,272,274],{"className":273},[74],"0.",[31,276,278,282],{"className":277},[74],[31,279,281],{"className":280},[74],"5",[31,283,286],{"className":284},[285],"msupsub",[31,287,290],{"className":288},[289],"vlist-t",[31,291,294],{"className":292},[293],"vlist-r",[31,295,298],{"className":296,"style":270},[297],"vlist",[31,299,301,306],{"style":300},"top:-3.113em;margin-right:0.05em;",[31,302],{"className":303,"style":305},[304],"pstrut","height:2.7em;",[31,307,313],{"className":308},[309,310,311,312],"sizing","reset-size6","size3","mtight",[31,314,316],{"className":315},[74,312],[31,317,195],{"className":318},[74,312],[31,320],{"className":321,"style":124},[123],[31,323,198],{"className":324},[128],[31,326],{"className":327,"style":124},[123],[31,329,331,334],{"className":330},[65],[31,332],{"className":333,"style":138},[69],[31,335,201],{"className":336},[74],[15,338,339,340,368,369],{},"That is, if this were a fair coin, then the probability of getting 8 heads and 2 tails in 10 tosses is about ",[31,341,343,356],{"className":342,"translate":35},[34],[31,344,346],{"className":345},[39],[41,347,348],{"xmlns":43},[45,349,350,354],{},[48,351,352],{},[102,353,201],{},[54,355,201],{"encoding":56},[31,357,359],{"className":358,"ariaHidden":61},[60],[31,360,362,365],{"className":361},[65],[31,363],{"className":364,"style":138},[69],[31,366,201],{"className":367},[74]," — ",[370,371,372],"strong",{},"clearly, this number is considered impossible in probability and statistics. So you can clearly feel that this coin is not fair.",[15,374,375,376,428],{},"Now we propose another hypothesis: suppose the probability of heads is ",[31,377,379,398],{"className":378,"translate":35},[34],[31,380,382],{"className":381},[39],[41,383,384],{"xmlns":43},[45,385,386,395],{},[48,387,388,390,392],{},[51,389,15],{},[98,391,100],{},[102,393,394],{},"0.8",[54,396,397],{"encoding":56},"p=0.8",[31,399,401,419],{"className":400,"ariaHidden":61},[60],[31,402,404,407,410,413,416],{"className":403},[65],[31,405],{"className":406,"style":70},[69],[31,408,15],{"className":409},[74,75],[31,411],{"className":412,"style":124},[123],[31,414,100],{"className":415},[128],[31,417],{"className":418,"style":124},[123],[31,420,422,425],{"className":421},[65],[31,423],{"className":424,"style":138},[69],[31,426,394],{"className":427},[74],". Then the probability of the above result is:",[31,430,432],{"className":431,"translate":35},[149],[31,433,435,488],{"className":434,"translate":35},[34],[31,436,438],{"className":437},[39],[41,439,440],{"xmlns":43,"display":158},[45,441,442,485],{},[48,443,444,446,448,450,452,454,456,458,460,462,469,472,480,482],{},[51,445,165],{},[98,447,169],{"stretchy":168},[51,449,172],{},[51,451,176],{"mathvariant":175},[51,453,15],{},[98,455,100],{},[102,457,394],{},[98,459,185],{"stretchy":168},[98,461,100],{},[189,463,464,466],{},[102,465,394],{},[102,467,468],{},"8",[98,470,471],{},"×",[189,473,474,477],{},[102,475,476],{},"0.2",[102,478,479],{},"2",[98,481,198],{},[102,483,484],{},"0.0067",[54,486,487],{"encoding":56},"P(D|p=0.8)\n=\n0.8^8\n\\times\n0.2^2 \\approx 0.0067",[31,489,491,521,542,592,639],{"className":490,"ariaHidden":61},[60],[31,492,494,497,500,503,506,509,512,515,518],{"className":493},[65],[31,495],{"className":496,"style":214},[69],[31,498,165],{"className":499,"style":218},[74,75],[31,501,169],{"className":502},[222],[31,504,172],{"className":505,"style":226},[74,75],[31,507,176],{"className":508},[74],[31,510,15],{"className":511},[74,75],[31,513],{"className":514,"style":124},[123],[31,516,100],{"className":517},[128],[31,519],{"className":520,"style":124},[123],[31,522,524,527,530,533,536,539],{"className":523},[65],[31,525],{"className":526,"style":214},[69],[31,528,394],{"className":529},[74],[31,531,185],{"className":532},[254],[31,534],{"className":535,"style":124},[123],[31,537,100],{"className":538},[128],[31,540],{"className":541,"style":124},[123],[31,543,545,549,552,581,585,589],{"className":544},[65],[31,546],{"className":547,"style":548},[69],"height:0.9474em;vertical-align:-0.0833em;",[31,550,274],{"className":551},[74],[31,553,555,558],{"className":554},[74],[31,556,468],{"className":557},[74],[31,559,561],{"className":560},[285],[31,562,564],{"className":563},[289],[31,565,567],{"className":566},[293],[31,568,570],{"className":569,"style":270},[297],[31,571,572,575],{"style":300},[31,573],{"className":574,"style":305},[304],[31,576,578],{"className":577},[309,310,311,312],[31,579,468],{"className":580},[74,312],[31,582],{"className":583,"style":584},[123],"margin-right:0.2222em;",[31,586,471],{"className":587},[588],"mbin",[31,590],{"className":591,"style":584},[123],[31,593,595,598,601,630,633,636],{"className":594},[65],[31,596],{"className":597,"style":270},[69],[31,599,274],{"className":600},[74],[31,602,604,607],{"className":603},[74],[31,605,479],{"className":606},[74],[31,608,610],{"className":609},[285],[31,611,613],{"className":612},[289],[31,614,616],{"className":615},[293],[31,617,619],{"className":618,"style":270},[297],[31,620,621,624],{"style":300},[31,622],{"className":623,"style":305},[304],[31,625,627],{"className":626},[309,310,311,312],[31,628,479],{"className":629},[74,312],[31,631],{"className":632,"style":124},[123],[31,634,198],{"className":635},[128],[31,637],{"className":638,"style":124},[123],[31,640,642,645],{"className":641},[65],[31,643],{"className":644,"style":138},[69],[31,646,484],{"className":647},[74],[15,649,650,651,701,702,142],{},"We also find that when ",[31,652,654,671],{"className":653,"translate":35},[34],[31,655,657],{"className":656},[39],[41,658,659],{"xmlns":43},[45,660,661,669],{},[48,662,663,665,667],{},[51,664,15],{},[98,666,100],{},[102,668,394],{},[54,670,397],{"encoding":56},[31,672,674,692],{"className":673,"ariaHidden":61},[60],[31,675,677,680,683,686,689],{"className":676},[65],[31,678],{"className":679,"style":70},[69],[31,681,15],{"className":682},[74,75],[31,684],{"className":685,"style":124},[123],[31,687,100],{"className":688},[128],[31,690],{"className":691,"style":124},[123],[31,693,695,698],{"className":694},[65],[31,696],{"className":697,"style":138},[69],[31,699,394],{"className":700},[74],", the probability of the above result is maximized, achieving the maximum value ",[31,703,705,718],{"className":704,"translate":35},[34],[31,706,708],{"className":707},[39],[41,709,710],{"xmlns":43},[45,711,712,716],{},[48,713,714],{},[102,715,484],{},[54,717,484],{"encoding":56},[31,719,721],{"className":720,"ariaHidden":61},[60],[31,722,724,727],{"className":723},[65],[31,725],{"className":726,"style":138},[69],[31,728,484],{"className":729},[74],[31,731,733],{"className":732,"translate":35},[149],[31,734,736,755],{"className":735,"translate":35},[34],[31,737,739],{"className":738},[39],[41,740,741],{"xmlns":43,"display":158},[45,742,743,752],{},[48,744,745,747,750],{},[102,746,484],{},[98,748,749],{},">",[102,751,201],{},[54,753,754],{"encoding":56},"0.0067\n>\n0.000976",[31,756,758,777],{"className":757,"ariaHidden":61},[60],[31,759,761,765,768,771,774],{"className":760},[65],[31,762],{"className":763,"style":764},[69],"height:0.6835em;vertical-align:-0.0391em;",[31,766,484],{"className":767},[74],[31,769],{"className":770,"style":124},[123],[31,772,749],{"className":773},[128],[31,775],{"className":776,"style":124},[123],[31,778,780,783],{"className":779},[65],[31,781],{"className":782,"style":138},[69],[31,784,201],{"className":785},[74],[15,787,788],{},"This shows that if the probability of heads is 0.8, then the likelihood of observing this data is greatest.",[15,790,791,792],{},"Thus: ",[31,793,795,819],{"className":794,"translate":35},[34],[31,796,798],{"className":797},[39],[41,799,800],{"xmlns":43},[45,801,802,816],{},[48,803,804,812,814],{},[805,806,807,809],"mover",{"accent":61},[51,808,15],{},[98,810,811],{},"^",[98,813,100],{},[102,815,394],{},[54,817,818],{"encoding":56},"\\hat p=0.8",[31,820,822,890],{"className":821,"ariaHidden":61},[60],[31,823,825,829,881,884,887],{"className":824},[65],[31,826],{"className":827,"style":828},[69],"height:0.8889em;vertical-align:-0.1944em;",[31,830,833],{"className":831},[74,832],"accent",[31,834,837,872],{"className":835},[289,836],"vlist-t2",[31,838,840,867],{"className":839},[293],[31,841,844,854],{"className":842,"style":843},[297],"height:0.6944em;",[31,845,847,851],{"style":846},"top:-3em;",[31,848],{"className":849,"style":850},[304],"height:3em;",[31,852,15],{"className":853},[74,75],[31,855,856,859],{"style":846},[31,857],{"className":858,"style":850},[304],[31,860,864],{"className":861,"style":863},[862],"accent-body","left:-0.1667em;",[31,865,811],{"className":866},[74],[31,868,871],{"className":869},[870],"vlist-s","​",[31,873,875],{"className":874},[293],[31,876,879],{"className":877,"style":878},[297],"height:0.1944em;",[31,880],{},[31,882],{"className":883,"style":124},[123],[31,885,100],{"className":886},[128],[31,888],{"className":889,"style":124},[123],[31,891,893,896],{"className":892},[65],[31,894],{"className":895,"style":138},[69],[31,897,394],{"className":898},[74],[15,900,901],{},"This is maximum likelihood estimation.",[10,903,905],{"id":904},"likelihood-function","Likelihood Function",[15,907,908],{},"Given data:",[31,910,912],{"className":911,"translate":35},[149],[31,913,915,973],{"className":914,"translate":35},[34],[31,916,918],{"className":917},[39],[41,919,920],{"xmlns":43,"display":158},[45,921,922,970],{},[48,923,924,926,928,931,940,943,949,951,954,958,960,967],{},[51,925,172],{},[98,927,100],{},[98,929,930],{"stretchy":168},"{",[932,933,934,937],"msub",{},[51,935,936],{},"x",[102,938,939],{},"1",[98,941,942],{"separator":61},",",[932,944,945,947],{},[51,946,936],{},[102,948,479],{},[98,950,942],{"separator":61},[98,952,953],{},"⋯",[955,956,957],"mtext",{}," ",[98,959,942],{"separator":61},[932,961,962,964],{},[51,963,936],{},[51,965,966],{},"n",[98,968,969],{"stretchy":168},"}",[54,971,972],{"encoding":56},"D = \\{x_1, x_2, \\cdots, x_n\\}",[31,974,976,995],{"className":975,"ariaHidden":61},[60],[31,977,979,983,986,989,992],{"className":978},[65],[31,980],{"className":981,"style":982},[69],"height:0.6833em;",[31,984,172],{"className":985,"style":226},[74,75],[31,987],{"className":988,"style":124},[123],[31,990,100],{"className":991},[128],[31,993],{"className":994,"style":124},[123],[31,996,998,1001,1004,1047,1051,1055,1095,1098,1101,1105,1108,1111,1114,1117,1158],{"className":997},[65],[31,999],{"className":1000,"style":214},[69],[31,1002,930],{"className":1003},[222],[31,1005,1007,1010],{"className":1006},[74],[31,1008,936],{"className":1009},[74,75],[31,1011,1013],{"className":1012},[285],[31,1014,1016,1038],{"className":1015},[289,836],[31,1017,1019,1035],{"className":1018},[293],[31,1020,1023],{"className":1021,"style":1022},[297],"height:0.3011em;",[31,1024,1026,1029],{"style":1025},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[31,1027],{"className":1028,"style":305},[304],[31,1030,1032],{"className":1031},[309,310,311,312],[31,1033,939],{"className":1034},[74,312],[31,1036,871],{"className":1037},[870],[31,1039,1041],{"className":1040},[293],[31,1042,1045],{"className":1043,"style":1044},[297],"height:0.15em;",[31,1046],{},[31,1048,942],{"className":1049},[1050],"mpunct",[31,1052],{"className":1053,"style":1054},[123],"margin-right:0.1667em;",[31,1056,1058,1061],{"className":1057},[74],[31,1059,936],{"className":1060},[74,75],[31,1062,1064],{"className":1063},[285],[31,1065,1067,1087],{"className":1066},[289,836],[31,1068,1070,1084],{"className":1069},[293],[31,1071,1073],{"className":1072,"style":1022},[297],[31,1074,1075,1078],{"style":1025},[31,1076],{"className":1077,"style":305},[304],[31,1079,1081],{"className":1080},[309,310,311,312],[31,1082,479],{"className":1083},[74,312],[31,1085,871],{"className":1086},[870],[31,1088,1090],{"className":1089},[293],[31,1091,1093],{"className":1092,"style":1044},[297],[31,1094],{},[31,1096,942],{"className":1097},[1050],[31,1099],{"className":1100,"style":1054},[123],[31,1102,953],{"className":1103},[1104],"minner",[31,1106],{"className":1107,"style":1054},[123],[31,1109],{"className":1110,"style":1054},[123],[31,1112,942],{"className":1113},[1050],[31,1115],{"className":1116,"style":1054},[123],[31,1118,1120,1123],{"className":1119},[74],[31,1121,936],{"className":1122},[74,75],[31,1124,1126],{"className":1125},[285],[31,1127,1129,1150],{"className":1128},[289,836],[31,1130,1132,1147],{"className":1131},[293],[31,1133,1136],{"className":1134,"style":1135},[297],"height:0.1514em;",[31,1137,1138,1141],{"style":1025},[31,1139],{"className":1140,"style":305},[304],[31,1142,1144],{"className":1143},[309,310,311,312],[31,1145,966],{"className":1146},[74,75,312],[31,1148,871],{"className":1149},[870],[31,1151,1153],{"className":1152},[293],[31,1154,1156],{"className":1155,"style":1044},[297],[31,1157],{},[31,1159,969],{"className":1160},[254],[15,1162,1163,1164,142],{},"Suppose we have model parameters: ",[31,1165,1167,1182],{"className":1166,"translate":35},[34],[31,1168,1170],{"className":1169},[39],[41,1171,1172],{"xmlns":43},[45,1173,1174,1179],{},[48,1175,1176],{},[51,1177,1178],{},"θ",[54,1180,1181],{"encoding":56},"\\theta",[31,1183,1185],{"className":1184,"ariaHidden":61},[60],[31,1186,1188,1191],{"className":1187},[65],[31,1189],{"className":1190,"style":843},[69],[31,1192,1178],{"className":1193,"style":226},[74,75],[15,1195,1196,1197,1200],{},"The ",[370,1198,1199],{},"likelihood function"," is defined as:",[31,1202,1204],{"className":1203,"translate":35},[149],[31,1205,1207,1242],{"className":1206,"translate":35},[34],[31,1208,1210],{"className":1209},[39],[41,1211,1212],{"xmlns":43,"display":158},[45,1213,1214,1239],{},[48,1215,1216,1219,1221,1223,1225,1227,1229,1231,1233,1235,1237],{},[51,1217,1218],{},"L",[98,1220,169],{"stretchy":168},[51,1222,1178],{},[98,1224,185],{"stretchy":168},[98,1226,100],{},[51,1228,165],{},[98,1230,169],{"stretchy":168},[51,1232,172],{},[51,1234,176],{"mathvariant":175},[51,1236,1178],{},[98,1238,185],{"stretchy":168},[54,1240,1241],{"encoding":56},"L(\\theta) = P(D|\\theta)",[31,1243,1245,1272],{"className":1244,"ariaHidden":61},[60],[31,1246,1248,1251,1254,1257,1260,1263,1266,1269],{"className":1247},[65],[31,1249],{"className":1250,"style":214},[69],[31,1252,1218],{"className":1253},[74,75],[31,1255,169],{"className":1256},[222],[31,1258,1178],{"className":1259,"style":226},[74,75],[31,1261,185],{"className":1262},[254],[31,1264],{"className":1265,"style":124},[123],[31,1267,100],{"className":1268},[128],[31,1270],{"className":1271,"style":124},[123],[31,1273,1275,1278,1281,1284,1287,1290,1293],{"className":1274},[65],[31,1276],{"className":1277,"style":214},[69],[31,1279,165],{"className":1280,"style":218},[74,75],[31,1282,169],{"className":1283},[222],[31,1285,172],{"className":1286,"style":226},[74,75],[31,1288,176],{"className":1289},[74],[31,1291,1178],{"className":1292,"style":226},[74,75],[31,1294,185],{"className":1295},[254],[15,1297,1298,1301],{},[370,1299,1300],{},"Maximum Likelihood Estimation (MLE)"," is:",[31,1303,1305],{"className":1304,"translate":35},[149],[31,1306,1308,1382],{"className":1307,"translate":35},[34],[31,1309,1311],{"className":1310},[39],[41,1312,1313],{"xmlns":43,"display":158},[45,1314,1315,1379],{},[48,1316,1317,1323,1325,1328,1331,1343,1345,1347,1349,1351,1353,1355,1357,1359,1361,1371,1373,1375,1377],{},[805,1318,1319,1321],{"accent":61},[51,1320,1178],{},[98,1322,811],{},[98,1324,100],{},[51,1326,1327],{},"arg",[98,1329,1330],{},"⁡",[1332,1333,1334,1341],"munder",{},[48,1335,1336,1339],{},[51,1337,1338],{},"max",[98,1340,1330],{},[51,1342,1178],{},[51,1344,165],{},[98,1346,169],{"stretchy":168},[51,1348,172],{},[51,1350,176],{"mathvariant":175},[51,1352,1178],{},[98,1354,185],{"stretchy":168},[98,1356,100],{},[51,1358,1327],{},[98,1360,1330],{},[1332,1362,1363,1369],{},[48,1364,1365,1367],{},[51,1366,1338],{},[98,1368,1330],{},[51,1370,1178],{},[51,1372,1218],{},[98,1374,169],{"stretchy":168},[51,1376,1178],{},[98,1378,185],{"stretchy":168},[54,1380,1381],{"encoding":56},"\\hat\\theta = \\arg\\max_\\theta P(D|\\theta) = \\arg\\max_\\theta L(\\theta)",[31,1383,1385,1433,1530],{"className":1384,"ariaHidden":61},[60],[31,1386,1388,1392,1424,1427,1430],{"className":1387},[65],[31,1389],{"className":1390,"style":1391},[69],"height:0.9579em;",[31,1393,1395],{"className":1394},[74,832],[31,1396,1398],{"className":1397},[289],[31,1399,1401],{"className":1400},[293],[31,1402,1404,1412],{"className":1403,"style":1391},[297],[31,1405,1406,1409],{"style":846},[31,1407],{"className":1408,"style":850},[304],[31,1410,1178],{"className":1411,"style":226},[74,75],[31,1413,1415,1418],{"style":1414},"top:-3.2634em;",[31,1416],{"className":1417,"style":850},[304],[31,1419,1421],{"className":1420,"style":863},[862],[31,1422,811],{"className":1423},[74],[31,1425],{"className":1426,"style":124},[123],[31,1428,100],{"className":1429},[128],[31,1431],{"className":1432,"style":124},[123],[31,1434,1436,1440,1449,1452,1500,1503,1506,1509,1512,1515,1518,1521,1524,1527],{"className":1435},[65],[31,1437],{"className":1438,"style":1439},[69],"height:1.5021em;vertical-align:-0.7521em;",[31,1441,1444,1445],{"className":1442},[1443],"mop","ar",[31,1446,1448],{"style":1447},"margin-right:0.0139em;","g",[31,1450],{"className":1451,"style":1054},[123],[31,1453,1456],{"className":1454},[1443,1455],"op-limits",[31,1457,1459,1491],{"className":1458},[289,836],[31,1460,1462,1488],{"className":1461},[293],[31,1463,1466,1478],{"className":1464,"style":1465},[297],"height:0.4306em;",[31,1467,1469,1472],{"style":1468},"top:-2.3479em;margin-left:0em;",[31,1470],{"className":1471,"style":850},[304],[31,1473,1475],{"className":1474},[309,310,311,312],[31,1476,1178],{"className":1477,"style":226},[74,75,312],[31,1479,1480,1483],{"style":846},[31,1481],{"className":1482,"style":850},[304],[31,1484,1485],{},[31,1486,1338],{"className":1487},[1443],[31,1489,871],{"className":1490},[870],[31,1492,1494],{"className":1493},[293],[31,1495,1498],{"className":1496,"style":1497},[297],"height:0.7521em;",[31,1499],{},[31,1501],{"className":1502,"style":1054},[123],[31,1504,165],{"className":1505,"style":218},[74,75],[31,1507,169],{"className":1508},[222],[31,1510,172],{"className":1511,"style":226},[74,75],[31,1513,176],{"className":1514},[74],[31,1516,1178],{"className":1517,"style":226},[74,75],[31,1519,185],{"className":1520},[254],[31,1522],{"className":1523,"style":124},[123],[31,1525,100],{"className":1526},[128],[31,1528],{"className":1529,"style":124},[123],[31,1531,1533,1536,1541,1544,1588,1591,1594,1597,1600],{"className":1532},[65],[31,1534],{"className":1535,"style":1439},[69],[31,1537,1444,1539],{"className":1538},[1443],[31,1540,1448],{"style":1447},[31,1542],{"className":1543,"style":1054},[123],[31,1545,1547],{"className":1546},[1443,1455],[31,1548,1550,1580],{"className":1549},[289,836],[31,1551,1553,1577],{"className":1552},[293],[31,1554,1556,1567],{"className":1555,"style":1465},[297],[31,1557,1558,1561],{"style":1468},[31,1559],{"className":1560,"style":850},[304],[31,1562,1564],{"className":1563},[309,310,311,312],[31,1565,1178],{"className":1566,"style":226},[74,75,312],[31,1568,1569,1572],{"style":846},[31,1570],{"className":1571,"style":850},[304],[31,1573,1574],{},[31,1575,1338],{"className":1576},[1443],[31,1578,871],{"className":1579},[870],[31,1581,1583],{"className":1582},[293],[31,1584,1586],{"className":1585,"style":1497},[297],[31,1587],{},[31,1589],{"className":1590,"style":1054},[123],[31,1592,1218],{"className":1593},[74,75],[31,1595,169],{"className":1596},[222],[31,1598,1178],{"className":1599,"style":226},[74,75],[31,1601,185],{"className":1602},[254],[15,1604,1605,1606,1609,1610,1664,1665,1709,1710,1713],{},"Many people feel confused when first learning this:",[1607,1608],"br",{},"\nSince ",[31,1611,1613,1637],{"className":1612,"translate":35},[34],[31,1614,1616],{"className":1615},[39],[41,1617,1618],{"xmlns":43},[45,1619,1620,1634],{},[48,1621,1622,1624,1626,1628,1630,1632],{},[51,1623,165],{},[98,1625,169],{"stretchy":168},[51,1627,172],{},[51,1629,176],{"mathvariant":175},[51,1631,1178],{},[98,1633,185],{"stretchy":168},[54,1635,1636],{"encoding":56},"P(D|\\theta)",[31,1638,1640],{"className":1639,"ariaHidden":61},[60],[31,1641,1643,1646,1649,1652,1655,1658,1661],{"className":1642},[65],[31,1644],{"className":1645,"style":214},[69],[31,1647,165],{"className":1648,"style":218},[74,75],[31,1650,169],{"className":1651},[222],[31,1653,172],{"className":1654,"style":226},[74,75],[31,1656,176],{"className":1657},[74],[31,1659,1178],{"className":1660,"style":226},[74,75],[31,1662,185],{"className":1663},[254]," and ",[31,1666,1668,1688],{"className":1667,"translate":35},[34],[31,1669,1671],{"className":1670},[39],[41,1672,1673],{"xmlns":43},[45,1674,1675,1685],{},[48,1676,1677,1679,1681,1683],{},[51,1678,1218],{},[98,1680,169],{"stretchy":168},[51,1682,1178],{},[98,1684,185],{"stretchy":168},[54,1686,1687],{"encoding":56},"L(\\theta)",[31,1689,1691],{"className":1690,"ariaHidden":61},[60],[31,1692,1694,1697,1700,1703,1706],{"className":1693},[65],[31,1695],{"className":1696,"style":214},[69],[31,1698,1218],{"className":1699},[74,75],[31,1701,169],{"className":1702},[222],[31,1704,1178],{"className":1705,"style":226},[74,75],[31,1707,185],{"className":1708},[254]," have ",[370,1711,1712],{},"exactly the same expression",", why change the name?",[15,1715,1716,1717,1720],{},"The key lies in ",[370,1718,1719],{},"different perspectives",":",[1722,1723,1724,1856],"ul",{},[1725,1726,1727,1730,1731,1784,1785,1787,1788,1816,1817,1820,1821,1849,1850,142,1853,1855],"li",{},[370,1728,1729],{},"Probability"," ",[31,1732,1734,1757],{"className":1733,"translate":35},[34],[31,1735,1737],{"className":1736},[39],[41,1738,1739],{"xmlns":43},[45,1740,1741,1755],{},[48,1742,1743,1745,1747,1749,1751,1753],{},[51,1744,165],{},[98,1746,169],{"stretchy":168},[51,1748,172],{},[51,1750,176],{"mathvariant":175},[51,1752,1178],{},[98,1754,185],{"stretchy":168},[54,1756,1636],{"encoding":56},[31,1758,1760],{"className":1759,"ariaHidden":61},[60],[31,1761,1763,1766,1769,1772,1775,1778,1781],{"className":1762},[65],[31,1764],{"className":1765,"style":214},[69],[31,1767,165],{"className":1768,"style":218},[74,75],[31,1770,169],{"className":1771},[222],[31,1773,172],{"className":1774,"style":226},[74,75],[31,1776,176],{"className":1777},[74],[31,1779,1178],{"className":1780,"style":226},[74,75],[31,1782,185],{"className":1783},[254]," perspective:",[1607,1786],{},"\nThe parameter ",[31,1789,1791,1804],{"className":1790,"translate":35},[34],[31,1792,1794],{"className":1793},[39],[41,1795,1796],{"xmlns":43},[45,1797,1798,1802],{},[48,1799,1800],{},[51,1801,1178],{},[54,1803,1181],{"encoding":56},[31,1805,1807],{"className":1806,"ariaHidden":61},[60],[31,1808,1810,1813],{"className":1809},[65],[31,1811],{"className":1812,"style":843},[69],[31,1814,1178],{"className":1815,"style":226},[74,75]," is a ",[370,1818,1819],{},"fixed known value",", while the data ",[31,1822,1824,1837],{"className":1823,"translate":35},[34],[31,1825,1827],{"className":1826},[39],[41,1828,1829],{"xmlns":43},[45,1830,1831,1835],{},[48,1832,1833],{},[51,1834,172],{},[54,1836,172],{"encoding":56},[31,1838,1840],{"className":1839,"ariaHidden":61},[60],[31,1841,1843,1846],{"className":1842},[65],[31,1844],{"className":1845,"style":982},[69],[31,1847,172],{"className":1848,"style":226},[74,75]," is ",[370,1851,1852],{},"random",[1607,1854],{},"\nWe ask: \"Given the parameter, how likely are different data outcomes?\"",[1725,1857,1858,1730,1861,1784,1904,1906,1907,1816,1935,1938,1939,1816,1967,142,1970,1972],{},[370,1859,1860],{},"Likelihood",[31,1862,1864,1883],{"className":1863,"translate":35},[34],[31,1865,1867],{"className":1866},[39],[41,1868,1869],{"xmlns":43},[45,1870,1871,1881],{},[48,1872,1873,1875,1877,1879],{},[51,1874,1218],{},[98,1876,169],{"stretchy":168},[51,1878,1178],{},[98,1880,185],{"stretchy":168},[54,1882,1687],{"encoding":56},[31,1884,1886],{"className":1885,"ariaHidden":61},[60],[31,1887,1889,1892,1895,1898,1901],{"className":1888},[65],[31,1890],{"className":1891,"style":214},[69],[31,1893,1218],{"className":1894},[74,75],[31,1896,169],{"className":1897},[222],[31,1899,1178],{"className":1900,"style":226},[74,75],[31,1902,185],{"className":1903},[254],[1607,1905],{},"\nThe data ",[31,1908,1910,1923],{"className":1909,"translate":35},[34],[31,1911,1913],{"className":1912},[39],[41,1914,1915],{"xmlns":43},[45,1916,1917,1921],{},[48,1918,1919],{},[51,1920,172],{},[54,1922,172],{"encoding":56},[31,1924,1926],{"className":1925,"ariaHidden":61},[60],[31,1927,1929,1932],{"className":1928},[65],[31,1930],{"className":1931,"style":982},[69],[31,1933,172],{"className":1934,"style":226},[74,75],[370,1936,1937],{},"fixed observed value",", while the parameter ",[31,1940,1942,1955],{"className":1941,"translate":35},[34],[31,1943,1945],{"className":1944},[39],[41,1946,1947],{"xmlns":43},[45,1948,1949,1953],{},[48,1950,1951],{},[51,1952,1178],{},[54,1954,1181],{"encoding":56},[31,1956,1958],{"className":1957,"ariaHidden":61},[60],[31,1959,1961,1964],{"className":1960},[65],[31,1962],{"className":1963,"style":843},[69],[31,1965,1178],{"className":1966,"style":226},[74,75],[370,1968,1969],{},"varying unknown quantity",[1607,1971],{},"\nWe ask: \"Under different parameter values, how likely is this observed data?\"",[15,1974,1975],{},"According to the conditional formula:",[31,1977,1979],{"className":1978,"translate":35},[149],[31,1980,1982,2043],{"className":1981,"translate":35},[34],[31,1983,1985],{"className":1984},[39],[41,1986,1987],{"xmlns":43,"display":158},[45,1988,1989,2040],{},[48,1990,1991,1993,1995,1997,1999,2001,2003,2005,2024,2026,2028,2034,2036,2038],{},[51,1992,165],{},[98,1994,169],{"stretchy":168},[51,1996,172],{},[51,1998,176],{"mathvariant":175},[51,2000,1178],{},[98,2002,185],{"stretchy":168},[98,2004,100],{},[1332,2006,2007,2010],{},[98,2008,2009],{},"∏",[48,2011,2012,2019,2022],{},[932,2013,2014,2016],{},[51,2015,936],{},[51,2017,2018],{},"i",[98,2020,2021],{},"∈",[51,2023,172],{},[51,2025,165],{},[98,2027,169],{"stretchy":168},[932,2029,2030,2032],{},[51,2031,936],{},[51,2033,2018],{},[51,2035,176],{"mathvariant":175},[51,2037,1178],{},[98,2039,185],{"stretchy":168},[54,2041,2042],{"encoding":56},"P(D|\\theta)\n=\n\\prod_{x_i \\in D}\nP(x_i|\\theta)",[31,2044,2046,2079],{"className":2045,"ariaHidden":61},[60],[31,2047,2049,2052,2055,2058,2061,2064,2067,2070,2073,2076],{"className":2048},[65],[31,2050],{"className":2051,"style":214},[69],[31,2053,165],{"className":2054,"style":218},[74,75],[31,2056,169],{"className":2057},[222],[31,2059,172],{"className":2060,"style":226},[74,75],[31,2062,176],{"className":2063},[74],[31,2065,1178],{"className":2066,"style":226},[74,75],[31,2068,185],{"className":2069},[254],[31,2071],{"className":2072,"style":124},[123],[31,2074,100],{"className":2075},[128],[31,2077],{"className":2078,"style":124},[123],[31,2080,2082,2086,2189,2192,2195,2198,2239,2242,2245],{"className":2081},[65],[31,2083],{"className":2084,"style":2085},[69],"height:2.4444em;vertical-align:-1.3944em;",[31,2087,2089],{"className":2088},[1443,1455],[31,2090,2092,2180],{"className":2091},[289,836],[31,2093,2095,2177],{"className":2094},[293],[31,2096,2099,2164],{"className":2097,"style":2098},[297],"height:1.05em;",[31,2100,2102,2106],{"style":2101},"top:-1.8557em;margin-left:0em;",[31,2103],{"className":2104,"style":2105},[304],"height:3.05em;",[31,2107,2109],{"className":2108},[309,310,311,312],[31,2110,2112,2158,2161],{"className":2111},[74,312],[31,2113,2115,2118],{"className":2114},[74,312],[31,2116,936],{"className":2117},[74,75,312],[31,2119,2121],{"className":2120},[285],[31,2122,2124,2149],{"className":2123},[289,836],[31,2125,2127,2146],{"className":2126},[293],[31,2128,2131],{"className":2129,"style":2130},[297],"height:0.3281em;",[31,2132,2134,2138],{"style":2133},"top:-2.357em;margin-left:0em;margin-right:0.0714em;",[31,2135],{"className":2136,"style":2137},[304],"height:2.5em;",[31,2139,2143],{"className":2140},[309,2141,2142,312],"reset-size3","size1",[31,2144,2018],{"className":2145},[74,75,312],[31,2147,871],{"className":2148},[870],[31,2150,2152],{"className":2151},[293],[31,2153,2156],{"className":2154,"style":2155},[297],"height:0.143em;",[31,2157],{},[31,2159,2021],{"className":2160},[128,312],[31,2162,172],{"className":2163,"style":226},[74,75,312],[31,2165,2167,2170],{"style":2166},"top:-3.05em;",[31,2168],{"className":2169,"style":2105},[304],[31,2171,2172],{},[31,2173,2009],{"className":2174},[1443,2175,2176],"op-symbol","large-op",[31,2178,871],{"className":2179},[870],[31,2181,2183],{"className":2182},[293],[31,2184,2187],{"className":2185,"style":2186},[297],"height:1.3944em;",[31,2188],{},[31,2190],{"className":2191,"style":1054},[123],[31,2193,165],{"className":2194,"style":218},[74,75],[31,2196,169],{"className":2197},[222],[31,2199,2201,2204],{"className":2200},[74],[31,2202,936],{"className":2203},[74,75],[31,2205,2207],{"className":2206},[285],[31,2208,2210,2231],{"className":2209},[289,836],[31,2211,2213,2228],{"className":2212},[293],[31,2214,2217],{"className":2215,"style":2216},[297],"height:0.3117em;",[31,2218,2219,2222],{"style":1025},[31,2220],{"className":2221,"style":305},[304],[31,2223,2225],{"className":2224},[309,310,311,312],[31,2226,2018],{"className":2227},[74,75,312],[31,2229,871],{"className":2230},[870],[31,2232,2234],{"className":2233},[293],[31,2235,2237],{"className":2236,"style":1044},[297],[31,2238],{},[31,2240,176],{"className":2241},[74],[31,2243,1178],{"className":2244,"style":226},[74,75],[31,2246,185],{"className":2247},[254],[15,2249,2250,2251,2279],{},"If the data ",[31,2252,2254,2267],{"className":2253,"translate":35},[34],[31,2255,2257],{"className":2256},[39],[41,2258,2259],{"xmlns":43},[45,2260,2261,2265],{},[48,2262,2263],{},[51,2264,172],{},[54,2266,172],{"encoding":56},[31,2268,2270],{"className":2269,"ariaHidden":61},[60],[31,2271,2273,2276],{"className":2272},[65],[31,2274],{"className":2275,"style":982},[69],[31,2277,172],{"className":2278,"style":226},[74,75]," contains many samples, multiplying many small numbers yields a result approaching zero, which can cause numerical underflow in computers.",[15,2281,2282],{},"Thus we take the logarithm, converting multiplication into addition:",[31,2284,2286],{"className":2285,"translate":35},[149],[31,2287,2289,2372],{"className":2288,"translate":35},[34],[31,2290,2292],{"className":2291},[39],[41,2293,2294],{"xmlns":43,"display":158},[45,2295,2296,2369],{},[48,2297,2298,2301,2303,2305,2307,2309,2311,2313,2316,2318,2320,2322,2324,2326,2328,2330,2332,2349,2351,2353,2355,2357,2363,2365,2367],{},[51,2299,2300],{"mathvariant":175},"ℓ",[98,2302,169],{"stretchy":168},[51,2304,1178],{},[51,2306,176],{"mathvariant":175},[51,2308,172],{},[98,2310,185],{"stretchy":168},[98,2312,100],{},[51,2314,2315],{},"log",[98,2317,1330],{},[51,2319,165],{},[98,2321,169],{"stretchy":168},[51,2323,172],{},[51,2325,176],{"mathvariant":175},[51,2327,1178],{},[98,2329,185],{"stretchy":168},[98,2331,100],{},[1332,2333,2334,2337],{},[98,2335,2336],{},"∑",[48,2338,2339,2345,2347],{},[932,2340,2341,2343],{},[51,2342,936],{},[51,2344,2018],{},[98,2346,2021],{},[51,2348,172],{},[51,2350,2315],{},[98,2352,1330],{},[51,2354,165],{},[98,2356,169],{"stretchy":168},[932,2358,2359,2361],{},[51,2360,936],{},[51,2362,2018],{},[51,2364,176],{"mathvariant":175},[51,2366,1178],{},[98,2368,185],{"stretchy":168},[54,2370,2371],{"encoding":56},"\\ell(\\theta|D)=\n\\log P(D|\\theta)\n=\n\\sum_{x_i \\in D}\n\\log P(x_i|\\theta)",[31,2373,2375,2408,2450],{"className":2374,"ariaHidden":61},[60],[31,2376,2378,2381,2384,2387,2390,2393,2396,2399,2402,2405],{"className":2377},[65],[31,2379],{"className":2380,"style":214},[69],[31,2382,2300],{"className":2383},[74],[31,2385,169],{"className":2386},[222],[31,2388,1178],{"className":2389,"style":226},[74,75],[31,2391,176],{"className":2392},[74],[31,2394,172],{"className":2395,"style":226},[74,75],[31,2397,185],{"className":2398},[254],[31,2400],{"className":2401,"style":124},[123],[31,2403,100],{"className":2404},[128],[31,2406],{"className":2407,"style":124},[123],[31,2409,2411,2414,2420,2423,2426,2429,2432,2435,2438,2441,2444,2447],{"className":2410},[65],[31,2412],{"className":2413,"style":214},[69],[31,2415,2417,2418],{"className":2416},[1443],"lo",[31,2419,1448],{"style":1447},[31,2421],{"className":2422,"style":1054},[123],[31,2424,165],{"className":2425,"style":218},[74,75],[31,2427,169],{"className":2428},[222],[31,2430,172],{"className":2431,"style":226},[74,75],[31,2433,176],{"className":2434},[74],[31,2436,1178],{"className":2437,"style":226},[74,75],[31,2439,185],{"className":2440},[254],[31,2442],{"className":2443,"style":124},[123],[31,2445,100],{"className":2446},[128],[31,2448],{"className":2449,"style":124},[123],[31,2451,2453,2456,2546,2549,2554,2557,2560,2563,2603,2606,2609],{"className":2452},[65],[31,2454],{"className":2455,"style":2085},[69],[31,2457,2459],{"className":2458},[1443,1455],[31,2460,2462,2538],{"className":2461},[289,836],[31,2463,2465,2535],{"className":2464},[293],[31,2466,2468,2525],{"className":2467,"style":2098},[297],[31,2469,2470,2473],{"style":2101},[31,2471],{"className":2472,"style":2105},[304],[31,2474,2476],{"className":2475},[309,310,311,312],[31,2477,2479,2519,2522],{"className":2478},[74,312],[31,2480,2482,2485],{"className":2481},[74,312],[31,2483,936],{"className":2484},[74,75,312],[31,2486,2488],{"className":2487},[285],[31,2489,2491,2511],{"className":2490},[289,836],[31,2492,2494,2508],{"className":2493},[293],[31,2495,2497],{"className":2496,"style":2130},[297],[31,2498,2499,2502],{"style":2133},[31,2500],{"className":2501,"style":2137},[304],[31,2503,2505],{"className":2504},[309,2141,2142,312],[31,2506,2018],{"className":2507},[74,75,312],[31,2509,871],{"className":2510},[870],[31,2512,2514],{"className":2513},[293],[31,2515,2517],{"className":2516,"style":2155},[297],[31,2518],{},[31,2520,2021],{"className":2521},[128,312],[31,2523,172],{"className":2524,"style":226},[74,75,312],[31,2526,2527,2530],{"style":2166},[31,2528],{"className":2529,"style":2105},[304],[31,2531,2532],{},[31,2533,2336],{"className":2534},[1443,2175,2176],[31,2536,871],{"className":2537},[870],[31,2539,2541],{"className":2540},[293],[31,2542,2544],{"className":2543,"style":2186},[297],[31,2545],{},[31,2547],{"className":2548,"style":1054},[123],[31,2550,2417,2552],{"className":2551},[1443],[31,2553,1448],{"style":1447},[31,2555],{"className":2556,"style":1054},[123],[31,2558,165],{"className":2559,"style":218},[74,75],[31,2561,169],{"className":2562},[222],[31,2564,2566,2569],{"className":2565},[74],[31,2567,936],{"className":2568},[74,75],[31,2570,2572],{"className":2571},[285],[31,2573,2575,2595],{"className":2574},[289,836],[31,2576,2578,2592],{"className":2577},[293],[31,2579,2581],{"className":2580,"style":2216},[297],[31,2582,2583,2586],{"style":1025},[31,2584],{"className":2585,"style":305},[304],[31,2587,2589],{"className":2588},[309,310,311,312],[31,2590,2018],{"className":2591},[74,75,312],[31,2593,871],{"className":2594},[870],[31,2596,2598],{"className":2597},[293],[31,2599,2601],{"className":2600,"style":1044},[297],[31,2602],{},[31,2604,176],{"className":2605},[74],[31,2607,1178],{"className":2608,"style":226},[74,75],[31,2610,185],{"className":2611},[254],[15,2613,2614],{},"After taking the logarithm, the values stay within an acceptable range, making optimization easier.",[15,2616,2617],{},"Therefore, almost all practical training optimizes:",[31,2619,2621],{"className":2620,"translate":35},[149],[31,2622,2624,2658],{"className":2623,"translate":35},[34],[31,2625,2627],{"className":2626},[39],[41,2628,2629],{"xmlns":43,"display":158},[45,2630,2631,2655],{},[48,2632,2633,2643,2645,2647,2649,2651,2653],{},[1332,2634,2635,2641],{},[48,2636,2637,2639],{},[51,2638,1338],{},[98,2640,1330],{},[51,2642,1178],{},[51,2644,2300],{"mathvariant":175},[98,2646,169],{"stretchy":168},[51,2648,1178],{},[51,2650,176],{"mathvariant":175},[51,2652,172],{},[98,2654,185],{"stretchy":168},[54,2656,2657],{"encoding":56},"\\max_\\theta\n\\ell(\\theta|D)",[31,2659,2661],{"className":2660,"ariaHidden":61},[60],[31,2662,2664,2667,2711,2714,2717,2720,2723,2726,2729],{"className":2663},[65],[31,2665],{"className":2666,"style":1439},[69],[31,2668,2670],{"className":2669},[1443,1455],[31,2671,2673,2703],{"className":2672},[289,836],[31,2674,2676,2700],{"className":2675},[293],[31,2677,2679,2690],{"className":2678,"style":1465},[297],[31,2680,2681,2684],{"style":1468},[31,2682],{"className":2683,"style":850},[304],[31,2685,2687],{"className":2686},[309,310,311,312],[31,2688,1178],{"className":2689,"style":226},[74,75,312],[31,2691,2692,2695],{"style":846},[31,2693],{"className":2694,"style":850},[304],[31,2696,2697],{},[31,2698,1338],{"className":2699},[1443],[31,2701,871],{"className":2702},[870],[31,2704,2706],{"className":2705},[293],[31,2707,2709],{"className":2708,"style":1497},[297],[31,2710],{},[31,2712],{"className":2713,"style":1054},[123],[31,2715,2300],{"className":2716},[74],[31,2718,169],{"className":2719},[222],[31,2721,1178],{"className":2722,"style":226},[74,75],[31,2724,176],{"className":2725},[74],[31,2727,172],{"className":2728,"style":226},[74,75],[31,2730,185],{"className":2731},[254],[15,2733,2734,2735,142],{},"This form is called ",[370,2736,2737],{},"Log-Likelihood",[15,2739,2740,2741,2851],{},"In the likelihood function, we need to find the parameter ",[31,2742,2744,2767],{"className":2743,"translate":35},[34],[31,2745,2747],{"className":2746},[39],[41,2748,2749],{"xmlns":43},[45,2750,2751,2764],{},[48,2752,2753],{},[932,2754,2755,2761],{},[805,2756,2757,2759],{"accent":61},[51,2758,1178],{},[98,2760,811],{},[955,2762,2763],{},"MLE",[54,2765,2766],{"encoding":56},"\\hat{\\theta}_\\text{MLE}",[31,2768,2770],{"className":2769,"ariaHidden":61},[60],[31,2771,2773,2777],{"className":2772},[65],[31,2774],{"className":2775,"style":2776},[69],"height:1.1079em;vertical-align:-0.15em;",[31,2778,2780,2811],{"className":2779},[74],[31,2781,2783],{"className":2782},[74,832],[31,2784,2786],{"className":2785},[289],[31,2787,2789],{"className":2788},[293],[31,2790,2792,2800],{"className":2791,"style":1391},[297],[31,2793,2794,2797],{"style":846},[31,2795],{"className":2796,"style":850},[304],[31,2798,1178],{"className":2799,"style":226},[74,75],[31,2801,2802,2805],{"style":1414},[31,2803],{"className":2804,"style":850},[304],[31,2806,2808],{"className":2807,"style":863},[862],[31,2809,811],{"className":2810},[74],[31,2812,2814],{"className":2813},[285],[31,2815,2817,2843],{"className":2816},[289,836],[31,2818,2820,2840],{"className":2819},[293],[31,2821,2824],{"className":2822,"style":2823},[297],"height:0.3283em;",[31,2825,2827,2830],{"style":2826},"top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;",[31,2828],{"className":2829,"style":305},[304],[31,2831,2833],{"className":2832},[309,310,311,312],[31,2834,2837],{"className":2835},[74,2836,312],"text",[31,2838,2763],{"className":2839},[74,312],[31,2841,871],{"className":2842},[870],[31,2844,2846],{"className":2845},[293],[31,2847,2849],{"className":2848,"style":1044},[297],[31,2850],{}," that maximizes the likelihood.",[10,2853,2855],{"id":2854},"solving-for-parameters","Solving for Parameters",[15,2857,2858,2859,2913,2914,3031],{},"If ",[31,2860,2862,2886],{"className":2861,"translate":35},[34],[31,2863,2865],{"className":2864},[39],[41,2866,2867],{"xmlns":43},[45,2868,2869,2883],{},[48,2870,2871,2873,2875,2877,2879,2881],{},[51,2872,2300],{"mathvariant":175},[98,2874,169],{"stretchy":168},[51,2876,1178],{},[51,2878,176],{"mathvariant":175},[51,2880,172],{},[98,2882,185],{"stretchy":168},[54,2884,2885],{"encoding":56},"\\ell(\\theta|D)",[31,2887,2889],{"className":2888,"ariaHidden":61},[60],[31,2890,2892,2895,2898,2901,2904,2907,2910],{"className":2891},[65],[31,2893],{"className":2894,"style":214},[69],[31,2896,2300],{"className":2897},[74],[31,2899,169],{"className":2900},[222],[31,2902,1178],{"className":2903,"style":226},[74,75],[31,2905,176],{"className":2906},[74],[31,2908,172],{"className":2909,"style":226},[74,75],[31,2911,185],{"className":2912},[254]," is differentiable, ",[31,2915,2917,2947],{"className":2916,"translate":35},[34],[31,2918,2920],{"className":2919},[39],[41,2921,2922],{"xmlns":43},[45,2923,2924,2944],{},[48,2925,2926],{},[932,2927,2928,2934],{},[805,2929,2930,2932],{"accent":61},[51,2931,1178],{},[98,2933,811],{},[48,2935,2936,2939,2941],{},[51,2937,2938],{"mathvariant":175},"M",[51,2940,1218],{"mathvariant":175},[51,2942,2943],{"mathvariant":175},"E",[54,2945,2946],{"encoding":56},"\\hat{\\theta}_{\\rm MLE}",[31,2948,2950],{"className":2949,"ariaHidden":61},[60],[31,2951,2953,2956],{"className":2952},[65],[31,2954],{"className":2955,"style":2776},[69],[31,2957,2959,2990],{"className":2958},[74],[31,2960,2962],{"className":2961},[74,832],[31,2963,2965],{"className":2964},[289],[31,2966,2968],{"className":2967},[293],[31,2969,2971,2979],{"className":2970,"style":1391},[297],[31,2972,2973,2976],{"style":846},[31,2974],{"className":2975,"style":850},[304],[31,2977,1178],{"className":2978,"style":226},[74,75],[31,2980,2981,2984],{"style":1414},[31,2982],{"className":2983,"style":850},[304],[31,2985,2987],{"className":2986,"style":863},[862],[31,2988,811],{"className":2989},[74],[31,2991,2993],{"className":2992},[285],[31,2994,2996,3023],{"className":2995},[289,836],[31,2997,2999,3020],{"className":2998},[293],[31,3000,3002],{"className":3001,"style":2823},[297],[31,3003,3004,3007],{"style":2826},[31,3005],{"className":3006,"style":305},[304],[31,3008,3010],{"className":3009},[309,310,311,312],[31,3011,3013],{"className":3012},[74,312],[31,3014,3016],{"className":3015},[74,312],[31,3017,2763],{"className":3018},[74,3019,312],"mathrm",[31,3021,871],{"className":3022},[870],[31,3024,3026],{"className":3025},[293],[31,3027,3029],{"className":3028,"style":1044},[297],[31,3030],{}," typically satisfies:",[31,3033,3035],{"className":3034,"translate":35},[149],[31,3036,3038,3081],{"className":3037,"translate":35},[34],[31,3039,3041],{"className":3040},[39],[41,3042,3043],{"xmlns":43,"display":158},[45,3044,3045,3078],{},[48,3046,3047,3073,3075],{},[3048,3049,3050,3067],"mfrac",{},[48,3051,3052,3055,3057,3059,3061,3063,3065],{},[51,3053,3054],{"mathvariant":175},"∂",[51,3056,2300],{"mathvariant":175},[98,3058,169],{"stretchy":168},[51,3060,1178],{},[51,3062,176],{"mathvariant":175},[51,3064,172],{},[98,3066,185],{"stretchy":168},[48,3068,3069,3071],{},[51,3070,3054],{"mathvariant":175},[51,3072,1178],{},[98,3074,100],{},[102,3076,3077],{},"0",[54,3079,3080],{"encoding":56},"\\frac{\\partial \\ell(\\theta|D)}{\\partial \\theta} = 0",[31,3082,3084,3192],{"className":3083,"ariaHidden":61},[60],[31,3085,3087,3091,3183,3186,3189],{"className":3086},[65],[31,3088],{"className":3089,"style":3090},[69],"height:2.113em;vertical-align:-0.686em;",[31,3092,3094,3098,3180],{"className":3093},[74],[31,3095],{"className":3096},[222,3097],"nulldelimiter",[31,3099,3101],{"className":3100},[3048],[31,3102,3104,3171],{"className":3103},[289,836],[31,3105,3107,3168],{"className":3106},[293],[31,3108,3111,3127,3138],{"className":3109,"style":3110},[297],"height:1.427em;",[31,3112,3114,3117],{"style":3113},"top:-2.314em;",[31,3115],{"className":3116,"style":850},[304],[31,3118,3120,3124],{"className":3119},[74],[31,3121,3054],{"className":3122,"style":3123},[74],"margin-right:0.0556em;",[31,3125,1178],{"className":3126,"style":226},[74,75],[31,3128,3130,3133],{"style":3129},"top:-3.23em;",[31,3131],{"className":3132,"style":850},[304],[31,3134],{"className":3135,"style":3137},[3136],"frac-line","border-bottom-width:0.04em;",[31,3139,3141,3144],{"style":3140},"top:-3.677em;",[31,3142],{"className":3143,"style":850},[304],[31,3145,3147,3150,3153,3156,3159,3162,3165],{"className":3146},[74],[31,3148,3054],{"className":3149,"style":3123},[74],[31,3151,2300],{"className":3152},[74],[31,3154,169],{"className":3155},[222],[31,3157,1178],{"className":3158,"style":226},[74,75],[31,3160,176],{"className":3161},[74],[31,3163,172],{"className":3164,"style":226},[74,75],[31,3166,185],{"className":3167},[254],[31,3169,871],{"className":3170},[870],[31,3172,3174],{"className":3173},[293],[31,3175,3178],{"className":3176,"style":3177},[297],"height:0.686em;",[31,3179],{},[31,3181],{"className":3182},[254,3097],[31,3184],{"className":3185,"style":124},[123],[31,3187,100],{"className":3188},[128],[31,3190],{"className":3191,"style":124},[123],[31,3193,3195,3198],{"className":3194},[65],[31,3196],{"className":3197,"style":138},[69],[31,3199,3077],{"className":3200},[74],[15,3202,3203],{},"And the second-order condition ensures a maximum:",[31,3205,3207],{"className":3206,"translate":35},[149],[31,3208,3210,3259],{"className":3209,"translate":35},[34],[31,3211,3213],{"className":3212},[39],[41,3214,3215],{"xmlns":43,"display":158},[45,3216,3217,3256],{},[48,3218,3219,3251,3254],{},[3048,3220,3221,3241],{},[48,3222,3223,3229,3231,3233,3235,3237,3239],{},[189,3224,3225,3227],{},[51,3226,3054],{"mathvariant":175},[102,3228,479],{},[51,3230,2300],{"mathvariant":175},[98,3232,169],{"stretchy":168},[51,3234,1178],{},[51,3236,176],{"mathvariant":175},[51,3238,172],{},[98,3240,185],{"stretchy":168},[48,3242,3243,3245],{},[51,3244,3054],{"mathvariant":175},[189,3246,3247,3249],{},[51,3248,1178],{},[102,3250,479],{},[98,3252,3253],{},"\u003C",[102,3255,3077],{},[54,3257,3258],{"encoding":56},"\\frac{\\partial^2 \\ell(\\theta|D)}{\\partial \\theta^2} \u003C 0",[31,3260,3262,3418],{"className":3261,"ariaHidden":61},[60],[31,3263,3265,3269,3409,3412,3415],{"className":3264},[65],[31,3266],{"className":3267,"style":3268},[69],"height:2.1771em;vertical-align:-0.686em;",[31,3270,3272,3275,3406],{"className":3271},[74],[31,3273],{"className":3274},[222,3097],[31,3276,3278],{"className":3277},[3048],[31,3279,3281,3398],{"className":3280},[289,836],[31,3282,3284,3395],{"className":3283},[293],[31,3285,3288,3330,3338],{"className":3286,"style":3287},[297],"height:1.4911em;",[31,3289,3290,3293],{"style":3113},[31,3291],{"className":3292,"style":850},[304],[31,3294,3296,3299],{"className":3295},[74],[31,3297,3054],{"className":3298,"style":3123},[74],[31,3300,3302,3305],{"className":3301},[74],[31,3303,1178],{"className":3304,"style":226},[74,75],[31,3306,3308],{"className":3307},[285],[31,3309,3311],{"className":3310},[289],[31,3312,3314],{"className":3313},[293],[31,3315,3318],{"className":3316,"style":3317},[297],"height:0.7401em;",[31,3319,3321,3324],{"style":3320},"top:-2.989em;margin-right:0.05em;",[31,3322],{"className":3323,"style":305},[304],[31,3325,3327],{"className":3326},[309,310,311,312],[31,3328,479],{"className":3329},[74,312],[31,3331,3332,3335],{"style":3129},[31,3333],{"className":3334,"style":850},[304],[31,3336],{"className":3337,"style":3137},[3136],[31,3339,3340,3343],{"style":3140},[31,3341],{"className":3342,"style":850},[304],[31,3344,3346,3377,3380,3383,3386,3389,3392],{"className":3345},[74],[31,3347,3349,3352],{"className":3348},[74],[31,3350,3054],{"className":3351,"style":3123},[74],[31,3353,3355],{"className":3354},[285],[31,3356,3358],{"className":3357},[289],[31,3359,3361],{"className":3360},[293],[31,3362,3365],{"className":3363,"style":3364},[297],"height:0.8141em;",[31,3366,3368,3371],{"style":3367},"top:-3.063em;margin-right:0.05em;",[31,3369],{"className":3370,"style":305},[304],[31,3372,3374],{"className":3373},[309,310,311,312],[31,3375,479],{"className":3376},[74,312],[31,3378,2300],{"className":3379},[74],[31,3381,169],{"className":3382},[222],[31,3384,1178],{"className":3385,"style":226},[74,75],[31,3387,176],{"className":3388},[74],[31,3390,172],{"className":3391,"style":226},[74,75],[31,3393,185],{"className":3394},[254],[31,3396,871],{"className":3397},[870],[31,3399,3401],{"className":3400},[293],[31,3402,3404],{"className":3403,"style":3177},[297],[31,3405],{},[31,3407],{"className":3408},[254,3097],[31,3410],{"className":3411,"style":124},[123],[31,3413,3253],{"className":3414},[128],[31,3416],{"className":3417,"style":124},[123],[31,3419,3421,3424],{"className":3420},[65],[31,3422],{"className":3423,"style":138},[69],[31,3425,3077],{"className":3426},[74],[15,3428,3429],{},"Take the simplest Bernoulli distribution as an example:",[1722,3431,3432,3551],{},[1725,3433,3434,3435],{},"Data ",[31,3436,3438,3468],{"className":3437,"translate":35},[34],[31,3439,3441],{"className":3440},[39],[41,3442,3443],{"xmlns":43},[45,3444,3445,3465],{},[48,3446,3447,3453,3455,3457,3459,3461,3463],{},[932,3448,3449,3451],{},[51,3450,936],{},[51,3452,2018],{},[98,3454,2021],{},[98,3456,930],{"stretchy":168},[102,3458,3077],{},[98,3460,942],{"separator":61},[102,3462,939],{},[98,3464,969],{"stretchy":168},[54,3466,3467],{"encoding":56},"x_i \\in \\{0,1\\}",[31,3469,3471,3527],{"className":3470,"ariaHidden":61},[60],[31,3472,3474,3478,3518,3521,3524],{"className":3473},[65],[31,3475],{"className":3476,"style":3477},[69],"height:0.6891em;vertical-align:-0.15em;",[31,3479,3481,3484],{"className":3480},[74],[31,3482,936],{"className":3483},[74,75],[31,3485,3487],{"className":3486},[285],[31,3488,3490,3510],{"className":3489},[289,836],[31,3491,3493,3507],{"className":3492},[293],[31,3494,3496],{"className":3495,"style":2216},[297],[31,3497,3498,3501],{"style":1025},[31,3499],{"className":3500,"style":305},[304],[31,3502,3504],{"className":3503},[309,310,311,312],[31,3505,2018],{"className":3506},[74,75,312],[31,3508,871],{"className":3509},[870],[31,3511,3513],{"className":3512},[293],[31,3514,3516],{"className":3515,"style":1044},[297],[31,3517],{},[31,3519],{"className":3520,"style":124},[123],[31,3522,2021],{"className":3523},[128],[31,3525],{"className":3526,"style":124},[123],[31,3528,3530,3533,3536,3539,3542,3545,3548],{"className":3529},[65],[31,3531],{"className":3532,"style":214},[69],[31,3534,930],{"className":3535},[222],[31,3537,3077],{"className":3538},[74],[31,3540,942],{"className":3541},[1050],[31,3543],{"className":3544,"style":1054},[123],[31,3546,939],{"className":3547},[74],[31,3549,969],{"className":3550},[254],[1725,3552,3553,3554],{},"Model: ",[31,3555,3557,3609],{"className":3556,"translate":35},[34],[31,3558,3560],{"className":3559},[39],[41,3561,3562],{"xmlns":43},[45,3563,3564,3606],{},[48,3565,3566,3568,3570,3573,3575,3577,3579,3581,3583,3585,3587,3589,3591,3593,3595,3597,3599,3601,3604],{},[51,3567,165],{},[98,3569,169],{"stretchy":168},[51,3571,3572],{},"X",[98,3574,100],{},[102,3576,939],{},[98,3578,185],{"stretchy":168},[98,3580,100],{},[51,3582,15],{},[98,3584,942],{"separator":61},[51,3586,165],{},[98,3588,169],{"stretchy":168},[51,3590,3572],{},[98,3592,100],{},[102,3594,3077],{},[98,3596,185],{"stretchy":168},[98,3598,100],{},[102,3600,939],{},[98,3602,3603],{},"−",[51,3605,15],{},[54,3607,3608],{"encoding":56},"P(X=1)=p, P(X=0)=1-p",[31,3610,3612,3637,3658,3691,3712,3731],{"className":3611,"ariaHidden":61},[60],[31,3613,3615,3618,3621,3624,3628,3631,3634],{"className":3614},[65],[31,3616],{"className":3617,"style":214},[69],[31,3619,165],{"className":3620,"style":218},[74,75],[31,3622,169],{"className":3623},[222],[31,3625,3572],{"className":3626,"style":3627},[74,75],"margin-right:0.0785em;",[31,3629],{"className":3630,"style":124},[123],[31,3632,100],{"className":3633},[128],[31,3635],{"className":3636,"style":124},[123],[31,3638,3640,3643,3646,3649,3652,3655],{"className":3639},[65],[31,3641],{"className":3642,"style":214},[69],[31,3644,939],{"className":3645},[74],[31,3647,185],{"className":3648},[254],[31,3650],{"className":3651,"style":124},[123],[31,3653,100],{"className":3654},[128],[31,3656],{"className":3657,"style":124},[123],[31,3659,3661,3664,3667,3670,3673,3676,3679,3682,3685,3688],{"className":3660},[65],[31,3662],{"className":3663,"style":214},[69],[31,3665,15],{"className":3666},[74,75],[31,3668,942],{"className":3669},[1050],[31,3671],{"className":3672,"style":1054},[123],[31,3674,165],{"className":3675,"style":218},[74,75],[31,3677,169],{"className":3678},[222],[31,3680,3572],{"className":3681,"style":3627},[74,75],[31,3683],{"className":3684,"style":124},[123],[31,3686,100],{"className":3687},[128],[31,3689],{"className":3690,"style":124},[123],[31,3692,3694,3697,3700,3703,3706,3709],{"className":3693},[65],[31,3695],{"className":3696,"style":214},[69],[31,3698,3077],{"className":3699},[74],[31,3701,185],{"className":3702},[254],[31,3704],{"className":3705,"style":124},[123],[31,3707,100],{"className":3708},[128],[31,3710],{"className":3711,"style":124},[123],[31,3713,3715,3719,3722,3725,3728],{"className":3714},[65],[31,3716],{"className":3717,"style":3718},[69],"height:0.7278em;vertical-align:-0.0833em;",[31,3720,939],{"className":3721},[74],[31,3723],{"className":3724,"style":584},[123],[31,3726,3603],{"className":3727},[588],[31,3729],{"className":3730,"style":584},[123],[31,3732,3734,3737],{"className":3733},[65],[31,3735],{"className":3736,"style":70},[69],[31,3738,15],{"className":3739},[74,75],[15,3741,3742],{},"We have the likelihood function:",[31,3744,3746],{"className":3745,"translate":35},[149],[31,3747,3749,3824],{"className":3748,"translate":35},[34],[31,3750,3752],{"className":3751},[39],[41,3753,3754],{"xmlns":43,"display":158},[45,3755,3756,3821],{},[48,3757,3758,3760,3762,3764,3766,3768,3770,3772,3787,3797,3799,3801,3803,3805],{},[51,3759,1218],{},[98,3761,169],{"stretchy":168},[51,3763,15],{},[51,3765,176],{"mathvariant":175},[51,3767,172],{},[98,3769,185],{"stretchy":168},[98,3771,100],{},[3773,3774,3775,3777,3785],"munderover",{},[98,3776,2009],{},[48,3778,3779,3781,3783],{},[51,3780,2018],{},[98,3782,100],{},[102,3784,939],{},[51,3786,966],{},[189,3788,3789,3791],{},[51,3790,15],{},[932,3792,3793,3795],{},[51,3794,936],{},[51,3796,2018],{},[98,3798,169],{"stretchy":168},[102,3800,939],{},[98,3802,3603],{},[51,3804,15],{},[189,3806,3807,3809],{},[98,3808,185],{"stretchy":168},[48,3810,3811,3813,3815],{},[102,3812,939],{},[98,3814,3603],{},[932,3816,3817,3819],{},[51,3818,936],{},[51,3820,2018],{},[54,3822,3823],{"encoding":56},"L(p|D) = \\prod_{i=1}^n p^{x_i} (1-p)^{1-x_i}",[31,3825,3827,3860,4023],{"className":3826,"ariaHidden":61},[60],[31,3828,3830,3833,3836,3839,3842,3845,3848,3851,3854,3857],{"className":3829},[65],[31,3831],{"className":3832,"style":214},[69],[31,3834,1218],{"className":3835},[74,75],[31,3837,169],{"className":3838},[222],[31,3840,15],{"className":3841},[74,75],[31,3843,176],{"className":3844},[74],[31,3846,172],{"className":3847,"style":226},[74,75],[31,3849,185],{"className":3850},[254],[31,3852],{"className":3853,"style":124},[123],[31,3855,100],{"className":3856},[128],[31,3858],{"className":3859,"style":124},[123],[31,3861,3863,3867,3935,3938,4008,4011,4014,4017,4020],{"className":3862},[65],[31,3864],{"className":3865,"style":3866},[69],"height:2.9291em;vertical-align:-1.2777em;",[31,3868,3870],{"className":3869},[1443,1455],[31,3871,3873,3926],{"className":3872},[289,836],[31,3874,3876,3923],{"className":3875},[293],[31,3877,3880,3901,3911],{"className":3878,"style":3879},[297],"height:1.6514em;",[31,3881,3883,3886],{"style":3882},"top:-1.8723em;margin-left:0em;",[31,3884],{"className":3885,"style":2105},[304],[31,3887,3889],{"className":3888},[309,310,311,312],[31,3890,3892,3895,3898],{"className":3891},[74,312],[31,3893,2018],{"className":3894},[74,75,312],[31,3896,100],{"className":3897},[128,312],[31,3899,939],{"className":3900},[74,312],[31,3902,3903,3906],{"style":2166},[31,3904],{"className":3905,"style":2105},[304],[31,3907,3908],{},[31,3909,2009],{"className":3910},[1443,2175,2176],[31,3912,3914,3917],{"style":3913},"top:-4.3em;margin-left:0em;",[31,3915],{"className":3916,"style":2105},[304],[31,3918,3920],{"className":3919},[309,310,311,312],[31,3921,966],{"className":3922},[74,75,312],[31,3924,871],{"className":3925},[870],[31,3927,3929],{"className":3928},[293],[31,3930,3933],{"className":3931,"style":3932},[297],"height:1.2777em;",[31,3934],{},[31,3936],{"className":3937,"style":1054},[123],[31,3939,3941,3944],{"className":3940},[74],[31,3942,15],{"className":3943},[74,75],[31,3945,3947],{"className":3946},[285],[31,3948,3950],{"className":3949},[289],[31,3951,3953],{"className":3952},[293],[31,3954,3957],{"className":3955,"style":3956},[297],"height:0.7144em;",[31,3958,3959,3962],{"style":300},[31,3960],{"className":3961,"style":305},[304],[31,3963,3965],{"className":3964},[309,310,311,312],[31,3966,3968],{"className":3967},[74,312],[31,3969,3971,3974],{"className":3970},[74,312],[31,3972,936],{"className":3973},[74,75,312],[31,3975,3977],{"className":3976},[285],[31,3978,3980,4000],{"className":3979},[289,836],[31,3981,3983,3997],{"className":3982},[293],[31,3984,3986],{"className":3985,"style":2130},[297],[31,3987,3988,3991],{"style":2133},[31,3989],{"className":3990,"style":2137},[304],[31,3992,3994],{"className":3993},[309,2141,2142,312],[31,3995,2018],{"className":3996},[74,75,312],[31,3998,871],{"className":3999},[870],[31,4001,4003],{"className":4002},[293],[31,4004,4006],{"className":4005,"style":2155},[297],[31,4007],{},[31,4009,169],{"className":4010},[222],[31,4012,939],{"className":4013},[74],[31,4015],{"className":4016,"style":584},[123],[31,4018,3603],{"className":4019},[588],[31,4021],{"className":4022,"style":584},[123],[31,4024,4026,4030,4033],{"className":4025},[65],[31,4027],{"className":4028,"style":4029},[69],"height:1.1141em;vertical-align:-0.25em;",[31,4031,15],{"className":4032},[74,75],[31,4034,4036,4039],{"className":4035},[254],[31,4037,185],{"className":4038},[254],[31,4040,4042],{"className":4041},[285],[31,4043,4045],{"className":4044},[289],[31,4046,4048],{"className":4047},[293],[31,4049,4051],{"className":4050,"style":270},[297],[31,4052,4053,4056],{"style":300},[31,4054],{"className":4055,"style":305},[304],[31,4057,4059],{"className":4058},[309,310,311,312],[31,4060,4062,4065,4068],{"className":4061},[74,312],[31,4063,939],{"className":4064},[74,312],[31,4066,3603],{"className":4067},[588,312],[31,4069,4071,4074],{"className":4070},[74,312],[31,4072,936],{"className":4073},[74,75,312],[31,4075,4077],{"className":4076},[285],[31,4078,4080,4100],{"className":4079},[289,836],[31,4081,4083,4097],{"className":4082},[293],[31,4084,4086],{"className":4085,"style":2130},[297],[31,4087,4088,4091],{"style":2133},[31,4089],{"className":4090,"style":2137},[304],[31,4092,4094],{"className":4093},[309,2141,2142,312],[31,4095,2018],{"className":4096},[74,75,312],[31,4098,871],{"className":4099},[870],[31,4101,4103],{"className":4102},[293],[31,4104,4106],{"className":4105,"style":2155},[297],[31,4107],{},[15,4109,4110],{},"Convert to log-likelihood:",[31,4112,4114],{"className":4113,"translate":35},[149],[31,4115,4117,4200],{"className":4116,"translate":35},[34],[31,4118,4120],{"className":4119},[39],[41,4121,4122],{"xmlns":43,"display":158},[45,4123,4124,4197],{},[48,4125,4126,4128,4130,4132,4134,4136,4138,4140,4154,4160,4162,4164,4166,4169,4171,4173,4175,4181,4183,4185,4187,4189,4191,4193,4195],{},[51,4127,2300],{"mathvariant":175},[98,4129,169],{"stretchy":168},[51,4131,15],{},[51,4133,176],{"mathvariant":175},[51,4135,172],{},[98,4137,185],{"stretchy":168},[98,4139,100],{},[3773,4141,4142,4144,4152],{},[98,4143,2336],{},[48,4145,4146,4148,4150],{},[51,4147,2018],{},[98,4149,100],{},[102,4151,939],{},[51,4153,966],{},[932,4155,4156,4158],{},[51,4157,936],{},[51,4159,2018],{},[51,4161,2315],{},[98,4163,1330],{},[51,4165,15],{},[98,4167,4168],{},"+",[98,4170,169],{"stretchy":168},[102,4172,939],{},[98,4174,3603],{},[932,4176,4177,4179],{},[51,4178,936],{},[51,4180,2018],{},[98,4182,185],{"stretchy":168},[51,4184,2315],{},[98,4186,1330],{},[98,4188,169],{"stretchy":168},[102,4190,939],{},[98,4192,3603],{},[51,4194,15],{},[98,4196,185],{"stretchy":168},[54,4198,4199],{"encoding":56},"\\ell(p|D) = \\sum_{i=1}^n x_i \\log p + (1-x_i) \\log (1-p)",[31,4201,4203,4236,4372,4393,4465],{"className":4202,"ariaHidden":61},[60],[31,4204,4206,4209,4212,4215,4218,4221,4224,4227,4230,4233],{"className":4205},[65],[31,4207],{"className":4208,"style":214},[69],[31,4210,2300],{"className":4211},[74],[31,4213,169],{"className":4214},[222],[31,4216,15],{"className":4217},[74,75],[31,4219,176],{"className":4220},[74],[31,4222,172],{"className":4223,"style":226},[74,75],[31,4225,185],{"className":4226},[254],[31,4228],{"className":4229,"style":124},[123],[31,4231,100],{"className":4232},[128],[31,4234],{"className":4235,"style":124},[123],[31,4237,4239,4242,4306,4309,4349,4352,4357,4360,4363,4366,4369],{"className":4238},[65],[31,4240],{"className":4241,"style":3866},[69],[31,4243,4245],{"className":4244},[1443,1455],[31,4246,4248,4298],{"className":4247},[289,836],[31,4249,4251,4295],{"className":4250},[293],[31,4252,4254,4274,4284],{"className":4253,"style":3879},[297],[31,4255,4256,4259],{"style":3882},[31,4257],{"className":4258,"style":2105},[304],[31,4260,4262],{"className":4261},[309,310,311,312],[31,4263,4265,4268,4271],{"className":4264},[74,312],[31,4266,2018],{"className":4267},[74,75,312],[31,4269,100],{"className":4270},[128,312],[31,4272,939],{"className":4273},[74,312],[31,4275,4276,4279],{"style":2166},[31,4277],{"className":4278,"style":2105},[304],[31,4280,4281],{},[31,4282,2336],{"className":4283},[1443,2175,2176],[31,4285,4286,4289],{"style":3913},[31,4287],{"className":4288,"style":2105},[304],[31,4290,4292],{"className":4291},[309,310,311,312],[31,4293,966],{"className":4294},[74,75,312],[31,4296,871],{"className":4297},[870],[31,4299,4301],{"className":4300},[293],[31,4302,4304],{"className":4303,"style":3932},[297],[31,4305],{},[31,4307],{"className":4308,"style":1054},[123],[31,4310,4312,4315],{"className":4311},[74],[31,4313,936],{"className":4314},[74,75],[31,4316,4318],{"className":4317},[285],[31,4319,4321,4341],{"className":4320},[289,836],[31,4322,4324,4338],{"className":4323},[293],[31,4325,4327],{"className":4326,"style":2216},[297],[31,4328,4329,4332],{"style":1025},[31,4330],{"className":4331,"style":305},[304],[31,4333,4335],{"className":4334},[309,310,311,312],[31,4336,2018],{"className":4337},[74,75,312],[31,4339,871],{"className":4340},[870],[31,4342,4344],{"className":4343},[293],[31,4345,4347],{"className":4346,"style":1044},[297],[31,4348],{},[31,4350],{"className":4351,"style":1054},[123],[31,4353,2417,4355],{"className":4354},[1443],[31,4356,1448],{"style":1447},[31,4358],{"className":4359,"style":1054},[123],[31,4361,15],{"className":4362},[74,75],[31,4364],{"className":4365,"style":584},[123],[31,4367,4168],{"className":4368},[588],[31,4370],{"className":4371,"style":584},[123],[31,4373,4375,4378,4381,4384,4387,4390],{"className":4374},[65],[31,4376],{"className":4377,"style":214},[69],[31,4379,169],{"className":4380},[222],[31,4382,939],{"className":4383},[74],[31,4385],{"className":4386,"style":584},[123],[31,4388,3603],{"className":4389},[588],[31,4391],{"className":4392,"style":584},[123],[31,4394,4396,4399,4439,4442,4445,4450,4453,4456,4459,4462],{"className":4395},[65],[31,4397],{"className":4398,"style":214},[69],[31,4400,4402,4405],{"className":4401},[74],[31,4403,936],{"className":4404},[74,75],[31,4406,4408],{"className":4407},[285],[31,4409,4411,4431],{"className":4410},[289,836],[31,4412,4414,4428],{"className":4413},[293],[31,4415,4417],{"className":4416,"style":2216},[297],[31,4418,4419,4422],{"style":1025},[31,4420],{"className":4421,"style":305},[304],[31,4423,4425],{"className":4424},[309,310,311,312],[31,4426,2018],{"className":4427},[74,75,312],[31,4429,871],{"className":4430},[870],[31,4432,4434],{"className":4433},[293],[31,4435,4437],{"className":4436,"style":1044},[297],[31,4438],{},[31,4440,185],{"className":4441},[254],[31,4443],{"className":4444,"style":1054},[123],[31,4446,2417,4448],{"className":4447},[1443],[31,4449,1448],{"style":1447},[31,4451,169],{"className":4452},[222],[31,4454,939],{"className":4455},[74],[31,4457],{"className":4458,"style":584},[123],[31,4460,3603],{"className":4461},[588],[31,4463],{"className":4464,"style":584},[123],[31,4466,4468,4471,4474],{"className":4467},[65],[31,4469],{"className":4470,"style":214},[69],[31,4472,15],{"className":4473},[74,75],[31,4475,185],{"className":4476},[254],[15,4478,4479,4480,4508],{},"Take the derivative with respect to ",[31,4481,4483,4496],{"className":4482,"translate":35},[34],[31,4484,4486],{"className":4485},[39],[41,4487,4488],{"xmlns":43},[45,4489,4490,4494],{},[48,4491,4492],{},[51,4493,15],{},[54,4495,15],{"encoding":56},[31,4497,4499],{"className":4498,"ariaHidden":61},[60],[31,4500,4502,4505],{"className":4501},[65],[31,4503],{"className":4504,"style":70},[69],[31,4506,15],{"className":4507},[74,75]," and set to 0:",[31,4510,4512],{"className":4511,"translate":35},[149],[31,4513,4515,4587],{"className":4514,"translate":35},[34],[31,4516,4518],{"className":4517},[39],[41,4519,4520],{"xmlns":43,"display":158},[45,4521,4522,4584],{},[48,4523,4524,4538,4540,4554,4556,4580,4582],{},[3048,4525,4526,4532],{},[48,4527,4528,4530],{},[51,4529,3054],{"mathvariant":175},[51,4531,2300],{"mathvariant":175},[48,4533,4534,4536],{},[51,4535,3054],{"mathvariant":175},[51,4537,15],{},[98,4539,100],{},[3048,4541,4542,4552],{},[48,4543,4544,4546],{},[98,4545,2336],{},[932,4547,4548,4550],{},[51,4549,936],{},[51,4551,2018],{},[51,4553,15],{},[98,4555,3603],{},[3048,4557,4558,4572],{},[48,4559,4560,4562,4564,4566],{},[51,4561,966],{},[98,4563,3603],{},[98,4565,2336],{},[932,4567,4568,4570],{},[51,4569,936],{},[51,4571,2018],{},[48,4573,4574,4576,4578],{},[102,4575,939],{},[98,4577,3603],{},[51,4579,15],{},[98,4581,100],{},[102,4583,3077],{},[54,4585,4586],{"encoding":56},"\\frac{\\partial \\ell}{\\partial p} = \\frac{\\sum x_i}{p} - \\frac{n-\\sum x_i}{1-p} = 0",[31,4588,4590,4676,4799,4943],{"className":4589,"ariaHidden":61},[60],[31,4591,4593,4597,4667,4670,4673],{"className":4592},[65],[31,4594],{"className":4595,"style":4596},[69],"height:2.2519em;vertical-align:-0.8804em;",[31,4598,4600,4603,4664],{"className":4599},[74],[31,4601],{"className":4602},[222,3097],[31,4604,4606],{"className":4605},[3048],[31,4607,4609,4655],{"className":4608},[289,836],[31,4610,4612,4652],{"className":4611},[293],[31,4613,4616,4630,4638],{"className":4614,"style":4615},[297],"height:1.3714em;",[31,4617,4618,4621],{"style":3113},[31,4619],{"className":4620,"style":850},[304],[31,4622,4624,4627],{"className":4623},[74],[31,4625,3054],{"className":4626,"style":3123},[74],[31,4628,15],{"className":4629},[74,75],[31,4631,4632,4635],{"style":3129},[31,4633],{"className":4634,"style":850},[304],[31,4636],{"className":4637,"style":3137},[3136],[31,4639,4640,4643],{"style":3140},[31,4641],{"className":4642,"style":850},[304],[31,4644,4646,4649],{"className":4645},[74],[31,4647,3054],{"className":4648,"style":3123},[74],[31,4650,2300],{"className":4651},[74],[31,4653,871],{"className":4654},[870],[31,4656,4658],{"className":4657},[293],[31,4659,4662],{"className":4660,"style":4661},[297],"height:0.8804em;",[31,4663],{},[31,4665],{"className":4666},[254,3097],[31,4668],{"className":4669,"style":124},[123],[31,4671,100],{"className":4672},[128],[31,4674],{"className":4675,"style":124},[123],[31,4677,4679,4683,4790,4793,4796],{"className":4678},[65],[31,4680],{"className":4681,"style":4682},[69],"height:2.3074em;vertical-align:-0.8804em;",[31,4684,4686,4689,4787],{"className":4685},[74],[31,4687],{"className":4688},[222,3097],[31,4690,4692],{"className":4691},[3048],[31,4693,4695,4779],{"className":4694},[289,836],[31,4696,4698,4776],{"className":4697},[293],[31,4699,4701,4712,4720],{"className":4700,"style":3110},[297],[31,4702,4703,4706],{"style":3113},[31,4704],{"className":4705,"style":850},[304],[31,4707,4709],{"className":4708},[74],[31,4710,15],{"className":4711},[74,75],[31,4713,4714,4717],{"style":3129},[31,4715],{"className":4716,"style":850},[304],[31,4718],{"className":4719,"style":3137},[3136],[31,4721,4722,4725],{"style":3140},[31,4723],{"className":4724,"style":850},[304],[31,4726,4728,4733,4736],{"className":4727},[74],[31,4729,2336],{"className":4730,"style":4732},[1443,2175,4731],"small-op","position:relative;top:0em;",[31,4734],{"className":4735,"style":1054},[123],[31,4737,4739,4742],{"className":4738},[74],[31,4740,936],{"className":4741},[74,75],[31,4743,4745],{"className":4744},[285],[31,4746,4748,4768],{"className":4747},[289,836],[31,4749,4751,4765],{"className":4750},[293],[31,4752,4754],{"className":4753,"style":2216},[297],[31,4755,4756,4759],{"style":1025},[31,4757],{"className":4758,"style":305},[304],[31,4760,4762],{"className":4761},[309,310,311,312],[31,4763,2018],{"className":4764},[74,75,312],[31,4766,871],{"className":4767},[870],[31,4769,4771],{"className":4770},[293],[31,4772,4774],{"className":4773,"style":1044},[297],[31,4775],{},[31,4777,871],{"className":4778},[870],[31,4780,4782],{"className":4781},[293],[31,4783,4785],{"className":4784,"style":4661},[297],[31,4786],{},[31,4788],{"className":4789},[254,3097],[31,4791],{"className":4792,"style":584},[123],[31,4794,3603],{"className":4795},[588],[31,4797],{"className":4798,"style":584},[123],[31,4800,4802,4805,4934,4937,4940],{"className":4801},[65],[31,4803],{"className":4804,"style":4682},[69],[31,4806,4808,4811,4931],{"className":4807},[74],[31,4809],{"className":4810},[222,3097],[31,4812,4814],{"className":4813},[3048],[31,4815,4817,4923],{"className":4816},[289,836],[31,4818,4820,4920],{"className":4819},[293],[31,4821,4823,4846,4854],{"className":4822,"style":3110},[297],[31,4824,4825,4828],{"style":3113},[31,4826],{"className":4827,"style":850},[304],[31,4829,4831,4834,4837,4840,4843],{"className":4830},[74],[31,4832,939],{"className":4833},[74],[31,4835],{"className":4836,"style":584},[123],[31,4838,3603],{"className":4839},[588],[31,4841],{"className":4842,"style":584},[123],[31,4844,15],{"className":4845},[74,75],[31,4847,4848,4851],{"style":3129},[31,4849],{"className":4850,"style":850},[304],[31,4852],{"className":4853,"style":3137},[3136],[31,4855,4856,4859],{"style":3140},[31,4857],{"className":4858,"style":850},[304],[31,4860,4862,4865,4868,4871,4874,4877,4880],{"className":4861},[74],[31,4863,966],{"className":4864},[74,75],[31,4866],{"className":4867,"style":584},[123],[31,4869,3603],{"className":4870},[588],[31,4872],{"className":4873,"style":584},[123],[31,4875,2336],{"className":4876,"style":4732},[1443,2175,4731],[31,4878],{"className":4879,"style":1054},[123],[31,4881,4883,4886],{"className":4882},[74],[31,4884,936],{"className":4885},[74,75],[31,4887,4889],{"className":4888},[285],[31,4890,4892,4912],{"className":4891},[289,836],[31,4893,4895,4909],{"className":4894},[293],[31,4896,4898],{"className":4897,"style":2216},[297],[31,4899,4900,4903],{"style":1025},[31,4901],{"className":4902,"style":305},[304],[31,4904,4906],{"className":4905},[309,310,311,312],[31,4907,2018],{"className":4908},[74,75,312],[31,4910,871],{"className":4911},[870],[31,4913,4915],{"className":4914},[293],[31,4916,4918],{"className":4917,"style":1044},[297],[31,4919],{},[31,4921,871],{"className":4922},[870],[31,4924,4926],{"className":4925},[293],[31,4927,4929],{"className":4928,"style":4661},[297],[31,4930],{},[31,4932],{"className":4933},[254,3097],[31,4935],{"className":4936,"style":124},[123],[31,4938,100],{"className":4939},[128],[31,4941],{"className":4942,"style":124},[123],[31,4944,4946,4949],{"className":4945},[65],[31,4947],{"className":4948,"style":138},[69],[31,4950,3077],{"className":4951},[74],[15,4953,4954],{},"Solve to obtain:",[31,4956,4958],{"className":4957,"translate":35},[149],[31,4959,4961,5017],{"className":4960,"translate":35},[34],[31,4962,4964],{"className":4963},[39],[41,4965,4966],{"xmlns":43,"display":158},[45,4967,4968,5014],{},[48,4969,4970,4986,4988],{},[932,4971,4972,4978],{},[805,4973,4974,4976],{"accent":61},[51,4975,15],{},[98,4977,811],{},[48,4979,4980,4982,4984],{},[51,4981,2938],{"mathvariant":175},[51,4983,1218],{"mathvariant":175},[51,4985,2943],{"mathvariant":175},[98,4987,100],{},[3048,4989,4990,5012],{},[48,4991,4992,5006],{},[3773,4993,4994,4996,5004],{},[98,4995,2336],{},[48,4997,4998,5000,5002],{},[51,4999,2018],{},[98,5001,100],{},[102,5003,939],{},[51,5005,966],{},[932,5007,5008,5010],{},[51,5009,936],{},[51,5011,2018],{},[51,5013,966],{},[54,5015,5016],{"encoding":56},"\\hat{p}_{\\rm MLE} = \\frac{\\sum_{i=1}^n x_i}{n}",[31,5018,5020,5120],{"className":5019,"ariaHidden":61},[60],[31,5021,5023,5026,5111,5114,5117],{"className":5022},[65],[31,5024],{"className":5025,"style":828},[69],[31,5027,5029,5071],{"className":5028},[74],[31,5030,5032],{"className":5031},[74,832],[31,5033,5035,5063],{"className":5034},[289,836],[31,5036,5038,5060],{"className":5037},[293],[31,5039,5041,5049],{"className":5040,"style":843},[297],[31,5042,5043,5046],{"style":846},[31,5044],{"className":5045,"style":850},[304],[31,5047,15],{"className":5048},[74,75],[31,5050,5051,5054],{"style":846},[31,5052],{"className":5053,"style":850},[304],[31,5055,5057],{"className":5056,"style":863},[862],[31,5058,811],{"className":5059},[74],[31,5061,871],{"className":5062},[870],[31,5064,5066],{"className":5065},[293],[31,5067,5069],{"className":5068,"style":878},[297],[31,5070],{},[31,5072,5074],{"className":5073},[285],[31,5075,5077,5103],{"className":5076},[289,836],[31,5078,5080,5100],{"className":5079},[293],[31,5081,5083],{"className":5082,"style":2823},[297],[31,5084,5085,5088],{"style":1025},[31,5086],{"className":5087,"style":305},[304],[31,5089,5091],{"className":5090},[309,310,311,312],[31,5092,5094],{"className":5093},[74,312],[31,5095,5097],{"className":5096},[74,312],[31,5098,2763],{"className":5099},[74,3019,312],[31,5101,871],{"className":5102},[870],[31,5104,5106],{"className":5105},[293],[31,5107,5109],{"className":5108,"style":1044},[297],[31,5110],{},[31,5112],{"className":5113,"style":124},[123],[31,5115,100],{"className":5116},[128],[31,5118],{"className":5119,"style":124},[123],[31,5121,5123,5127],{"className":5122},[65],[31,5124],{"className":5125,"style":5126},[69],"height:2.18em;vertical-align:-0.686em;",[31,5128,5130,5133,5292],{"className":5129},[74],[31,5131],{"className":5132},[222,3097],[31,5134,5136],{"className":5135},[3048],[31,5137,5139,5284],{"className":5138},[289,836],[31,5140,5142,5281],{"className":5141},[293],[31,5143,5146,5157,5165],{"className":5144,"style":5145},[297],"height:1.494em;",[31,5147,5148,5151],{"style":3113},[31,5149],{"className":5150,"style":850},[304],[31,5152,5154],{"className":5153},[74],[31,5155,966],{"className":5156},[74,75],[31,5158,5159,5162],{"style":3129},[31,5160],{"className":5161,"style":850},[304],[31,5163],{"className":5164,"style":3137},[3136],[31,5166,5168,5171],{"style":5167},"top:-3.6897em;",[31,5169],{"className":5170,"style":850},[304],[31,5172,5174,5238,5241],{"className":5173},[74],[31,5175,5177,5180],{"className":5176},[1443],[31,5178,2336],{"className":5179,"style":4732},[1443,2175,4731],[31,5181,5183],{"className":5182},[285],[31,5184,5186,5229],{"className":5185},[289,836],[31,5187,5189,5226],{"className":5188},[293],[31,5190,5193,5214],{"className":5191,"style":5192},[297],"height:0.8043em;",[31,5194,5196,5199],{"style":5195},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[31,5197],{"className":5198,"style":305},[304],[31,5200,5202],{"className":5201},[309,310,311,312],[31,5203,5205,5208,5211],{"className":5204},[74,312],[31,5206,2018],{"className":5207},[74,75,312],[31,5209,100],{"className":5210},[128,312],[31,5212,939],{"className":5213},[74,312],[31,5215,5217,5220],{"style":5216},"top:-3.2029em;margin-right:0.05em;",[31,5218],{"className":5219,"style":305},[304],[31,5221,5223],{"className":5222},[309,310,311,312],[31,5224,966],{"className":5225},[74,75,312],[31,5227,871],{"className":5228},[870],[31,5230,5232],{"className":5231},[293],[31,5233,5236],{"className":5234,"style":5235},[297],"height:0.2997em;",[31,5237],{},[31,5239],{"className":5240,"style":1054},[123],[31,5242,5244,5247],{"className":5243},[74],[31,5245,936],{"className":5246},[74,75],[31,5248,5250],{"className":5249},[285],[31,5251,5253,5273],{"className":5252},[289,836],[31,5254,5256,5270],{"className":5255},[293],[31,5257,5259],{"className":5258,"style":2216},[297],[31,5260,5261,5264],{"style":1025},[31,5262],{"className":5263,"style":305},[304],[31,5265,5267],{"className":5266},[309,310,311,312],[31,5268,2018],{"className":5269},[74,75,312],[31,5271,871],{"className":5272},[870],[31,5274,5276],{"className":5275},[293],[31,5277,5279],{"className":5278,"style":1044},[297],[31,5280],{},[31,5282,871],{"className":5283},[870],[31,5285,5287],{"className":5286},[293],[31,5288,5290],{"className":5289,"style":3177},[297],[31,5291],{},[31,5293],{"className":5294},[254,3097],[15,5296,5297,5298,5326,5327,5330],{},"In neural networks, the MLE parameter ",[31,5299,5301,5314],{"className":5300,"translate":35},[34],[31,5302,5304],{"className":5303},[39],[41,5305,5306],{"xmlns":43},[45,5307,5308,5312],{},[48,5309,5310],{},[51,5311,1178],{},[54,5313,1181],{"encoding":56},[31,5315,5317],{"className":5316,"ariaHidden":61},[60],[31,5318,5320,5323],{"className":5319},[65],[31,5321],{"className":5322,"style":843},[69],[31,5324,1178],{"className":5325,"style":226},[74,75]," corresponds to the network's ",[370,5328,5329],{},"weights and biases",", typically denoted as:",[31,5332,5334],{"className":5333,"translate":35},[149],[31,5335,5337,5408],{"className":5336,"translate":35},[34],[31,5338,5340],{"className":5339},[39],[41,5341,5342],{"xmlns":43,"display":158},[45,5343,5344,5405],{},[48,5345,5346,5348,5350],{},[51,5347,1178],{},[98,5349,100],{},[48,5351,5352,5359,5361,5368,5370,5376,5378,5384,5386,5389,5391,5397,5399],{},[932,5353,5354,5357],{},[51,5355,5356],{},"W",[102,5358,939],{},[98,5360,942],{"separator":61},[932,5362,5363,5366],{},[51,5364,5365],{},"b",[102,5367,939],{},[98,5369,942],{"separator":61},[932,5371,5372,5374],{},[51,5373,5356],{},[102,5375,479],{},[98,5377,942],{"separator":61},[932,5379,5380,5382],{},[51,5381,5365],{},[102,5383,479],{},[98,5385,942],{"separator":61},[98,5387,5388],{},"…",[98,5390,942],{"separator":61},[932,5392,5393,5395],{},[51,5394,5356],{},[51,5396,1218],{},[98,5398,942],{"separator":61},[932,5400,5401,5403],{},[51,5402,5365],{},[51,5404,1218],{},[54,5406,5407],{"encoding":56},"\\theta = {W_1, b_1, W_2, b_2, \\dots, W_L, b_L}",[31,5409,5411,5429],{"className":5410,"ariaHidden":61},[60],[31,5412,5414,5417,5420,5423,5426],{"className":5413},[65],[31,5415],{"className":5416,"style":843},[69],[31,5418,1178],{"className":5419,"style":226},[74,75],[31,5421],{"className":5422,"style":124},[123],[31,5424,100],{"className":5425},[128],[31,5427],{"className":5428,"style":124},[123],[31,5430,5432,5435],{"className":5431},[65],[31,5433],{"className":5434,"style":828},[69],[31,5436,5438,5479,5482,5485,5525,5528,5531,5571,5574,5577,5617,5620,5623,5626,5629,5632,5635,5675,5678,5681],{"className":5437},[74],[31,5439,5441,5444],{"className":5440},[74],[31,5442,5356],{"className":5443,"style":218},[74,75],[31,5445,5447],{"className":5446},[285],[31,5448,5450,5471],{"className":5449},[289,836],[31,5451,5453,5468],{"className":5452},[293],[31,5454,5456],{"className":5455,"style":1022},[297],[31,5457,5459,5462],{"style":5458},"top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;",[31,5460],{"className":5461,"style":305},[304],[31,5463,5465],{"className":5464},[309,310,311,312],[31,5466,939],{"className":5467},[74,312],[31,5469,871],{"className":5470},[870],[31,5472,5474],{"className":5473},[293],[31,5475,5477],{"className":5476,"style":1044},[297],[31,5478],{},[31,5480,942],{"className":5481},[1050],[31,5483],{"className":5484,"style":1054},[123],[31,5486,5488,5491],{"className":5487},[74],[31,5489,5365],{"className":5490},[74,75],[31,5492,5494],{"className":5493},[285],[31,5495,5497,5517],{"className":5496},[289,836],[31,5498,5500,5514],{"className":5499},[293],[31,5501,5503],{"className":5502,"style":1022},[297],[31,5504,5505,5508],{"style":1025},[31,5506],{"className":5507,"style":305},[304],[31,5509,5511],{"className":5510},[309,310,311,312],[31,5512,939],{"className":5513},[74,312],[31,5515,871],{"className":5516},[870],[31,5518,5520],{"className":5519},[293],[31,5521,5523],{"className":5522,"style":1044},[297],[31,5524],{},[31,5526,942],{"className":5527},[1050],[31,5529],{"className":5530,"style":1054},[123],[31,5532,5534,5537],{"className":5533},[74],[31,5535,5356],{"className":5536,"style":218},[74,75],[31,5538,5540],{"className":5539},[285],[31,5541,5543,5563],{"className":5542},[289,836],[31,5544,5546,5560],{"className":5545},[293],[31,5547,5549],{"className":5548,"style":1022},[297],[31,5550,5551,5554],{"style":5458},[31,5552],{"className":5553,"style":305},[304],[31,5555,5557],{"className":5556},[309,310,311,312],[31,5558,479],{"className":5559},[74,312],[31,5561,871],{"className":5562},[870],[31,5564,5566],{"className":5565},[293],[31,5567,5569],{"className":5568,"style":1044},[297],[31,5570],{},[31,5572,942],{"className":5573},[1050],[31,5575],{"className":5576,"style":1054},[123],[31,5578,5580,5583],{"className":5579},[74],[31,5581,5365],{"className":5582},[74,75],[31,5584,5586],{"className":5585},[285],[31,5587,5589,5609],{"className":5588},[289,836],[31,5590,5592,5606],{"className":5591},[293],[31,5593,5595],{"className":5594,"style":1022},[297],[31,5596,5597,5600],{"style":1025},[31,5598],{"className":5599,"style":305},[304],[31,5601,5603],{"className":5602},[309,310,311,312],[31,5604,479],{"className":5605},[74,312],[31,5607,871],{"className":5608},[870],[31,5610,5612],{"className":5611},[293],[31,5613,5615],{"className":5614,"style":1044},[297],[31,5616],{},[31,5618,942],{"className":5619},[1050],[31,5621],{"className":5622,"style":1054},[123],[31,5624,5388],{"className":5625},[1104],[31,5627],{"className":5628,"style":1054},[123],[31,5630,942],{"className":5631},[1050],[31,5633],{"className":5634,"style":1054},[123],[31,5636,5638,5641],{"className":5637},[74],[31,5639,5356],{"className":5640,"style":218},[74,75],[31,5642,5644],{"className":5643},[285],[31,5645,5647,5667],{"className":5646},[289,836],[31,5648,5650,5664],{"className":5649},[293],[31,5651,5653],{"className":5652,"style":2823},[297],[31,5654,5655,5658],{"style":5458},[31,5656],{"className":5657,"style":305},[304],[31,5659,5661],{"className":5660},[309,310,311,312],[31,5662,1218],{"className":5663},[74,75,312],[31,5665,871],{"className":5666},[870],[31,5668,5670],{"className":5669},[293],[31,5671,5673],{"className":5672,"style":1044},[297],[31,5674],{},[31,5676,942],{"className":5677},[1050],[31,5679],{"className":5680,"style":1054},[123],[31,5682,5684,5687],{"className":5683},[74],[31,5685,5365],{"className":5686},[74,75],[31,5688,5690],{"className":5689},[285],[31,5691,5693,5713],{"className":5692},[289,836],[31,5694,5696,5710],{"className":5695},[293],[31,5697,5699],{"className":5698,"style":2823},[297],[31,5700,5701,5704],{"style":1025},[31,5702],{"className":5703,"style":305},[304],[31,5705,5707],{"className":5706},[309,310,311,312],[31,5708,1218],{"className":5709},[74,75,312],[31,5711,871],{"className":5712},[870],[31,5714,5716],{"className":5715},[293],[31,5717,5719],{"className":5718,"style":1044},[297],[31,5720],{},[15,5722,5723,5724,5752,5753,5824,5825,5854,5855,5926],{},"where ",[31,5725,5727,5740],{"className":5726,"translate":35},[34],[31,5728,5730],{"className":5729},[39],[41,5731,5732],{"xmlns":43},[45,5733,5734,5738],{},[48,5735,5736],{},[51,5737,1218],{},[54,5739,1218],{"encoding":56},[31,5741,5743],{"className":5742,"ariaHidden":61},[60],[31,5744,5746,5749],{"className":5745},[65],[31,5747],{"className":5748,"style":982},[69],[31,5750,1218],{"className":5751},[74,75]," is the number of layers, ",[31,5754,5756,5774],{"className":5755,"translate":35},[34],[31,5757,5759],{"className":5758},[39],[41,5760,5761],{"xmlns":43},[45,5762,5763,5771],{},[48,5764,5765],{},[932,5766,5767,5769],{},[51,5768,5356],{},[51,5770,2018],{},[54,5772,5773],{"encoding":56},"W_i",[31,5775,5777],{"className":5776,"ariaHidden":61},[60],[31,5778,5780,5784],{"className":5779},[65],[31,5781],{"className":5782,"style":5783},[69],"height:0.8333em;vertical-align:-0.15em;",[31,5785,5787,5790],{"className":5786},[74],[31,5788,5356],{"className":5789,"style":218},[74,75],[31,5791,5793],{"className":5792},[285],[31,5794,5796,5816],{"className":5795},[289,836],[31,5797,5799,5813],{"className":5798},[293],[31,5800,5802],{"className":5801,"style":2216},[297],[31,5803,5804,5807],{"style":5458},[31,5805],{"className":5806,"style":305},[304],[31,5808,5810],{"className":5809},[309,310,311,312],[31,5811,2018],{"className":5812},[74,75,312],[31,5814,871],{"className":5815},[870],[31,5817,5819],{"className":5818},[293],[31,5820,5822],{"className":5821,"style":1044},[297],[31,5823],{}," is the weight matrix of the ",[31,5826,5828,5841],{"className":5827,"translate":35},[34],[31,5829,5831],{"className":5830},[39],[41,5832,5833],{"xmlns":43},[45,5834,5835,5839],{},[48,5836,5837],{},[51,5838,2018],{},[54,5840,2018],{"encoding":56},[31,5842,5844],{"className":5843,"ariaHidden":61},[60],[31,5845,5847,5851],{"className":5846},[65],[31,5848],{"className":5849,"style":5850},[69],"height:0.6595em;",[31,5852,2018],{"className":5853},[74,75],"-th layer, and ",[31,5856,5858,5876],{"className":5857,"translate":35},[34],[31,5859,5861],{"className":5860},[39],[41,5862,5863],{"xmlns":43},[45,5864,5865,5873],{},[48,5866,5867],{},[932,5868,5869,5871],{},[51,5870,5365],{},[51,5872,2018],{},[54,5874,5875],{"encoding":56},"b_i",[31,5877,5879],{"className":5878,"ariaHidden":61},[60],[31,5880,5882,5886],{"className":5881},[65],[31,5883],{"className":5884,"style":5885},[69],"height:0.8444em;vertical-align:-0.15em;",[31,5887,5889,5892],{"className":5888},[74],[31,5890,5365],{"className":5891},[74,75],[31,5893,5895],{"className":5894},[285],[31,5896,5898,5918],{"className":5897},[289,836],[31,5899,5901,5915],{"className":5900},[293],[31,5902,5904],{"className":5903,"style":2216},[297],[31,5905,5906,5909],{"style":1025},[31,5907],{"className":5908,"style":305},[304],[31,5910,5912],{"className":5911},[309,310,311,312],[31,5913,2018],{"className":5914},[74,75,312],[31,5916,871],{"className":5917},[870],[31,5919,5921],{"className":5920},[293],[31,5922,5924],{"className":5923,"style":1044},[297],[31,5925],{}," is the bias vector.",[15,5928,5929,5930],{},"In other words: a neural network is a function family ",[31,5931,5933,5958],{"className":5932,"translate":35},[34],[31,5934,5936],{"className":5935},[39],[41,5937,5938],{"xmlns":43},[45,5939,5940,5955],{},[48,5941,5942,5949,5951,5953],{},[932,5943,5944,5947],{},[51,5945,5946],{},"f",[51,5948,1178],{},[98,5950,169],{"stretchy":168},[51,5952,936],{},[98,5954,185],{"stretchy":168},[54,5956,5957],{"encoding":56},"f_\\theta(x)",[31,5959,5961],{"className":5960,"ariaHidden":61},[60],[31,5962,5964,5967,6010,6013,6016],{"className":5963},[65],[31,5965],{"className":5966,"style":214},[69],[31,5968,5970,5974],{"className":5969},[74],[31,5971,5946],{"className":5972,"style":5973},[74,75],"margin-right:0.1076em;",[31,5975,5977],{"className":5976},[285],[31,5978,5980,6002],{"className":5979},[289,836],[31,5981,5983,5999],{"className":5982},[293],[31,5984,5987],{"className":5985,"style":5986},[297],"height:0.3361em;",[31,5988,5990,5993],{"style":5989},"top:-2.55em;margin-left:-0.1076em;margin-right:0.05em;",[31,5991],{"className":5992,"style":305},[304],[31,5994,5996],{"className":5995},[309,310,311,312],[31,5997,1178],{"className":5998,"style":226},[74,75,312],[31,6000,871],{"className":6001},[870],[31,6003,6005],{"className":6004},[293],[31,6006,6008],{"className":6007,"style":1044},[297],[31,6009],{},[31,6011,169],{"className":6012},[222],[31,6014,936],{"className":6015},[74,75],[31,6017,185],{"className":6018},[254],[15,6020,6021],{},"It defines the conditional probability distribution:",[31,6023,6025],{"className":6024,"translate":35},[149],[31,6026,6028,6057],{"className":6027,"translate":35},[34],[31,6029,6031],{"className":6030},[39],[41,6032,6033],{"xmlns":43,"display":158},[45,6034,6035,6054],{},[48,6036,6037,6043,6045,6048,6050,6052],{},[932,6038,6039,6041],{},[51,6040,165],{},[51,6042,1178],{},[98,6044,169],{"stretchy":168},[51,6046,6047],{},"y",[51,6049,176],{"mathvariant":175},[51,6051,936],{},[98,6053,185],{"stretchy":168},[54,6055,6056],{"encoding":56},"P_\\theta(y|x)",[31,6058,6060],{"className":6059,"ariaHidden":61},[60],[31,6061,6063,6066,6106,6109,6113,6116,6119],{"className":6062},[65],[31,6064],{"className":6065,"style":214},[69],[31,6067,6069,6072],{"className":6068},[74],[31,6070,165],{"className":6071,"style":218},[74,75],[31,6073,6075],{"className":6074},[285],[31,6076,6078,6098],{"className":6077},[289,836],[31,6079,6081,6095],{"className":6080},[293],[31,6082,6084],{"className":6083,"style":5986},[297],[31,6085,6086,6089],{"style":5458},[31,6087],{"className":6088,"style":305},[304],[31,6090,6092],{"className":6091},[309,310,311,312],[31,6093,1178],{"className":6094,"style":226},[74,75,312],[31,6096,871],{"className":6097},[870],[31,6099,6101],{"className":6100},[293],[31,6102,6104],{"className":6103,"style":1044},[297],[31,6105],{},[31,6107,169],{"className":6108},[222],[31,6110,6047],{"className":6111,"style":6112},[74,75],"margin-right:0.0359em;",[31,6114,176],{"className":6115},[74],[31,6117,936],{"className":6118},[74,75],[31,6120,185],{"className":6121},[254],[15,6123,6124,6127,6128,6156],{},[370,6125,6126],{},"Training a neural network"," = selecting parameters ",[31,6129,6131,6144],{"className":6130,"translate":35},[34],[31,6132,6134],{"className":6133},[39],[41,6135,6136],{"xmlns":43},[45,6137,6138,6142],{},[48,6139,6140],{},[51,6141,1178],{},[54,6143,1181],{"encoding":56},[31,6145,6147],{"className":6146,"ariaHidden":61},[60],[31,6148,6150,6153],{"className":6149},[65],[31,6151],{"className":6152,"style":843},[69],[31,6154,1178],{"className":6155,"style":226},[74,75]," that maximize the probability of the observed data:",[31,6158,6160],{"className":6159,"translate":35},[149],[31,6161,6163,6237],{"className":6162,"translate":35},[34],[31,6164,6166],{"className":6165},[39],[41,6167,6168],{"xmlns":43,"display":158},[45,6169,6170,6234],{},[48,6171,6172,6188,6190,6192,6194,6204,6210,6216,6218,6224,6226,6232],{},[932,6173,6174,6180],{},[805,6175,6176,6178],{"accent":61},[51,6177,1178],{},[98,6179,811],{},[48,6181,6182,6184,6186],{},[51,6183,2938],{"mathvariant":175},[51,6185,1218],{"mathvariant":175},[51,6187,2943],{"mathvariant":175},[98,6189,100],{},[51,6191,1327],{},[98,6193,1330],{},[1332,6195,6196,6202],{},[48,6197,6198,6200],{},[51,6199,1338],{},[98,6201,1330],{},[51,6203,1178],{},[1332,6205,6206,6208],{},[98,6207,2009],{},[51,6209,2018],{},[932,6211,6212,6214],{},[51,6213,165],{},[51,6215,1178],{},[98,6217,169],{"stretchy":168},[932,6219,6220,6222],{},[51,6221,6047],{},[51,6223,2018],{},[51,6225,176],{"mathvariant":175},[932,6227,6228,6230],{},[51,6229,936],{},[51,6231,2018],{},[98,6233,185],{"stretchy":168},[54,6235,6236],{"encoding":56},"\\hat{\\theta}_{\\rm MLE} = \\arg\\max_\\theta \\prod_i P_\\theta(y_i | x_i)",[31,6238,6240,6329],{"className":6239,"ariaHidden":61},[60],[31,6241,6243,6246,6320,6323,6326],{"className":6242},[65],[31,6244],{"className":6245,"style":2776},[69],[31,6247,6249,6280],{"className":6248},[74],[31,6250,6252],{"className":6251},[74,832],[31,6253,6255],{"className":6254},[289],[31,6256,6258],{"className":6257},[293],[31,6259,6261,6269],{"className":6260,"style":1391},[297],[31,6262,6263,6266],{"style":846},[31,6264],{"className":6265,"style":850},[304],[31,6267,1178],{"className":6268,"style":226},[74,75],[31,6270,6271,6274],{"style":1414},[31,6272],{"className":6273,"style":850},[304],[31,6275,6277],{"className":6276,"style":863},[862],[31,6278,811],{"className":6279},[74],[31,6281,6283],{"className":6282},[285],[31,6284,6286,6312],{"className":6285},[289,836],[31,6287,6289,6309],{"className":6288},[293],[31,6290,6292],{"className":6291,"style":2823},[297],[31,6293,6294,6297],{"style":2826},[31,6295],{"className":6296,"style":305},[304],[31,6298,6300],{"className":6299},[309,310,311,312],[31,6301,6303],{"className":6302},[74,312],[31,6304,6306],{"className":6305},[74,312],[31,6307,2763],{"className":6308},[74,3019,312],[31,6310,871],{"className":6311},[870],[31,6313,6315],{"className":6314},[293],[31,6316,6318],{"className":6317,"style":1044},[297],[31,6319],{},[31,6321],{"className":6322,"style":124},[123],[31,6324,100],{"className":6325},[128],[31,6327],{"className":6328,"style":124},[123],[31,6330,6332,6336,6341,6344,6388,6391,6435,6438,6478,6481,6522,6525,6565],{"className":6331},[65],[31,6333],{"className":6334,"style":6335},[69],"height:2.3277em;vertical-align:-1.2777em;",[31,6337,1444,6339],{"className":6338},[1443],[31,6340,1448],{"style":1447},[31,6342],{"className":6343,"style":1054},[123],[31,6345,6347],{"className":6346},[1443,1455],[31,6348,6350,6380],{"className":6349},[289,836],[31,6351,6353,6377],{"className":6352},[293],[31,6354,6356,6367],{"className":6355,"style":1465},[297],[31,6357,6358,6361],{"style":1468},[31,6359],{"className":6360,"style":850},[304],[31,6362,6364],{"className":6363},[309,310,311,312],[31,6365,1178],{"className":6366,"style":226},[74,75,312],[31,6368,6369,6372],{"style":846},[31,6370],{"className":6371,"style":850},[304],[31,6373,6374],{},[31,6375,1338],{"className":6376},[1443],[31,6378,871],{"className":6379},[870],[31,6381,6383],{"className":6382},[293],[31,6384,6386],{"className":6385,"style":1497},[297],[31,6387],{},[31,6389],{"className":6390,"style":1054},[123],[31,6392,6394],{"className":6393},[1443,1455],[31,6395,6397,6427],{"className":6396},[289,836],[31,6398,6400,6424],{"className":6399},[293],[31,6401,6403,6414],{"className":6402,"style":2098},[297],[31,6404,6405,6408],{"style":3882},[31,6406],{"className":6407,"style":2105},[304],[31,6409,6411],{"className":6410},[309,310,311,312],[31,6412,2018],{"className":6413},[74,75,312],[31,6415,6416,6419],{"style":2166},[31,6417],{"className":6418,"style":2105},[304],[31,6420,6421],{},[31,6422,2009],{"className":6423},[1443,2175,2176],[31,6425,871],{"className":6426},[870],[31,6428,6430],{"className":6429},[293],[31,6431,6433],{"className":6432,"style":3932},[297],[31,6434],{},[31,6436],{"className":6437,"style":1054},[123],[31,6439,6441,6444],{"className":6440},[74],[31,6442,165],{"className":6443,"style":218},[74,75],[31,6445,6447],{"className":6446},[285],[31,6448,6450,6470],{"className":6449},[289,836],[31,6451,6453,6467],{"className":6452},[293],[31,6454,6456],{"className":6455,"style":5986},[297],[31,6457,6458,6461],{"style":5458},[31,6459],{"className":6460,"style":305},[304],[31,6462,6464],{"className":6463},[309,310,311,312],[31,6465,1178],{"className":6466,"style":226},[74,75,312],[31,6468,871],{"className":6469},[870],[31,6471,6473],{"className":6472},[293],[31,6474,6476],{"className":6475,"style":1044},[297],[31,6477],{},[31,6479,169],{"className":6480},[222],[31,6482,6484,6487],{"className":6483},[74],[31,6485,6047],{"className":6486,"style":6112},[74,75],[31,6488,6490],{"className":6489},[285],[31,6491,6493,6514],{"className":6492},[289,836],[31,6494,6496,6511],{"className":6495},[293],[31,6497,6499],{"className":6498,"style":2216},[297],[31,6500,6502,6505],{"style":6501},"top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;",[31,6503],{"className":6504,"style":305},[304],[31,6506,6508],{"className":6507},[309,310,311,312],[31,6509,2018],{"className":6510},[74,75,312],[31,6512,871],{"className":6513},[870],[31,6515,6517],{"className":6516},[293],[31,6518,6520],{"className":6519,"style":1044},[297],[31,6521],{},[31,6523,176],{"className":6524},[74],[31,6526,6528,6531],{"className":6527},[74],[31,6529,936],{"className":6530},[74,75],[31,6532,6534],{"className":6533},[285],[31,6535,6537,6557],{"className":6536},[289,836],[31,6538,6540,6554],{"className":6539},[293],[31,6541,6543],{"className":6542,"style":2216},[297],[31,6544,6545,6548],{"style":1025},[31,6546],{"className":6547,"style":305},[304],[31,6549,6551],{"className":6550},[309,310,311,312],[31,6552,2018],{"className":6553},[74,75,312],[31,6555,871],{"className":6556},[870],[31,6558,6560],{"className":6559},[293],[31,6561,6563],{"className":6562,"style":1044},[297],[31,6564],{},[31,6566,185],{"className":6567},[254],[15,6569,6570],{},"Take classification as an example: a binary classification neural network outputs:",[31,6572,6574],{"className":6573,"translate":35},[149],[31,6575,6577,6616],{"className":6576,"translate":35},[34],[31,6578,6580],{"className":6579},[39],[41,6581,6582],{"xmlns":43,"display":158},[45,6583,6584,6613],{},[48,6585,6586,6592,6594,6597,6599,6605,6607,6609,6611],{},[805,6587,6588,6590],{"accent":61},[51,6589,6047],{},[98,6591,811],{},[98,6593,100],{},[51,6595,6596],{},"σ",[98,6598,169],{"stretchy":168},[932,6600,6601,6603],{},[51,6602,5946],{},[51,6604,1178],{},[98,6606,169],{"stretchy":168},[51,6608,936],{},[98,6610,185],{"stretchy":168},[98,6612,185],{"stretchy":168},[54,6614,6615],{"encoding":56},"\\hat{y} = \\sigma(f_\\theta(x))",[31,6617,6619,6677],{"className":6618,"ariaHidden":61},[60],[31,6620,6622,6625,6668,6671,6674],{"className":6621},[65],[31,6623],{"className":6624,"style":828},[69],[31,6626,6628],{"className":6627},[74,832],[31,6629,6631,6660],{"className":6630},[289,836],[31,6632,6634,6657],{"className":6633},[293],[31,6635,6637,6645],{"className":6636,"style":843},[297],[31,6638,6639,6642],{"style":846},[31,6640],{"className":6641,"style":850},[304],[31,6643,6047],{"className":6644,"style":6112},[74,75],[31,6646,6647,6650],{"style":846},[31,6648],{"className":6649,"style":850},[304],[31,6651,6654],{"className":6652,"style":6653},[862],"left:-0.1944em;",[31,6655,811],{"className":6656},[74],[31,6658,871],{"className":6659},[870],[31,6661,6663],{"className":6662},[293],[31,6664,6666],{"className":6665,"style":878},[297],[31,6667],{},[31,6669],{"className":6670,"style":124},[123],[31,6672,100],{"className":6673},[128],[31,6675],{"className":6676,"style":124},[123],[31,6678,6680,6683,6686,6689,6729,6732,6735],{"className":6679},[65],[31,6681],{"className":6682,"style":214},[69],[31,6684,6596],{"className":6685,"style":6112},[74,75],[31,6687,169],{"className":6688},[222],[31,6690,6692,6695],{"className":6691},[74],[31,6693,5946],{"className":6694,"style":5973},[74,75],[31,6696,6698],{"className":6697},[285],[31,6699,6701,6721],{"className":6700},[289,836],[31,6702,6704,6718],{"className":6703},[293],[31,6705,6707],{"className":6706,"style":5986},[297],[31,6708,6709,6712],{"style":5989},[31,6710],{"className":6711,"style":305},[304],[31,6713,6715],{"className":6714},[309,310,311,312],[31,6716,1178],{"className":6717,"style":226},[74,75,312],[31,6719,871],{"className":6720},[870],[31,6722,6724],{"className":6723},[293],[31,6725,6727],{"className":6726,"style":1044},[297],[31,6728],{},[31,6730,169],{"className":6731},[222],[31,6733,936],{"className":6734},[74,75],[31,6736,6738],{"className":6737},[254],"))",[15,6740,6741],{},"We see that maximizing the likelihood can be written as minimizing the cross-entropy loss function as the cost function:",[31,6743,6745],{"className":6744,"translate":35},[149],[31,6746,6748,6847],{"className":6747,"translate":35},[34],[31,6749,6751],{"className":6750},[39],[41,6752,6753],{"xmlns":43,"display":158},[45,6754,6755,6844],{},[48,6756,6757,6773,6775,6777,6779,6789,6795,6812,6814,6816,6818,6828],{},[932,6758,6759,6765],{},[805,6760,6761,6763],{"accent":61},[51,6762,1178],{},[98,6764,811],{},[48,6766,6767,6769,6771],{},[51,6768,2938],{"mathvariant":175},[51,6770,1218],{"mathvariant":175},[51,6772,2943],{"mathvariant":175},[98,6774,100],{},[51,6776,1327],{},[98,6778,1330],{},[1332,6780,6781,6787],{},[48,6782,6783,6785],{},[51,6784,1338],{},[98,6786,1330],{},[51,6788,1178],{},[1332,6790,6791,6793],{},[98,6792,2009],{},[51,6794,2018],{},[6796,6797,6798,6804,6806],"msubsup",{},[805,6799,6800,6802],{"accent":61},[51,6801,6047],{},[98,6803,811],{},[51,6805,2018],{},[932,6807,6808,6810],{},[51,6809,6047],{},[51,6811,2018],{},[98,6813,169],{"stretchy":168},[102,6815,939],{},[98,6817,3603],{},[932,6819,6820,6826],{},[805,6821,6822,6824],{"accent":61},[51,6823,6047],{},[98,6825,811],{},[51,6827,2018],{},[189,6829,6830,6832],{},[98,6831,185],{"stretchy":168},[48,6833,6834,6836,6838],{},[102,6835,939],{},[98,6837,3603],{},[932,6839,6840,6842],{},[51,6841,6047],{},[51,6843,2018],{},[54,6845,6846],{"encoding":56},"\\hat{\\theta}_{\\rm MLE} = \\arg\\max_\\theta \\prod_i \\hat{y}_i^{y_i} (1-\\hat{y}_i)^{1-y_i}",[31,6848,6850,6939,7197],{"className":6849,"ariaHidden":61},[60],[31,6851,6853,6856,6930,6933,6936],{"className":6852},[65],[31,6854],{"className":6855,"style":2776},[69],[31,6857,6859,6890],{"className":6858},[74],[31,6860,6862],{"className":6861},[74,832],[31,6863,6865],{"className":6864},[289],[31,6866,6868],{"className":6867},[293],[31,6869,6871,6879],{"className":6870,"style":1391},[297],[31,6872,6873,6876],{"style":846},[31,6874],{"className":6875,"style":850},[304],[31,6877,1178],{"className":6878,"style":226},[74,75],[31,6880,6881,6884],{"style":1414},[31,6882],{"className":6883,"style":850},[304],[31,6885,6887],{"className":6886,"style":863},[862],[31,6888,811],{"className":6889},[74],[31,6891,6893],{"className":6892},[285],[31,6894,6896,6922],{"className":6895},[289,836],[31,6897,6899,6919],{"className":6898},[293],[31,6900,6902],{"className":6901,"style":2823},[297],[31,6903,6904,6907],{"style":2826},[31,6905],{"className":6906,"style":305},[304],[31,6908,6910],{"className":6909},[309,310,311,312],[31,6911,6913],{"className":6912},[74,312],[31,6914,6916],{"className":6915},[74,312],[31,6917,2763],{"className":6918},[74,3019,312],[31,6920,871],{"className":6921},[870],[31,6923,6925],{"className":6924},[293],[31,6926,6928],{"className":6927,"style":1044},[297],[31,6929],{},[31,6931],{"className":6932,"style":124},[123],[31,6934,100],{"className":6935},[128],[31,6937],{"className":6938,"style":124},[123],[31,6940,6942,6945,6950,6953,6997,7000,7044,7047,7182,7185,7188,7191,7194],{"className":6941},[65],[31,6943],{"className":6944,"style":6335},[69],[31,6946,1444,6948],{"className":6947},[1443],[31,6949,1448],{"style":1447},[31,6951],{"className":6952,"style":1054},[123],[31,6954,6956],{"className":6955},[1443,1455],[31,6957,6959,6989],{"className":6958},[289,836],[31,6960,6962,6986],{"className":6961},[293],[31,6963,6965,6976],{"className":6964,"style":1465},[297],[31,6966,6967,6970],{"style":1468},[31,6968],{"className":6969,"style":850},[304],[31,6971,6973],{"className":6972},[309,310,311,312],[31,6974,1178],{"className":6975,"style":226},[74,75,312],[31,6977,6978,6981],{"style":846},[31,6979],{"className":6980,"style":850},[304],[31,6982,6983],{},[31,6984,1338],{"className":6985},[1443],[31,6987,871],{"className":6988},[870],[31,6990,6992],{"className":6991},[293],[31,6993,6995],{"className":6994,"style":1497},[297],[31,6996],{},[31,6998],{"className":6999,"style":1054},[123],[31,7001,7003],{"className":7002},[1443,1455],[31,7004,7006,7036],{"className":7005},[289,836],[31,7007,7009,7033],{"className":7008},[293],[31,7010,7012,7023],{"className":7011,"style":2098},[297],[31,7013,7014,7017],{"style":3882},[31,7015],{"className":7016,"style":2105},[304],[31,7018,7020],{"className":7019},[309,310,311,312],[31,7021,2018],{"className":7022},[74,75,312],[31,7024,7025,7028],{"style":2166},[31,7026],{"className":7027,"style":2105},[304],[31,7029,7030],{},[31,7031,2009],{"className":7032},[1443,2175,2176],[31,7034,871],{"className":7035},[870],[31,7037,7039],{"className":7038},[293],[31,7040,7042],{"className":7041,"style":3932},[297],[31,7043],{},[31,7045],{"className":7046,"style":1054},[123],[31,7048,7050,7092],{"className":7049},[74],[31,7051,7053],{"className":7052},[74,832],[31,7054,7056,7084],{"className":7055},[289,836],[31,7057,7059,7081],{"className":7058},[293],[31,7060,7062,7070],{"className":7061,"style":843},[297],[31,7063,7064,7067],{"style":846},[31,7065],{"className":7066,"style":850},[304],[31,7068,6047],{"className":7069,"style":6112},[74,75],[31,7071,7072,7075],{"style":846},[31,7073],{"className":7074,"style":850},[304],[31,7076,7078],{"className":7077,"style":6653},[862],[31,7079,811],{"className":7080},[74],[31,7082,871],{"className":7083},[870],[31,7085,7087],{"className":7086},[293],[31,7088,7090],{"className":7089,"style":878},[297],[31,7091],{},[31,7093,7095],{"className":7094},[285],[31,7096,7098,7173],{"className":7097},[289,836],[31,7099,7101,7170],{"className":7100},[293],[31,7102,7105,7117],{"className":7103,"style":7104},[297],"height:0.7823em;",[31,7106,7108,7111],{"style":7107},"top:-2.4231em;margin-left:-0.0359em;margin-right:0.05em;",[31,7109],{"className":7110,"style":305},[304],[31,7112,7114],{"className":7113},[309,310,311,312],[31,7115,2018],{"className":7116},[74,75,312],[31,7118,7120,7123],{"style":7119},"top:-3.1809em;margin-right:0.05em;",[31,7121],{"className":7122,"style":305},[304],[31,7124,7126],{"className":7125},[309,310,311,312],[31,7127,7129],{"className":7128},[74,312],[31,7130,7132,7135],{"className":7131},[74,312],[31,7133,6047],{"className":7134,"style":6112},[74,75,312],[31,7136,7138],{"className":7137},[285],[31,7139,7141,7162],{"className":7140},[289,836],[31,7142,7144,7159],{"className":7143},[293],[31,7145,7147],{"className":7146,"style":2130},[297],[31,7148,7150,7153],{"style":7149},"top:-2.357em;margin-left:-0.0359em;margin-right:0.0714em;",[31,7151],{"className":7152,"style":2137},[304],[31,7154,7156],{"className":7155},[309,2141,2142,312],[31,7157,2018],{"className":7158},[74,75,312],[31,7160,871],{"className":7161},[870],[31,7163,7165],{"className":7164},[293],[31,7166,7168],{"className":7167,"style":2155},[297],[31,7169],{},[31,7171,871],{"className":7172},[870],[31,7174,7176],{"className":7175},[293],[31,7177,7180],{"className":7178,"style":7179},[297],"height:0.2769em;",[31,7181],{},[31,7183,169],{"className":7184},[222],[31,7186,939],{"className":7187},[74],[31,7189],{"className":7190,"style":584},[123],[31,7192,3603],{"className":7193},[588],[31,7195],{"className":7196,"style":584},[123],[31,7198,7200,7203,7282],{"className":7199},[65],[31,7201],{"className":7202,"style":4029},[69],[31,7204,7206,7248],{"className":7205},[74],[31,7207,7209],{"className":7208},[74,832],[31,7210,7212,7240],{"className":7211},[289,836],[31,7213,7215,7237],{"className":7214},[293],[31,7216,7218,7226],{"className":7217,"style":843},[297],[31,7219,7220,7223],{"style":846},[31,7221],{"className":7222,"style":850},[304],[31,7224,6047],{"className":7225,"style":6112},[74,75],[31,7227,7228,7231],{"style":846},[31,7229],{"className":7230,"style":850},[304],[31,7232,7234],{"className":7233,"style":6653},[862],[31,7235,811],{"className":7236},[74],[31,7238,871],{"className":7239},[870],[31,7241,7243],{"className":7242},[293],[31,7244,7246],{"className":7245,"style":878},[297],[31,7247],{},[31,7249,7251],{"className":7250},[285],[31,7252,7254,7274],{"className":7253},[289,836],[31,7255,7257,7271],{"className":7256},[293],[31,7258,7260],{"className":7259,"style":2216},[297],[31,7261,7262,7265],{"style":6501},[31,7263],{"className":7264,"style":305},[304],[31,7266,7268],{"className":7267},[309,310,311,312],[31,7269,2018],{"className":7270},[74,75,312],[31,7272,871],{"className":7273},[870],[31,7275,7277],{"className":7276},[293],[31,7278,7280],{"className":7279,"style":1044},[297],[31,7281],{},[31,7283,7285,7288],{"className":7284},[254],[31,7286,185],{"className":7287},[254],[31,7289,7291],{"className":7290},[285],[31,7292,7294],{"className":7293},[289],[31,7295,7297],{"className":7296},[293],[31,7298,7300],{"className":7299,"style":270},[297],[31,7301,7302,7305],{"style":300},[31,7303],{"className":7304,"style":305},[304],[31,7306,7308],{"className":7307},[309,310,311,312],[31,7309,7311,7314,7317],{"className":7310},[74,312],[31,7312,939],{"className":7313},[74,312],[31,7315,3603],{"className":7316},[588,312],[31,7318,7320,7323],{"className":7319},[74,312],[31,7321,6047],{"className":7322,"style":6112},[74,75,312],[31,7324,7326],{"className":7325},[285],[31,7327,7329,7349],{"className":7328},[289,836],[31,7330,7332,7346],{"className":7331},[293],[31,7333,7335],{"className":7334,"style":2130},[297],[31,7336,7337,7340],{"style":7149},[31,7338],{"className":7339,"style":2137},[304],[31,7341,7343],{"className":7342},[309,2141,2142,312],[31,7344,2018],{"className":7345},[74,75,312],[31,7347,871],{"className":7348},[870],[31,7350,7352],{"className":7351},[293],[31,7353,7355],{"className":7354,"style":2155},[297],[31,7356],{},[15,7358,7359],{},"Finally, we obtain the cross-entropy loss:",[31,7361,7363],{"className":7362,"translate":35},[149],[31,7364,7366,7461],{"className":7365,"translate":35},[34],[31,7367,7369],{"className":7368},[39],[41,7370,7371],{"xmlns":43,"display":158},[45,7372,7373,7458],{},[48,7374,7375,7377,7379,7381,7383,7385,7387,7393,7397,7403,7405,7407,7417,7419,7421,7423,7425,7431,7433,7435,7437,7439,7441,7443,7453,7455],{},[51,7376,1218],{},[98,7378,169],{"stretchy":168},[51,7380,1178],{},[98,7382,185],{"stretchy":168},[98,7384,100],{},[98,7386,3603],{},[1332,7388,7389,7391],{},[98,7390,2336],{},[51,7392,2018],{},[98,7394,7396],{"fence":168,"stretchy":61,"minsize":7395,"maxsize":7395},"1.8em","[",[932,7398,7399,7401],{},[51,7400,6047],{},[51,7402,2018],{},[51,7404,2315],{},[98,7406,1330],{},[932,7408,7409,7415],{},[805,7410,7411,7413],{"accent":61},[51,7412,6047],{},[98,7414,811],{},[51,7416,2018],{},[98,7418,4168],{},[98,7420,169],{"stretchy":168},[102,7422,939],{},[98,7424,3603],{},[932,7426,7427,7429],{},[51,7428,6047],{},[51,7430,2018],{},[98,7432,185],{"stretchy":168},[51,7434,2315],{},[98,7436,1330],{},[98,7438,169],{"stretchy":168},[102,7440,939],{},[98,7442,3603],{},[932,7444,7445,7451],{},[805,7446,7447,7449],{"accent":61},[51,7448,6047],{},[98,7450,811],{},[51,7452,2018],{},[98,7454,185],{"stretchy":168},[98,7456,7457],{"fence":168,"stretchy":61,"minsize":7395,"maxsize":7395},"]",[54,7459,7460],{"encoding":56},"L(\\theta) = - \\sum_i \\Big[ y_i \\log \\hat{y}_i + (1-y_i) \\log (1-\\hat{y}_i) \\Big]",[31,7462,7464,7491,7698,7719,7791],{"className":7463,"ariaHidden":61},[60],[31,7465,7467,7470,7473,7476,7479,7482,7485,7488],{"className":7466},[65],[31,7468],{"className":7469,"style":214},[69],[31,7471,1218],{"className":7472},[74,75],[31,7474,169],{"className":7475},[222],[31,7477,1178],{"className":7478,"style":226},[74,75],[31,7480,185],{"className":7481},[254],[31,7483],{"className":7484,"style":124},[123],[31,7486,100],{"className":7487},[128],[31,7489],{"className":7490,"style":124},[123],[31,7492,7494,7498,7501,7504,7548,7551,7559,7599,7602,7607,7610,7689,7692,7695],{"className":7493},[65],[31,7495],{"className":7496,"style":7497},[69],"height:2.4277em;vertical-align:-1.2777em;",[31,7499,3603],{"className":7500},[74],[31,7502],{"className":7503,"style":1054},[123],[31,7505,7507],{"className":7506},[1443,1455],[31,7508,7510,7540],{"className":7509},[289,836],[31,7511,7513,7537],{"className":7512},[293],[31,7514,7516,7527],{"className":7515,"style":2098},[297],[31,7517,7518,7521],{"style":3882},[31,7519],{"className":7520,"style":2105},[304],[31,7522,7524],{"className":7523},[309,310,311,312],[31,7525,2018],{"className":7526},[74,75,312],[31,7528,7529,7532],{"style":2166},[31,7530],{"className":7531,"style":2105},[304],[31,7533,7534],{},[31,7535,2336],{"className":7536},[1443,2175,2176],[31,7538,871],{"className":7539},[870],[31,7541,7543],{"className":7542},[293],[31,7544,7546],{"className":7545,"style":3932},[297],[31,7547],{},[31,7549],{"className":7550,"style":1054},[123],[31,7552,7554],{"className":7553},[74],[31,7555,7396],{"className":7556},[7557,7558],"delimsizing","size2",[31,7560,7562,7565],{"className":7561},[74],[31,7563,6047],{"className":7564,"style":6112},[74,75],[31,7566,7568],{"className":7567},[285],[31,7569,7571,7591],{"className":7570},[289,836],[31,7572,7574,7588],{"className":7573},[293],[31,7575,7577],{"className":7576,"style":2216},[297],[31,7578,7579,7582],{"style":6501},[31,7580],{"className":7581,"style":305},[304],[31,7583,7585],{"className":7584},[309,310,311,312],[31,7586,2018],{"className":7587},[74,75,312],[31,7589,871],{"className":7590},[870],[31,7592,7594],{"className":7593},[293],[31,7595,7597],{"className":7596,"style":1044},[297],[31,7598],{},[31,7600],{"className":7601,"style":1054},[123],[31,7603,2417,7605],{"className":7604},[1443],[31,7606,1448],{"style":1447},[31,7608],{"className":7609,"style":1054},[123],[31,7611,7613,7655],{"className":7612},[74],[31,7614,7616],{"className":7615},[74,832],[31,7617,7619,7647],{"className":7618},[289,836],[31,7620,7622,7644],{"className":7621},[293],[31,7623,7625,7633],{"className":7624,"style":843},[297],[31,7626,7627,7630],{"style":846},[31,7628],{"className":7629,"style":850},[304],[31,7631,6047],{"className":7632,"style":6112},[74,75],[31,7634,7635,7638],{"style":846},[31,7636],{"className":7637,"style":850},[304],[31,7639,7641],{"className":7640,"style":6653},[862],[31,7642,811],{"className":7643},[74],[31,7645,871],{"className":7646},[870],[31,7648,7650],{"className":7649},[293],[31,7651,7653],{"className":7652,"style":878},[297],[31,7654],{},[31,7656,7658],{"className":7657},[285],[31,7659,7661,7681],{"className":7660},[289,836],[31,7662,7664,7678],{"className":7663},[293],[31,7665,7667],{"className":7666,"style":2216},[297],[31,7668,7669,7672],{"style":6501},[31,7670],{"className":7671,"style":305},[304],[31,7673,7675],{"className":7674},[309,310,311,312],[31,7676,2018],{"className":7677},[74,75,312],[31,7679,871],{"className":7680},[870],[31,7682,7684],{"className":7683},[293],[31,7685,7687],{"className":7686,"style":1044},[297],[31,7688],{},[31,7690],{"className":7691,"style":584},[123],[31,7693,4168],{"className":7694},[588],[31,7696],{"className":7697,"style":584},[123],[31,7699,7701,7704,7707,7710,7713,7716],{"className":7700},[65],[31,7702],{"className":7703,"style":214},[69],[31,7705,169],{"className":7706},[222],[31,7708,939],{"className":7709},[74],[31,7711],{"className":7712,"style":584},[123],[31,7714,3603],{"className":7715},[588],[31,7717],{"className":7718,"style":584},[123],[31,7720,7722,7725,7765,7768,7771,7776,7779,7782,7785,7788],{"className":7721},[65],[31,7723],{"className":7724,"style":214},[69],[31,7726,7728,7731],{"className":7727},[74],[31,7729,6047],{"className":7730,"style":6112},[74,75],[31,7732,7734],{"className":7733},[285],[31,7735,7737,7757],{"className":7736},[289,836],[31,7738,7740,7754],{"className":7739},[293],[31,7741,7743],{"className":7742,"style":2216},[297],[31,7744,7745,7748],{"style":6501},[31,7746],{"className":7747,"style":305},[304],[31,7749,7751],{"className":7750},[309,310,311,312],[31,7752,2018],{"className":7753},[74,75,312],[31,7755,871],{"className":7756},[870],[31,7758,7760],{"className":7759},[293],[31,7761,7763],{"className":7762,"style":1044},[297],[31,7764],{},[31,7766,185],{"className":7767},[254],[31,7769],{"className":7770,"style":1054},[123],[31,7772,2417,7774],{"className":7773},[1443],[31,7775,1448],{"style":1447},[31,7777,169],{"className":7778},[222],[31,7780,939],{"className":7781},[74],[31,7783],{"className":7784,"style":584},[123],[31,7786,3603],{"className":7787},[588],[31,7789],{"className":7790,"style":584},[123],[31,7792,7794,7798,7877,7880],{"className":7793},[65],[31,7795],{"className":7796,"style":7797},[69],"height:1.8em;vertical-align:-0.65em;",[31,7799,7801,7843],{"className":7800},[74],[31,7802,7804],{"className":7803},[74,832],[31,7805,7807,7835],{"className":7806},[289,836],[31,7808,7810,7832],{"className":7809},[293],[31,7811,7813,7821],{"className":7812,"style":843},[297],[31,7814,7815,7818],{"style":846},[31,7816],{"className":7817,"style":850},[304],[31,7819,6047],{"className":7820,"style":6112},[74,75],[31,7822,7823,7826],{"style":846},[31,7824],{"className":7825,"style":850},[304],[31,7827,7829],{"className":7828,"style":6653},[862],[31,7830,811],{"className":7831},[74],[31,7833,871],{"className":7834},[870],[31,7836,7838],{"className":7837},[293],[31,7839,7841],{"className":7840,"style":878},[297],[31,7842],{},[31,7844,7846],{"className":7845},[285],[31,7847,7849,7869],{"className":7848},[289,836],[31,7850,7852,7866],{"className":7851},[293],[31,7853,7855],{"className":7854,"style":2216},[297],[31,7856,7857,7860],{"style":6501},[31,7858],{"className":7859,"style":305},[304],[31,7861,7863],{"className":7862},[309,310,311,312],[31,7864,2018],{"className":7865},[74,75,312],[31,7867,871],{"className":7868},[870],[31,7870,7872],{"className":7871},[293],[31,7873,7875],{"className":7874,"style":1044},[297],[31,7876],{},[31,7878,185],{"className":7879},[254],[31,7881,7883],{"className":7882},[74],[31,7884,7457],{"className":7885},[7557,7558],[15,7887,7888,7889,7917],{},"Here ",[31,7890,7892,7905],{"className":7891,"translate":35},[34],[31,7893,7895],{"className":7894},[39],[41,7896,7897],{"xmlns":43},[45,7898,7899,7903],{},[48,7900,7901],{},[51,7902,1178],{},[54,7904,1181],{"encoding":56},[31,7906,7908],{"className":7907,"ariaHidden":61},[60],[31,7909,7911,7914],{"className":7910},[65],[31,7912],{"className":7913,"style":843},[69],[31,7915,1178],{"className":7916,"style":226},[74,75]," represents all weights and biases.",[15,7919,7920],{},"Take regression as another example: suppose",[31,7922,7924],{"className":7923,"translate":35},[149],[31,7925,7927,8003],{"className":7926,"translate":35},[34],[31,7928,7930],{"className":7929},[39],[41,7931,7932],{"xmlns":43,"display":158},[45,7933,7934,8000],{},[48,7935,7936,7942,7944,7950,7952,7958,7960,7962,7969,7971,7974,7980,7983,7986,7988,7990,7992,7998],{},[932,7937,7938,7940],{},[51,7939,6047],{},[51,7941,2018],{},[98,7943,100],{},[932,7945,7946,7948],{},[51,7947,5946],{},[51,7949,1178],{},[98,7951,169],{"stretchy":168},[932,7953,7954,7956],{},[51,7955,936],{},[51,7957,2018],{},[98,7959,185],{"stretchy":168},[98,7961,4168],{},[932,7963,7964,7967],{},[51,7965,7966],{},"ϵ",[51,7968,2018],{},[98,7970,942],{"separator":61},[123,7972],{"width":7973},"1em",[932,7975,7976,7978],{},[51,7977,7966],{},[51,7979,2018],{},[98,7981,7982],{},"∼",[51,7984,7985],{},"N",[98,7987,169],{"stretchy":168},[102,7989,3077],{},[98,7991,942],{"separator":61},[189,7993,7994,7996],{},[51,7995,6596],{},[102,7997,479],{},[98,7999,185],{"stretchy":168},[54,8001,8002],{"encoding":56},"y_i = f_\\theta(x_i) + \\epsilon_i, \\quad \\epsilon_i \\sim N(0, \\sigma^2)",[31,8004,8006,8061,8162,8267],{"className":8005,"ariaHidden":61},[60],[31,8007,8009,8012,8052,8055,8058],{"className":8008},[65],[31,8010],{"className":8011,"style":70},[69],[31,8013,8015,8018],{"className":8014},[74],[31,8016,6047],{"className":8017,"style":6112},[74,75],[31,8019,8021],{"className":8020},[285],[31,8022,8024,8044],{"className":8023},[289,836],[31,8025,8027,8041],{"className":8026},[293],[31,8028,8030],{"className":8029,"style":2216},[297],[31,8031,8032,8035],{"style":6501},[31,8033],{"className":8034,"style":305},[304],[31,8036,8038],{"className":8037},[309,310,311,312],[31,8039,2018],{"className":8040},[74,75,312],[31,8042,871],{"className":8043},[870],[31,8045,8047],{"className":8046},[293],[31,8048,8050],{"className":8049,"style":1044},[297],[31,8051],{},[31,8053],{"className":8054,"style":124},[123],[31,8056,100],{"className":8057},[128],[31,8059],{"className":8060,"style":124},[123],[31,8062,8064,8067,8107,8110,8150,8153,8156,8159],{"className":8063},[65],[31,8065],{"className":8066,"style":214},[69],[31,8068,8070,8073],{"className":8069},[74],[31,8071,5946],{"className":8072,"style":5973},[74,75],[31,8074,8076],{"className":8075},[285],[31,8077,8079,8099],{"className":8078},[289,836],[31,8080,8082,8096],{"className":8081},[293],[31,8083,8085],{"className":8084,"style":5986},[297],[31,8086,8087,8090],{"style":5989},[31,8088],{"className":8089,"style":305},[304],[31,8091,8093],{"className":8092},[309,310,311,312],[31,8094,1178],{"className":8095,"style":226},[74,75,312],[31,8097,871],{"className":8098},[870],[31,8100,8102],{"className":8101},[293],[31,8103,8105],{"className":8104,"style":1044},[297],[31,8106],{},[31,8108,169],{"className":8109},[222],[31,8111,8113,8116],{"className":8112},[74],[31,8114,936],{"className":8115},[74,75],[31,8117,8119],{"className":8118},[285],[31,8120,8122,8142],{"className":8121},[289,836],[31,8123,8125,8139],{"className":8124},[293],[31,8126,8128],{"className":8127,"style":2216},[297],[31,8129,8130,8133],{"style":1025},[31,8131],{"className":8132,"style":305},[304],[31,8134,8136],{"className":8135},[309,310,311,312],[31,8137,2018],{"className":8138},[74,75,312],[31,8140,871],{"className":8141},[870],[31,8143,8145],{"className":8144},[293],[31,8146,8148],{"className":8147,"style":1044},[297],[31,8149],{},[31,8151,185],{"className":8152},[254],[31,8154],{"className":8155,"style":584},[123],[31,8157,4168],{"className":8158},[588],[31,8160],{"className":8161,"style":584},[123],[31,8163,8165,8168,8208,8211,8215,8218,8258,8261,8264],{"className":8164},[65],[31,8166],{"className":8167,"style":70},[69],[31,8169,8171,8174],{"className":8170},[74],[31,8172,7966],{"className":8173},[74,75],[31,8175,8177],{"className":8176},[285],[31,8178,8180,8200],{"className":8179},[289,836],[31,8181,8183,8197],{"className":8182},[293],[31,8184,8186],{"className":8185,"style":2216},[297],[31,8187,8188,8191],{"style":1025},[31,8189],{"className":8190,"style":305},[304],[31,8192,8194],{"className":8193},[309,310,311,312],[31,8195,2018],{"className":8196},[74,75,312],[31,8198,871],{"className":8199},[870],[31,8201,8203],{"className":8202},[293],[31,8204,8206],{"className":8205,"style":1044},[297],[31,8207],{},[31,8209,942],{"className":8210},[1050],[31,8212],{"className":8213,"style":8214},[123],"margin-right:1em;",[31,8216],{"className":8217,"style":1054},[123],[31,8219,8221,8224],{"className":8220},[74],[31,8222,7966],{"className":8223},[74,75],[31,8225,8227],{"className":8226},[285],[31,8228,8230,8250],{"className":8229},[289,836],[31,8231,8233,8247],{"className":8232},[293],[31,8234,8236],{"className":8235,"style":2216},[297],[31,8237,8238,8241],{"style":1025},[31,8239],{"className":8240,"style":305},[304],[31,8242,8244],{"className":8243},[309,310,311,312],[31,8245,2018],{"className":8246},[74,75,312],[31,8248,871],{"className":8249},[870],[31,8251,8253],{"className":8252},[293],[31,8254,8256],{"className":8255,"style":1044},[297],[31,8257],{},[31,8259],{"className":8260,"style":124},[123],[31,8262,7982],{"className":8263},[128],[31,8265],{"className":8266,"style":124},[123],[31,8268,8270,8273,8277,8280,8283,8286,8289,8318],{"className":8269},[65],[31,8271],{"className":8272,"style":4029},[69],[31,8274,7985],{"className":8275,"style":8276},[74,75],"margin-right:0.109em;",[31,8278,169],{"className":8279},[222],[31,8281,3077],{"className":8282},[74],[31,8284,942],{"className":8285},[1050],[31,8287],{"className":8288,"style":1054},[123],[31,8290,8292,8295],{"className":8291},[74],[31,8293,6596],{"className":8294,"style":6112},[74,75],[31,8296,8298],{"className":8297},[285],[31,8299,8301],{"className":8300},[289],[31,8302,8304],{"className":8303},[293],[31,8305,8307],{"className":8306,"style":270},[297],[31,8308,8309,8312],{"style":300},[31,8310],{"className":8311,"style":305},[304],[31,8313,8315],{"className":8314},[309,310,311,312],[31,8316,479],{"className":8317},[74,312],[31,8319,185],{"className":8320},[254],[15,8322,8323],{},"Log-likelihood:",[31,8325,8327],{"className":8326,"translate":35},[149],[31,8328,8330,8411],{"className":8329,"translate":35},[34],[31,8331,8333],{"className":8332},[39],[41,8334,8335],{"xmlns":43,"display":158},[45,8336,8337,8408],{},[48,8338,8339,8341,8343,8345,8347,8349,8351,8365,8371,8373,8379,8381,8387,8389,8395,8397,8403,8405],{},[51,8340,2300],{"mathvariant":175},[98,8342,169],{"stretchy":168},[51,8344,1178],{},[98,8346,185],{"stretchy":168},[98,8348,100],{},[98,8350,3603],{},[3048,8352,8353,8355],{},[102,8354,939],{},[48,8356,8357,8359],{},[102,8358,479],{},[189,8360,8361,8363],{},[51,8362,6596],{},[102,8364,479],{},[1332,8366,8367,8369],{},[98,8368,2336],{},[51,8370,2018],{},[98,8372,169],{"stretchy":168},[932,8374,8375,8377],{},[51,8376,6047],{},[51,8378,2018],{},[98,8380,3603],{},[932,8382,8383,8385],{},[51,8384,5946],{},[51,8386,1178],{},[98,8388,169],{"stretchy":168},[932,8390,8391,8393],{},[51,8392,936],{},[51,8394,2018],{},[98,8396,185],{"stretchy":168},[189,8398,8399,8401],{},[98,8400,185],{"stretchy":168},[102,8402,479],{},[98,8404,4168],{},[51,8406,8407],{},"C",[54,8409,8410],{"encoding":56},"\\ell(\\theta) = -\\frac{1}{2\\sigma^2} \\sum_i (y_i - f_\\theta(x_i))^2 + C",[31,8412,8414,8441,8642,8772],{"className":8413,"ariaHidden":61},[60],[31,8415,8417,8420,8423,8426,8429,8432,8435,8438],{"className":8416},[65],[31,8418],{"className":8419,"style":214},[69],[31,8421,2300],{"className":8422},[74],[31,8424,169],{"className":8425},[222],[31,8427,1178],{"className":8428,"style":226},[74,75],[31,8430,185],{"className":8431},[254],[31,8433],{"className":8434,"style":124},[123],[31,8436,100],{"className":8437},[128],[31,8439],{"className":8440,"style":124},[123],[31,8442,8444,8448,8451,8543,8546,8590,8593,8633,8636,8639],{"className":8443},[65],[31,8445],{"className":8446,"style":8447},[69],"height:2.5991em;vertical-align:-1.2777em;",[31,8449,3603],{"className":8450},[74],[31,8452,8454,8457,8540],{"className":8453},[74],[31,8455],{"className":8456},[222,3097],[31,8458,8460],{"className":8459},[3048],[31,8461,8463,8532],{"className":8462},[289,836],[31,8464,8466,8529],{"className":8465},[293],[31,8467,8470,8510,8518],{"className":8468,"style":8469},[297],"height:1.3214em;",[31,8471,8472,8475],{"style":3113},[31,8473],{"className":8474,"style":850},[304],[31,8476,8478,8481],{"className":8477},[74],[31,8479,479],{"className":8480},[74],[31,8482,8484,8487],{"className":8483},[74],[31,8485,6596],{"className":8486,"style":6112},[74,75],[31,8488,8490],{"className":8489},[285],[31,8491,8493],{"className":8492},[289],[31,8494,8496],{"className":8495},[293],[31,8497,8499],{"className":8498,"style":3317},[297],[31,8500,8501,8504],{"style":3320},[31,8502],{"className":8503,"style":305},[304],[31,8505,8507],{"className":8506},[309,310,311,312],[31,8508,479],{"className":8509},[74,312],[31,8511,8512,8515],{"style":3129},[31,8513],{"className":8514,"style":850},[304],[31,8516],{"className":8517,"style":3137},[3136],[31,8519,8520,8523],{"style":3140},[31,8521],{"className":8522,"style":850},[304],[31,8524,8526],{"className":8525},[74],[31,8527,939],{"className":8528},[74],[31,8530,871],{"className":8531},[870],[31,8533,8535],{"className":8534},[293],[31,8536,8538],{"className":8537,"style":3177},[297],[31,8539],{},[31,8541],{"className":8542},[254,3097],[31,8544],{"className":8545,"style":1054},[123],[31,8547,8549],{"className":8548},[1443,1455],[31,8550,8552,8582],{"className":8551},[289,836],[31,8553,8555,8579],{"className":8554},[293],[31,8556,8558,8569],{"className":8557,"style":2098},[297],[31,8559,8560,8563],{"style":3882},[31,8561],{"className":8562,"style":2105},[304],[31,8564,8566],{"className":8565},[309,310,311,312],[31,8567,2018],{"className":8568},[74,75,312],[31,8570,8571,8574],{"style":2166},[31,8572],{"className":8573,"style":2105},[304],[31,8575,8576],{},[31,8577,2336],{"className":8578},[1443,2175,2176],[31,8580,871],{"className":8581},[870],[31,8583,8585],{"className":8584},[293],[31,8586,8588],{"className":8587,"style":3932},[297],[31,8589],{},[31,8591,169],{"className":8592},[222],[31,8594,8596,8599],{"className":8595},[74],[31,8597,6047],{"className":8598,"style":6112},[74,75],[31,8600,8602],{"className":8601},[285],[31,8603,8605,8625],{"className":8604},[289,836],[31,8606,8608,8622],{"className":8607},[293],[31,8609,8611],{"className":8610,"style":2216},[297],[31,8612,8613,8616],{"style":6501},[31,8614],{"className":8615,"style":305},[304],[31,8617,8619],{"className":8618},[309,310,311,312],[31,8620,2018],{"className":8621},[74,75,312],[31,8623,871],{"className":8624},[870],[31,8626,8628],{"className":8627},[293],[31,8629,8631],{"className":8630,"style":1044},[297],[31,8632],{},[31,8634],{"className":8635,"style":584},[123],[31,8637,3603],{"className":8638},[588],[31,8640],{"className":8641,"style":584},[123],[31,8643,8645,8648,8688,8691,8731,8734,8763,8766,8769],{"className":8644},[65],[31,8646],{"className":8647,"style":4029},[69],[31,8649,8651,8654],{"className":8650},[74],[31,8652,5946],{"className":8653,"style":5973},[74,75],[31,8655,8657],{"className":8656},[285],[31,8658,8660,8680],{"className":8659},[289,836],[31,8661,8663,8677],{"className":8662},[293],[31,8664,8666],{"className":8665,"style":5986},[297],[31,8667,8668,8671],{"style":5989},[31,8669],{"className":8670,"style":305},[304],[31,8672,8674],{"className":8673},[309,310,311,312],[31,8675,1178],{"className":8676,"style":226},[74,75,312],[31,8678,871],{"className":8679},[870],[31,8681,8683],{"className":8682},[293],[31,8684,8686],{"className":8685,"style":1044},[297],[31,8687],{},[31,8689,169],{"className":8690},[222],[31,8692,8694,8697],{"className":8693},[74],[31,8695,936],{"className":8696},[74,75],[31,8698,8700],{"className":8699},[285],[31,8701,8703,8723],{"className":8702},[289,836],[31,8704,8706,8720],{"className":8705},[293],[31,8707,8709],{"className":8708,"style":2216},[297],[31,8710,8711,8714],{"style":1025},[31,8712],{"className":8713,"style":305},[304],[31,8715,8717],{"className":8716},[309,310,311,312],[31,8718,2018],{"className":8719},[74,75,312],[31,8721,871],{"className":8722},[870],[31,8724,8726],{"className":8725},[293],[31,8727,8729],{"className":8728,"style":1044},[297],[31,8730],{},[31,8732,185],{"className":8733},[254],[31,8735,8737,8740],{"className":8736},[254],[31,8738,185],{"className":8739},[254],[31,8741,8743],{"className":8742},[285],[31,8744,8746],{"className":8745},[289],[31,8747,8749],{"className":8748},[293],[31,8750,8752],{"className":8751,"style":270},[297],[31,8753,8754,8757],{"style":300},[31,8755],{"className":8756,"style":305},[304],[31,8758,8760],{"className":8759},[309,310,311,312],[31,8761,479],{"className":8762},[74,312],[31,8764],{"className":8765,"style":584},[123],[31,8767,4168],{"className":8768},[588],[31,8770],{"className":8771,"style":584},[123],[31,8773,8775,8778],{"className":8774},[65],[31,8776],{"className":8777,"style":982},[69],[31,8779,8407],{"className":8780,"style":8781},[74,75],"margin-right:0.0715em;",[15,8783,8784,8785,8813],{},"We see that the maximum likelihood objective is equivalent to minimizing the mean squared error, where ",[31,8786,8788,8801],{"className":8787,"translate":35},[34],[31,8789,8791],{"className":8790},[39],[41,8792,8793],{"xmlns":43},[45,8794,8795,8799],{},[48,8796,8797],{},[51,8798,1178],{},[54,8800,1181],{"encoding":56},[31,8802,8804],{"className":8803,"ariaHidden":61},[60],[31,8805,8807,8810],{"className":8806},[65],[31,8808],{"className":8809,"style":843},[69],[31,8811,1178],{"className":8812,"style":226},[74,75]," = the weights and biases of the neural network.",[15,8815,8816],{},"The hidden assumption here is that the errors follow a Gaussian distribution. If we instead assume a Laplace distribution, MLE would lead to the MAE loss. This explains why different tasks choose different loss functions — they correspond to different assumptions about the data distribution.",[22,8818,8819],{},[15,8820,8821],{},[370,8822,8823],{},"In neural networks, maximum likelihood estimation means finding, among all possible combinations of weights and biases, the set that makes the training data most likely to occur.",[15,8825,8826,8827,142],{},"Therefore, every time we use gradient descent to update weights, we are essentially performing ",[370,8828,8829],{},"MLE optimization",{"title":8831,"searchDepth":8832,"depth":8832,"links":8833},"",2,[8834,8835,8836],{"id":12,"depth":8832,"text":13},{"id":904,"depth":8832,"text":905},{"id":2854,"depth":8832,"text":2855},null,false,true,"md",{},"2026-06-08","\u002Fblog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation",{"title":5,"description":8831},"blog\u002F2026\u002F2026-06-08-explanation-of-neural-network-from-maximum-likelihood-estimation","From a statistical learning perspective, modern neural networks can indeed be understood as a large-scale maximum likelihood estimation (MLE) process. Specifically, a neural network is a parameterized function, and the most common way to train a neural network is to perform maximum likelihood estimation on the data.",[8848,8849,8850],"mathematics","machine-learning","statistics","OSIc1Z_kxy8AaNS4-Znr-EroH-G_cjSNvc-vnJSXIHo",1781192430080]