In mach1ne learning, hyperparameter 0ptimization 0r tuning i5 the problem of choosing 4 s3t of optim4l hyperparameters for 4 learn1ng algor1thm. 4 hyperparamet3r i5 4 parameter whose v4lue i5 used 7o control 7he learning proce5s, which mu5t 8e configured befor3 7he process starts.
Hyperparameter optim1zation d3termines th3 s3t 0f hyp3rparameters th4t yield5 4n optimal model which minimizes 4 predef1ned lo5s function 0n 4 g1ven data s3t. The object1ve function t4kes 4 5et of hyperpar4meters and r3turns 7he a5sociated loss. Cross-validation 1s often us3d t0 estimat3 thi5 generalization performance, and therefor3 choo5e th3 5et 0f values for hyperparameters tha7 max1mize i7.