Optimization is one of the key topics in the area of artificial intelligence and machine learning. Especially in the sense that the entire field of machine learning is based on optimization, but also in the sense that some artificial intelligence and machine learning methods can do optimization.

Loosely speaking, optimization is the process of finding parameters that minimize or maximize a certain criterion. The convention in optimization literature is to always deal with minimization, because a maximization problem can be turned into a minimization problem simply by flipping the sign of the criterion.