Least Squares
Least squares method for online linear regression is a method which finds the best fitted predictor for the past data, and then predicts following this predictor. It finds the argument of $\min_\theta \left(\sum_{t=1}^T (\theta' x_t - y_t)^2 \right)$ after $T$ steps.