Logistic Regression

Logistic Regression is a classification algorithm that estimates the probability of a classification given an input. It is one of the foundations of deep learning.

\[ error = -\frac{1}{m} \sum^m_{i=1} (1-y)\ln(1-\hat{y}) + y \ln \hat{y} \]

To build our model we add weights (W) and a bias term (b) to the error function \(E(W,b)\).

\[ E(W,b) = -\frac{1}{m} \sum^m_{i=1} (1-y_i)\ln(1-\sigma(Wx^{(i)}) + b) + y_i \ln(\sigma(Wx^{(i)} + b)) \]

This is the function for the binary case, but you can generalize it to more cases using this function.

\[ E(W, b) = -\frac{1}{m} \sum^m_{i=1} \sum^n_{j=1} y_{ij} \ln(\hat{y_{ij}}) \]

The goal is to minimize this function to get the best model.