Logistic Regression in Machine Learning: How It Works with Examples
Logistic Regression is one of the most important beginner-friendly algorithms for classification in machine learning.
It helps a model answer questions such as:
- Is this email spam or not spam?
- Will this customer buy or not buy?
- Is this transaction fraudulent or legitimate?
- Is a medical test positive or negative?
Unlike Linear Regression, which predicts a continuous numerical value, Logistic Regression predicts the probability of a class and then converts that probability into a category.
A simple memory rule is:
Linear Regression → Predict a number
Logistic Regression → Predict a probability for classification
What Is Logistic Regression?
Logistic Regression is a supervised machine learning algorithm mainly used for classification problems.
It learns from labeled examples and estimates the probability that a new example belongs to a particular class.
For example, an email system might predict:
Probability of Spam = 92%
The system can then classify the email as:
Spam
If the probability were much lower, it might classify it as:
Not Spam
So the basic process is:
Input Features → Probability → Class Prediction
Quick Answer
What is Logistic Regression in machine learning?
Logistic Regression is a supervised learning algorithm used mainly for classification. It calculates the probability of an outcome and converts that probability into a category such as:
- Yes / No
- True / False
- Spam / Not Spam
- Fraud / Legitimate
- 0 / 1
Why Is It Called Logistic Regression?
This name confuses many beginners.
Despite the word Regression, Logistic Regression is mainly used for classification.
Why?
Because the model first creates a weighted mathematical score from the input features.
It then applies a special function called the sigmoid function to convert that score into a probability between 0 and 1.
That probability is then used to make a classification decision.
So Logistic Regression contains a regression-style mathematical model internally, but its main practical use is classification.
Linear Regression vs Logistic Regression
| Linear Regression | Logistic Regression |
|---|---|
| Predicts numerical values | Predicts class probabilities |
| Used mainly for regression | Used mainly for classification |
| Output can be any real number | Output probability is between 0 and 1 |
| Example: House price | Example: Spam detection |
| Uses a linear prediction equation | Uses a linear score followed by sigmoid |
Memory Rule
Linear Regression → Number
Logistic Regression → Probability → Class
What Is Classification?
Classification means assigning data to a category.
Examples include:
→ Spam / Not Spam
Transaction
→ Fraud / Legitimate
Customer
→ Will Buy / Will Not Buy
Medical Test
→ Positive / Negative
Logistic Regression is especially useful for binary classification, where there are two possible outcomes.
What Is Binary Classification?
Binary classification means the model chooses between two classes.
Examples:
- 0 or 1
- Yes or No
- Positive or Negative
- Spam or Not Spam
- Churn or Stay
Suppose we encode:
0 = Not Spam
1 = Spam
The model does not simply guess 0 or 1.
It first estimates a probability.
Example:
P(Spam) = 0.92
Then it makes the final classification.
How Logistic Regression Works
A beginner-friendly Logistic Regression workflow is:
Input Data
↓
Calculate Weighted Score
↓
Apply Sigmoid Function
↓
Generate Probability
↓
Compare With Threshold
↓
Predict Class
Let's break this down.
Step 1: Provide Input Features
Suppose we want to detect spam emails.
Possible features might include:
- number of suspicious links
- sender information
- words in the subject line
- number of promotional phrases
- message length
The model uses these features as clues.
Step 2: Calculate a Weighted Score
Each feature receives a learned weight.
The model combines the features into a score.
A simplified equation is:
z = w₁x₁ + w₂x₂ + ... + b
Where:
x = input features
w = learned weights
b = bias or intercept
z = weighted score
For beginners, think:
Features + Learned Weights + Bias = Score
This score can be any real number.
For example:
- −4
- −1.5
- 0
- 2
- 6
But classification needs something easier to interpret.
That is where the sigmoid function becomes useful.
What Is the Sigmoid Function?
The sigmoid function converts the weighted score into a value between:
0 and 1
The formula is:
Sigmoid(z) = 1 / (1 + e⁻ᶻ)
You do not need to memorize the mathematics immediately.
The important idea is:
Any score → Sigmoid → Probability-like value between 0 and 1
Examples:
0.95 → 95% probability
0.80 → 80% probability
0.10 → 10% probability
Why Is the Sigmoid Function Useful?
Without sigmoid, the weighted score could be:
4.7
or:
−3.2
Those values are not directly intuitive as probabilities.
The sigmoid function transforms them into something like:
0.99
or:
0.04
This makes classification easier.
So:
Weighted Score → Sigmoid → Probability → Decision
Step 3: Generate a Probability
Suppose a spam model outputs:
0.92
This can be interpreted as:
92% probability of Spam
Another email might produce:
0.18
or:
18% probability of Spam
The model can then use a threshold to choose the final class.
Step 4: Use a Classification Threshold
A common example threshold is:
0.50
If:
Probability ≥ 0.50
predict:
Positive Class
If:
Probability < 0.50
predict:
Negative Class
Example:
Spam probability = 0.80
Prediction:
Spam
Another example:
Spam probability = 0.30
Prediction:
Not Spam
Is 0.50 Always the Best Threshold?
No.
This is an important point.
A threshold of 0.50 is common for teaching, but it is not automatically best for every problem.
Suppose you are detecting fraud.
Missing a fraudulent transaction may be very expensive.
You might choose a lower threshold to catch more possible fraud cases.
In another application, false alarms may be very costly.
Then a different threshold may be more appropriate.
Beginner Rule
Probability and final decision are not the same thing.
The threshold converts one into the other.
What Is a Decision Boundary?
The decision boundary is the point where the model changes from one predicted class to another.
In a simple binary problem:
- one side of the boundary may be Class 0
- the other side may be Class 1
Logistic Regression learns a linear decision boundary in the feature space.
For one feature, this can be easy to visualize.
For many features, the boundary exists in a higher-dimensional space.
Real-Life Example 1: Spam Detection
Suppose the model receives an email.
Features might include:
- suspicious words
- number of links
- sender reputation
- subject-line patterns
The model calculates:
Spam probability = 0.92
Threshold:
0.50
Final prediction:
Spam
Real-Life Example 2: Medical Classification
Suppose a model helps estimate whether a medical test is positive.
Possible features:
- age
- symptoms
- test results
- medical measurements
Output:
Probability of Positive = 0.78
If the threshold is 0.50:
Prediction = Positive
Important: in real medical applications, evaluation, calibration, safety, and human oversight matter much more than this simplified example.
Real-Life Example 3: Fraud Detection
Features may include:
- transaction amount
- location
- spending pattern
- time of purchase
- account history
The model might output:
Fraud probability = 0.84
Final prediction:
Fraud
Real-Life Example 4: Customer Churn
Suppose a company wants to predict whether a customer may cancel.
Features:
- account age
- number of support requests
- usage level
- payment history
- recent activity
Prediction:
Churn probability = 0.67
Final class:
Likely to Churn
Logistic Regression Formula Explained Simply
The process has two main parts.
Part 1: Calculate the Score
z = w₁x₁ + w₂x₂ + ... + b
Part 2: Convert Score to Probability
p = 1 / (1 + e⁻ᶻ)
Then:
p → probability
Example:
p = 0.91
means:
91% predicted probability for the positive class
What Do the Weights Mean?
Each feature has a learned weight.
The sign and size of a coefficient affect how that feature changes the model's score.
A positive weight generally pushes the score toward the positive class.
A negative weight pushes it toward the negative class.
For example, in a churn model:
More support complaints
might increase predicted churn probability.
But coefficient interpretation requires care because:
- feature scaling matters
- correlated features can affect coefficients
- the relationship is expressed through log-odds internally
For beginners, the simplest idea is:
Weights show how features contribute to the model's decision.
What Is the Bias or Intercept?
The bias is a baseline term added to the weighted feature score.
Conceptually:
Weighted Features + Bias = z
Then:
z → Sigmoid → Probability
The bias helps shift the model's decision boundary.
How Is Logistic Regression Trained?
During training, the model sees examples where the correct class is already known.
For example:
| Actual Class | |
|---|---|
| Prize message | Spam |
| Meeting reminder | Not Spam |
| Claim reward | Spam |
| Project update | Not Spam |
The model:
- calculates probabilities
- compares them with the true labels
- measures error
- adjusts weights
- repeats
Over time, the parameters improve.
What Loss Function Does Logistic Regression Use?
Logistic Regression commonly uses a classification loss called:
Log Loss
or:
Binary Cross-Entropy
The basic idea is simple.
If the model is highly confident and wrong, the loss becomes large.
Example:
Actual class:
Spam
Model prediction:
Spam probability = 0.02
That is a very confident wrong prediction.
The loss should strongly penalize it.
If the model predicts:
Spam probability = 0.95
and the message is actually spam, the loss is much smaller.
Memory Rule
Log Loss rewards confident correct predictions and penalizes confident wrong predictions.
Why Not Use MSE Like Linear Regression?
Linear Regression often uses squared-error-based losses.
Logistic Regression is solving a classification probability problem.
Log Loss is better matched to probability-based binary classification.
This is another important difference between Linear and Logistic Regression.
Logistic Regression Python Example
A simple scikit-learn example looks like this:
from sklearn.linear_model import LogisticRegressionmodel = LogisticRegression()model.fit(X_train, y_train)predictions = model.predict(X_test)To get probabilities:
probabilities = model.predict_proba(X_test)The first process predicts classes.
The second gives estimated class probabilities.
What Does fit() Do?
This line:
model.fit(X_train, y_train)trains the Logistic Regression model.
It learns:
- feature weights
- intercept
- decision relationship
from the training data.
What Does predict() Do?
This line:
model.predict(X_test)returns predicted classes.
Example:
0
1
1
0These might represent:
0 = Not Spam
1 = Spam
What Does predict_proba() Do?
This gives class probabilities.
For example:
[0.08, 0.92]This could mean:
8% Not Spam
92% Spam
That distinction is very useful.
Logistic Regression vs Linear Regression
| Concept | Linear Regression | Logistic Regression |
|---|---|---|
| Main task | Regression | Classification |
| Target | Numerical | Class label |
| Output | Continuous number | Probability |
| Final prediction | Number | Category |
| Typical function | Linear equation | Linear score + sigmoid |
| Example | House price | Spam detection |
| Common loss | MSE | Log Loss |
Logistic Regression vs Decision Tree
These models can both solve classification problems, but they learn differently.
Logistic Regression
Learns a linear decision boundary.
Advantages:
- simple
- fast
- interpretable
- strong baseline
Decision Tree
Learns IF/THEN-style splits.
Advantages:
- handles nonlinear rules
- easy to visualize
- captures interactions naturally
Neither is always better.
The right choice depends on the data.
Binary vs Multiclass Logistic Regression
Logistic Regression is commonly introduced with binary classification.
Example:
Spam / Not Spam
But variants can also support multiclass classification.
Example:
Cat / Dog / Bird
Common strategies include:
- One-vs-Rest
- Multinomial Logistic Regression
Beginners do not need to master these immediately.
Just remember:
Logistic Regression is not limited to only two classes in practical machine learning libraries.
Advantages of Logistic Regression
1. Simple to Understand
The core process is easy to follow:
Features → Score → Sigmoid → Probability → Class
2. Fast to Train
It is often computationally efficient on structured datasets.
3. Strong Baseline
It is an excellent first model for many classification problems.
4. Probability Output
It naturally produces class probabilities.
5. Relatively Interpretable
Coefficients can provide insight into how features influence predictions.
6. Works Well for Many Binary Problems
Especially when the decision boundary is approximately linear.
Limitations of Logistic Regression
1. Linear Decision Boundary
Basic Logistic Regression cannot automatically capture every complex nonlinear relationship.
2. Sensitive to Poor Features
Weak or irrelevant features can reduce performance.
3. Feature Scaling May Matter
Especially when regularization is involved.
4. Multicollinearity Can Complicate Interpretation
Highly correlated features can make coefficients harder to interpret.
5. Complex Data May Need More Flexible Models
Images, audio, and highly nonlinear patterns may require other approaches.
When Should You Use Logistic Regression?
Consider Logistic Regression when:
- the target is categorical
- binary classification is needed
- the dataset is structured
- probability estimates are useful
- interpretability matters
- a simple baseline is needed
- the relationship is reasonably separable by a linear boundary
When Might Logistic Regression Be a Poor Choice?
It may be less suitable when:
- relationships are highly nonlinear
- raw image/audio data is used directly
- complex interactions dominate
- the decision boundary is extremely complicated
- feature engineering cannot represent the structure well
In those cases, models such as:
- Decision Trees
- Random Forests
- Gradient Boosting
- Neural Networks
may be worth comparing.
Common Beginner Mistakes
Mistake 1: Thinking Logistic Regression Predicts Continuous Values
Despite its name, it is mainly used for classification.
Mistake 2: Confusing Probability With Class
0.82
is a probability.
Spam
is the final class prediction.
Mistake 3: Assuming 0.50 Is Always the Best Threshold
The best threshold depends on the application.
Mistake 4: Ignoring Class Imbalance
If one class is very rare, accuracy alone may be misleading.
Mistake 5: Ignoring Feature Scaling
Scaling can matter depending on the data and regularization.
Mistake 6: Assuming Logistic Regression Is Too Simple to Be Useful
Simple does not mean weak.
It remains a strong baseline for many classification tasks.
Mistake 7: Using Accuracy Alone
Classification should often be evaluated with several metrics, such as:
- accuracy
- precision
- recall
- F1
- ROC-AUC
The appropriate metric depends on the problem.
How Is Logistic Regression Evaluated?
Common classification metrics include:
Accuracy
How many predictions were correct overall?
Precision
When the model predicts positive, how often is it correct?
Recall
How many actual positive cases did the model find?
F1 Score
Balances precision and recall.
ROC-AUC
Measures ranking performance across classification thresholds.
The best metric depends on the real-world cost of errors.
Example: Fraud Detection
Suppose fraud is rare.
A model predicts:
Legitimate
for every transaction.
Accuracy could still be very high.
But recall for fraud would be:
0%
That is why accuracy alone may be misleading.
Logistic Regression and Modern AI
Logistic Regression is much simpler than modern deep-learning models.
But it teaches several foundational ideas:
- weighted features
- bias
- probability
- sigmoid
- classification
- loss functions
- decision boundaries
These ideas make later concepts easier to understand.
Neural networks also use weighted inputs, biases, activation functions, and loss functions.
The scale and structure are more complex, but the basic learning ideas are related.
Is Logistic Regression Still Used Today?
Yes.
Logistic Regression is still useful because it can be:
- fast
- interpretable
- reliable
- easy to train
- easy to deploy
It is commonly used as:
- a baseline classifier
- an interpretable model
- a probability model for structured data
More complex models do not automatically make Logistic Regression obsolete.
Logistic Regression in One Simple Flow
Remember:
Training Data
↓
Features + Class Labels
↓
Learn Weights and Bias
↓
Calculate Weighted Score
↓
Apply Sigmoid
↓
Generate Probability
↓
Apply Threshold
↓
Predict Class
Example:
Email Features
↓
Probability of Spam = 92%
↓
Spam
That is Logistic Regression.
Frequently Asked Questions
What is Logistic Regression in machine learning?
Logistic Regression is a supervised classification algorithm that estimates class probabilities and converts them into category predictions.
Is Logistic Regression a classification algorithm?
Yes. It is mainly used for classification.
Why is it called Logistic Regression?
It models a linear weighted score and then transforms that score using the logistic or sigmoid function.
What is the sigmoid function?
The sigmoid function converts any real-valued score into a value between 0 and 1.
What is binary classification?
Binary classification means predicting between two classes, such as Spam and Not Spam.
What is the difference between Linear and Logistic Regression?
Linear Regression predicts numerical values. Logistic Regression predicts probabilities used for classification.
Is Logistic Regression still used today?
Yes. It remains an important and widely used baseline and interpretable classification model.
Can Logistic Regression be used in Python?
Yes. Libraries such as scikit-learn provide Logistic Regression implementations.
Can Logistic Regression handle multiple classes?
Yes. Practical implementations can support multiclass classification using methods such as multinomial Logistic Regression or One-vs-Rest.
Is 0.5 always the classification threshold?
No. The threshold should be selected according to the real problem and cost of errors.
Key Takeaways
Remember:
- Logistic Regression is mainly used for classification.
- It estimates probabilities.
- The sigmoid function converts scores into values between 0 and 1.
- A threshold converts probability into a class.
- 0.50 is common, but not always optimal.
- Features receive learned weights.
- The model also learns a bias or intercept.
- Log Loss is commonly used during training.
- Logistic Regression is especially useful for binary classification.
- It can also support multiclass problems.
- It is fast, interpretable, and a strong baseline.
- It works best when the class relationship can be represented reasonably well with a linear decision boundary.
- Classification metrics should be selected according to the real problem.
Final Thoughts
Logistic Regression is one of the best algorithms for understanding how machine learning turns data into decisions.
The process is simple:
Features → Weighted Score → Sigmoid → Probability → Threshold → Class
That single flow teaches several important machine learning ideas at once.
Once Logistic Regression is clear, it becomes easier to understand:
- classification metrics
- decision boundaries
- probability models
- neural networks
- more advanced classification algorithms
The next logical topic is:
Decision Trees in Machine Learning: How They Work with Examples
Continue Learning
- Linear Regression in Machine Learning: Simple & Multiple Regression Explained
- How to Evaluate Regression Models: MAE, MSE, RMSE & R²
- 9 Most Important Machine Learning Algorithms Explained for Beginners
- How to Choose the Right Machine Learning Algorithm
- Classification vs Regression in Machine Learning
Written and reviewed by Hina Majeed.
Last updated: October 21, 2026.