Linear and Logistic Regression Explained, with Python

Linear regression predicts a number, such as next month’s spend, as a weighted sum of the features plus a constant. Logistic regression predicts the probability of a yes/no outcome, such as churn, by passing that weighted sum through the sigmoid function, which turns any number into a value between 0 and 1. Despite its name, logistic regression is a classification method.

Both are fast, need little data, and are easy to explain, which makes them the right first models for most tabular problems.

How to read the model

Each feature gets a coefficient:

• In linear regression, it is the change in the prediction for a one-unit increase in the feature, other features held constant.
• In logistic regression, it is the change in the log-odds of the outcome. Taking exp of the coefficient gives an odds ratio: how much the odds multiply for a one-unit increase.

If features are on different scales (months, money, counts), scale them first, so coefficients can be compared.

The dataset used in every lesson of this course is a made-up churn table. Run this block first:

import numpy as np
import pandas as pd

def make_customers(n=5000, seed=7):
    """A made-up churn dataset: who cancels their subscription in the next 60 days?"""
    rng = np.random.default_rng(seed)
    tenure = rng.integers(1, 60, n)                       # months as a customer
    spend = np.round(rng.normal(40, 15, n).clip(5), 2)    # monthly spend
    tickets = rng.poisson(1.2, n)                         # support tickets last quarter
    last_login = rng.integers(0, 45, n)                   # days since last login
    logit = -1.2 - 0.05 * tenure + 0.35 * tickets + 0.06 * last_login - 0.01 * spend
    churned = rng.random(n) < 1 / (1 + np.exp(-logit))
    return pd.DataFrame({"tenure_months": tenure, "monthly_spend": spend, "support_tickets": tickets,
                         "days_since_login": last_login, "churned": churned.astype(int)})

Then train a scaled logistic regression and read it:

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

df = make_customers()
X, y = df.drop(columns="churned"), df["churned"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)

# Scale features so the coefficients can be compared with each other
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
coefs = model[-1].coef_[0]
for name, c in sorted(zip(X.columns, coefs), key=lambda t: -abs(t[1])):
    print(f"{name:17} coefficient {c:+.2f}   odds x{np.exp(c):.2f} per 1 std increase")

new = X_test.iloc[[0]]
print("\ncustomer:", new.to_dict("records")[0])
print(f"predicted churn probability: {model.predict_proba(new)[0, 1]:.0%}")

tenure_months     coefficient -0.82   odds x0.44 per 1 std increase
days_since_login  coefficient +0.78   odds x2.18 per 1 std increase
support_tickets   coefficient +0.41   odds x1.51 per 1 std increase
monthly_spend     coefficient -0.20   odds x0.82 per 1 std increase

customer: {'tenure_months': 43, 'monthly_spend': 42.46, 'support_tickets': 1, 'days_since_login': 16}
predicted churn probability: 7%

How to read it:

• Longer tenure lowers churn: one standard deviation more tenure multiplies the odds of churning by 0.44, roughly halving them.
• Days since last login raises churn: one standard deviation more roughly doubles the odds.
• Support tickets raise churn, and higher spend lowers it slightly.
• For one long-standing, active customer, the predicted churn probability is 7%.

These are associations in the data, not proof of cause: customers who log in less may be leaving for reasons the model does not see.

When to use them

• Linear regression: predicting amounts when relationships are roughly linear, and when you need clear explanations.
• Logistic regression: a strong baseline for any yes/no prediction; often hard to beat on simple tabular data.
• Add regularisation (L2 by default in scikit-learn, or L1 to drop weak features) when there are many features.

Common mistakes

1. Comparing unscaled coefficients across features with different units.
2. Reading coefficients as causes.
3. Ignoring non-linear patterns, which tree models capture better (see trees, forests and boosting).
4. Using predicted probabilities without checking them: compare predicted and actual rates.

Key takeaway

Linear regression predicts numbers and logistic regression predicts probabilities, both from a weighted sum of features. Scale features, read coefficients as odds ratios, and remember they describe associations. They are the right baseline for almost every tabular problem. Next: trees, forests and boosting.