Neural Networks Explained Simply, with a Network You Can Run

A neural network is a model made of layers of simple units called neurons. Each neuron takes numbers in, multiplies each by a weight, adds them up with a bias, and passes the result through an activation function. Stacking layers of these neurons lets the network learn very complicated patterns, from the shape of a handwritten digit to the meaning of a sentence. Deep learning simply means neural networks with many layers.

The parts

• Input layer: the features, such as pixel values or measurements.
• Hidden layers: neurons that combine the inputs into more useful signals. Each layer builds on the one before.
• Output layer: the prediction, such as a probability.
• Weights and biases: the numbers learned during training. Large models have billions of them.
• Activation functions such as ReLU, tanh or sigmoid: they bend the signal. Without them, any number of layers would collapse into one straight-line model.

Why layers and activations matter

A logistic regression draws a straight line between classes. Many real problems need curved boundaries. In this example, points inside a circle belong to class 1 and points outside to class 0. A straight line cannot separate them; a small network with one hidden layer of 8 neurons can:

import numpy as np
from sklearn.linear_model import LogisticRegression

rng = np.random.default_rng(0)
# Points inside a circle are class 1, outside are class 0: no straight line can separate them.
X = rng.uniform(-1, 1, size=(1000, 2))
y = (X[:, 0] ** 2 + X[:, 1] ** 2 < 0.5).astype(float)

line = LogisticRegression().fit(X, y)
print(f"logistic regression (a straight line): accuracy {line.score(X, y):.2f}")

# A neural network with one hidden layer of 8 neurons, written with NumPy
W1, b1 = rng.normal(0, 1, (2, 8)), np.zeros(8)
W2, b2 = rng.normal(0, 1, (8, 1)), np.zeros(1)
sigmoid = lambda z: 1 / (1 + np.exp(-z))

lr = 1.0
for epoch in range(2001):
    h = np.tanh(X @ W1 + b1)                     # hidden layer: weighted sums, then a non-linear function
    p = sigmoid(h @ W2 + b2).ravel()             # output: probability of class 1
    # backpropagation: how much each weight contributed to the error
    d_out = (p - y)[:, None] / len(y)
    dW2, db2 = h.T @ d_out, d_out.sum(0)
    d_h = (d_out @ W2.T) * (1 - h ** 2)
    dW1, db1 = X.T @ d_h, d_h.sum(0)
    W1 -= lr * dW1; b1 -= lr * db1; W2 -= lr * dW2; b2 -= lr * db2
    if epoch % 500 == 0:
        loss = -np.mean(y * np.log(p + 1e-9) + (1 - y) * np.log(1 - p + 1e-9))
        print(f"epoch {epoch:4}: loss {loss:.3f}, accuracy {np.mean((p > 0.5) == y):.2f}")

logistic regression (a straight line): accuracy 0.61
epoch    0: loss 0.894, accuracy 0.49
epoch  500: loss 0.124, accuracy 0.97
epoch 1000: loss 0.088, accuracy 0.99
epoch 1500: loss 0.071, accuracy 0.99
epoch 2000: loss 0.059, accuracy 0.99

The straight line gets 61%, barely better than guessing. The network starts at random (49%) and reaches 99% as training adjusts its weights. The hidden neurons each learn a different line, and the output combines them into a circle. How the weights are adjusted is the subject of the next lesson.

When to use deep learning

• Images, audio, video and text: deep learning is the standard choice, usually starting from a pre-trained model.
• Very large datasets with complex patterns.
• Tabular business data: classic models such as gradient boosting are often as good, cheaper and easier to explain. See AI vs machine learning vs deep learning.

Common misconceptions

1. “Neural networks work like the brain.” The idea was inspired by neurons, but the maths is simple weighted sums and functions.
2. “More layers is always better.” Bigger networks need more data and compute, and can overfit.
3. “You must train from scratch.” In practice, most work adapts pre-trained models.

Key takeaway

A neural network stacks layers of neurons that compute weighted sums and pass them through activation functions. The activations let it learn curved, complex patterns that straight-line models cannot. Next: how neural networks learn.