Descriptive Statistics: Mean, Median, Spread and Outliers

Descriptive statistics summarise a set of numbers so people can understand it quickly: a typical value, how spread out the values are, and any unusual ones. They are the first thing you calculate in any analysis, and choosing the wrong summary is one of the most common ways to mislead without meaning to.

The typical value: mean, median and mode

• The mean (average) adds all values and divides by how many there are.
• The median is the middle value when the values are sorted.
• The mode is the most common value, useful for categories.

For symmetric data, the mean and median are close. For skewed data, such as order values, salaries or session times, a few very large values pull the mean up, and the median describes a typical case better.

Spread: standard deviation, percentiles and IQR

• Standard deviation measures how far values typically are from the mean. It is sensitive to extreme values.
• Percentiles split sorted data: the 25th percentile (Q1) has a quarter of values below it, the 75th (Q3) has three quarters below it.
• The interquartile range (IQR) is Q3 minus Q1: the spread of the middle half of the data. It ignores extremes, so it suits skewed data.

A worked example

Here are 1,000 simulated order values, plus two very large business orders:

import numpy as np

rng = np.random.default_rng(5)
orders = np.round(rng.lognormal(mean=3.6, sigma=0.6, size=1000), 2)   # order values: most small, a few large
orders = np.append(orders, [2400.0, 3100.0])                           # two very large business orders

print(f"count  {orders.size}")
print(f"mean   {orders.mean():8.2f}")
print(f"median {np.median(orders):8.2f}")
print(f"std    {orders.std(ddof=1):8.2f}")
q1, q3 = np.percentile(orders, [25, 75])
print(f"Q1 {q1:.2f}   Q3 {q3:.2f}   IQR {q3 - q1:.2f}")
upper = q3 + 1.5 * (q3 - q1)
outliers = orders[orders > upper]
print(f"values above Q3 + 1.5 x IQR ({upper:.2f}): {outliers.size}, largest {outliers.max():.2f}")
trimmed = orders[orders <= np.percentile(orders, 99)]
print(f"mean without the top 1%: {trimmed.mean():.2f}")

count  1002
mean      49.13
median    37.11
std      125.08
Q1 24.60   Q3 56.04   IQR 31.44
values above Q3 + 1.5 x IQR (103.19): 48, largest 3100.00
mean without the top 1%: 42.42

What this tells you:

• The mean (49.13) is well above the median (37.11): the data is right-skewed, and a “typical order” is closer to 37.
• The standard deviation (125.08) is huge compared with the typical order, because two orders of 2,400 and 3,100 dominate it. The IQR (31.44) describes the usual spread much better.
• The IQR rule flags 48 values above Q3 + 1.5 × IQR. Most are normal larger orders in a skewed distribution, not errors. Investigate before removing anything.
• Removing the top 1% drops the mean from 49.13 to 42.42, which shows how sensitive the mean is to a few values.

How to report a skewed metric

Report the median and the mean together, add percentiles (such as the 90th) for the long tail, and say how outliers were treated. For revenue, the mean still matters, because totals are what the business earns; just do not present it as “typical”.

Outliers: error or real?

Before removing an outlier, find out what it is. Errors (a price typed with extra zeros, a test order) should be fixed or excluded, and the rule written down. Real extreme values, such as a large business customer, carry important information: analyse them separately or report results with and without them.

Common mistakes

1. Reporting only the mean for skewed data.
2. Deleting outliers automatically, without checking whether they are real.
3. Using standard deviation to describe heavily skewed data.
4. Comparing averages of groups with very different sizes or mixes without checking the distributions.

Key takeaway

Describe data with a typical value and a spread. Use the median and IQR for skewed data, the mean and standard deviation for roughly symmetric data, and investigate outliers before acting on them. Next: distributions and the central limit theorem.