The vast majority of us have been taught to compute an average of a data set by “adding up all the numbers in the data set and dividing that sum by the total number of numbers in the data set”. For example, to compute the average of \(\{1,2,3,6\}\) we would first find that \(1 + 2 + 3 + 6 = 12\) and then recognizing that there are four numbers in the data set, we would compute \(12 \div 4 = 3\) and thus, the average is \(3\).

In a condensed mathematical notation, if we let \(x_{i}\) represent the \(i^{th}\) data point in a data set of \(N\) points, then the average (also called the arithmetic mean) is written as $$\bar{x} = \frac{\sum_{i = 1}^{N}x_{i}}{N}$$

So the question is, “Why Do We Calculate Average The Way We Do”? The answer can be a bit complicated, but in short, we don’t have to calculate it this way. In fact, depending on the type of data we have, its average can be computed differently. The grade school formula we were taught works in a certain set of cases, and typically those cases are what the majority of people tend to deal with on a regular basis.

This formula
$$\bar{x} = \frac{\sum_{i = 1}^{N}x_{i}}{N}$$

does something special, geometrically. The trick to understanding what’s going on is to ask the following question using this numerical example:

“Is there one number such that the total `distance’ from that number to all the numbers in the data set (where my data set is \(\{1,2,3,6\}\)) is minimal?”

Now, answering this question requires us to first agree on how we want to measure distance. It sounds like a simple question, but resolving how we want to compute distance is at the heart of understanding why “average” is the way it is.

Enter Pythagoras

Pythagoras?! Yes, Pythagoras, as in the formula we all have been told to know by heart \(a^{2} + b^{2} = c^{2}\). We know that this formula is supposed to relate the length of the two legs of a right triangle (\(a\) and \(b\)) to the hypotenuse, \(c\) (the longest side of the triangle). Thus, once we know \(c^{2}\) we have \(c\) via a square root.

We’ve also had burned into our memories things like “3, 4, 5” triangles because \(3^{2} + 4^{2} = 5^{2}\). But now I digress. What do triangles have to do with averages? Actually, nothing. It’s the Pythagorean Formula (\(a^{2} + b^{2} = c^{2}\)) that is the key to all this.

This formula is how we want to measure distance. Specifically, \(c = \sqrt{a^{2} + b^{2}}\).

A Numerical Example

Let’s consider our data set, \(\{1,2,3,6\}\). We computed the average to be \(3\). Now, and humor me here, let’s say we wanted to know the “Pythagorean distance” of our data points to our average. To find this, we simply have to extend the Pythagorean Formula, sum the square distances, and take a square root. In other words, we want to do this:
$$\begin{eqnarray*}
(1-3)^{2} + (2-3)^{2} + (3-3)^{2} + (6-3)^{2} & = & 4 + 1 + 0 + 9\\
& = & 14\\
& = & c_{\bar{x}}^{2}
\end{eqnarray*}$$

Thus, our distance is \(c_{\bar{x}} = \sqrt{14} \approx 3.741657\). To keep the numbers “friendly”, we’ll stick with \(c_{\bar{x}}^{2} = 14\).

Here is the interesting part. What would \(c^{2}\) be, if we chose a different estimate for our average. For example, what would \(c^{2}\) be if we thought that our average should be \(4\) (we’ll call this \(c_{4}^{2}\))? Well, here’s what we’d get.

$$\begin{eqnarray*}
(1-4)^{2} + (2-4)^{2} + (3-4)^{2} + (6-4)^{2} & = & 9 + 4 + 1 + 4\\
& = & 18\\
& = & c_{4}^{2}
\end{eqnarray*}$$

Notice that \(c_{4}^{2} > c_{\bar{x}}^{2}\) since \(18 > 14\).

How about \(c_{2}\)?
$$\begin{eqnarray*}
(1-2)^{2} + (2-2)^{2} + (3-2)^{2} + (6-2)^{2} & = & 1 + 0 + 1 + 16\\
& = & 18\\
& = & c_{2}^{2}
\end{eqnarray*}$$

And \(c_{2}^{2} > c_{\bar{x}}^{2}\)

So a challenge then to the reader, is it possible to find a value \(v\) such that \((1-v)^{2} + (2-v)^{2} + (3-v)^{2} + (6-v)^{2} < 14\)? And the answer is, no.

The average, computed as
$$\bar{x} = \frac{\sum_{i = 1}^{N}x_{i}}{N}$$
minimizes the sum of the square differences.

In fact, the reader should pick as many values of \(v\) as he/she wants and for each value of \(v\) compute \((1-v)^{2} + (2-v)^{2} + (3-v)^{2} + (6-v)^{2}\). The average will always give the smallest sum.

A Proof

We would like to find a value \(v\) such that $$\sum_{i = 1}^{N}(x_{i}-v)^{2}$$ is minimized.

First, write $$\sum_{i = 1}^{N}(x_{i}-v)^{2}$$ as
$$\sum_{i=1}^{N}(x_{i}^{2} – 2vx_{i} + v^{2})$$
which we will call \(f(v)\) and write it as
$$f(v) = Nv^{2} – 2v\sum_{i=1}^{N}x_{i} + \sum_{i=1}^{N}x_{i}^{2}$$
Now, we recognize that \(f(v)\) is a parabola in \(v\) whose global minimum occurs at its vertex (since \(N > 0\)). The vertex of a parabola written as \(ax^{2} + bx + c\) is given as \(x = \frac{-b}{2a}\). In our case, \(a = N, b = -2\sum_{i=1}^{N}x_{i}, c = \sum_{i=1}^{N}x_{i}^{2}\). Thus, \(f(v)\) is minimized at \(v = \frac{-b}{2a} = \frac{\sum_{i=1}^{N}x_{i}}{N}\). Notice that this is just the formula for average (arithmetic mean).

Summary

Here is basically what you should take away from this post.

1. The arithmetic mean is not an arbitrary metric.
2. The arithmetic mean minimizes a specific distance measure.
3. If you are using the arithmetic mean as a statistic to report, ask yourself if this makes sense. You may be more interested in a different type of average if you are minimizing a different distance measure. For example, search for “geometric mean” to get an idea of another type of “average” and where it is used.
4. For the reader who remembers or has had exposure to statistics, our Pythogrean distance measure is very closely tied to “standard deviation” and “variance”. Recall that the population variance is just
   $$\mbox{var} = \sigma^{2} = \frac{\sum_{i=1}^{N}(x_{i} – \mu)^{2}}{N}$$ where \(\mu\) is the population mean. Notice the similarity? The division by \(N\) is just “averaging” the total square distance (the numerator). Standard deviation is just \(\sqrt{\mbox{var}} = \sigma\).
5. When we talk about a “line of best fit”, we are in effect doing the same thing that was done here. The arithmetic mean is the “constant of best fit”.