Methods for Studying Coincidences

One of my favorite mathematics papers of all time is called "Methods for Studying Coincidences." By Persi Diaconis and Frederick Mosteller, it aims to provide a rigorous mathematical framework for the study of coincidences.

Using probabilistic analysis, the paper explores everything from why we see newly learned words almost immediately after first learning them, to why double lottery winners exist, to even the frequency of meeting people with the same birthday. They even explore whether or not we can statistically state that Shakespeare used alliteration, or if the frequency of words with similar-sounding beginnings could simply be explained by chance alone.

For example, when it comes to newly learned words, we are often astonished that as soon as we learn a new word, we begin to see it quite frequently, or at least soon after we learn it. Now it could just be due to our heightened perception. But Diaconis and Mosteller argue that statistics can also explain why this happens. A newly learned word is generally quite rare, as otherwise we would have known it already. And for some of these rare words, they will appear far later in our experience (i.e., later in life) than the average expected time, assuming they adhere to what is known as a Poisson process. Furthermore, some of these late-appearing words might also reappear much more rapidly than we expect. Since we know that there are many rare words in each language, we therefore shouldn't be surprised if some fraction of these rare words appear in our daily lives in close proximity, yielding the appearance of coincidence.

Their analyses hinge on something that we often forget: while something might seem astonishing and a remarkable coincidence, if enough people are involved, chances are very good that one of them will have something "coincidental" happen to them. Think double lottery winners. This leads us to the Law of Truly Large Numbers:

With a large enough sample, any outrageous thing is likely to happen. The point is that truly rare events, say events that occur only once in a million [as the mathematician Littlewood (1953) required for an event to be surprising] are bound to be plentiful in a population of 250 million people. If a coincidence occurs to one person in a million each day, then we expect 250 occurrences a day and close to 100,000 such occurrences a year.

Going from a year to a lifetime and from the population of the United States to that of the world (5 billion at this writing), we can be absolutely sure that we will see incredibly remarkable events. When such events occur, they are often noted and recorded. If they happen to us or someone we know, it is hard to escape that spooky feeling.

Ultimately, they conclude that coincidences are often in the mind of the observer and not in the probabilities.

The whole paper is well worth a read.

Top image: Brent Newhall/Flickr/CC-licensed