Dark Data: Why What You Don't Know Matters

David J. Hand (Author)


A practical guide to making good decisions in a world of missing data

In the era of big data, it is easy to imagine that we have all the information we need to make good decisions. But in fact the data we have are never complete, and may be only the tip of the iceberg. Just as much of the universe is composed of dark matter, invisible to us but nonetheless present, the universe of information is full of dark data that we overlook at our peril. In Dark Data, data expert David Hand takes us on a fascinating and enlightening journey into the world of the data we don't see.

Dark Data explores the many ways in which we can be blind to missing data and how that can lead us to conclusions and actions that are mistaken, dangerous, or even disastrous. Examining a wealth of real-life examples, from the Challenger shuttle explosion to complex financial frauds, Hand gives us a practical taxonomy of the types of dark data that exist and the situations in which they can arise, so that we can learn to recognize and control for them. In doing so, he teaches us not only to be alert to the problems presented by the things we don't know, but also shows how dark data can be used to our advantage, leading to greater understanding and better decisions.

Today, we all make decisions using data. Dark Data shows us all how to reduce the risk of making bad ones.

Product Details

$29.95  $27.55
Princeton University Press
Publish Date
February 18, 2020
5.8 X 8.6 X 1.3 inches | 1.35 pounds

Earn by promoting books

Earn money by sharing your favorite books through our Affiliate program.

Become an affiliate

About the Author

David J. Hand is emeritus professor of mathematics and senior research investigator at Imperial College London, a former president of the Royal Statistical Society, and a fellow of the British Academy. His many previous books include The Improbability Principle, Measurement: A Very Short Introduction, Statistics: A Very Short Introduction, and Principles of Data Mining.


[A] penetrating study of missing ('dark') data and its impacts on decisions . . . Hand offers expert training, from recognizing when facts are being cherry-picked to designing randomized trials. A book illuminating shadowed corners in science, medicine and policy.---Barbara Kiser, Nature
An exploration of a major problem in data analysis with an attempt of classification, analysing causes, mechanisms, and to some extent also suggest mitigations.---Adhemar Bultheel, European Mathematical Society
An excellent guide to the many reasons for caution in interepreting data.---Diane Coyle, Enlightened Economist
You need to read [Dark Data], and be convinced by David's reasoning and his examples of cases in which unseen or unreported data play a critical and sometimes even a fatal role. You are likely to walk away with the feeling that the term dark data is indeed a very effective one to arouse both curiosity and suspicion, mixed with happiness that finally a great term was coined by a statistician--and sadness that the statistician is not you.---Xiao-Li Meng, IMS Bulletin
A topical and relevant book. . . . Hand knows his subject well and takes us on a tour de force of the many, varied reasons why data, and the elements that we don't have - the dark data - is such a complex area to get right.---Simon Cocking, Irish Tech News