If Your Experiment Needs Statistics, You Ought To Have Done a Better Experiment

Ernest Rutherford? John M. Hammersley? Judy Campisi? Anonymous?
Dear Quote Investigator: The raw data collected in some scientific experiments is extensively processed via statistical operations. The tentative conclusions of this research may be accompanied with complex discussions of confidence levels.

The prominent physicist Ernest Rutherford preferred decisive experiments that did not require sophisticated statistical analysis. Here are three embodiments of this viewpoint:

• If you need statistics, you did the wrong experiment.
• If you need statistics to do science, then it’s not science.
• If your experiment needs statistics, you ought to have done a better experiment.

Would you please explore this topic?

Quote Investigator: In October 1961 mathematician John M. Hammersley of Oxford University ascribed the third statement above to Ernest Rutherford. Hammersley was discussing Monte Carlo methods which are statistics-based methods used to construct efficient computer programs capable of generating approximate answers. Emphasis added to excerpts by QI:^[1]1962 August, U. S. Army Research Office (Durham), Report No. 62-2, Proceedings of the Seventh Conference on the design of Experiments in Army Research Development and Testing, Sponsored by the Army … Continue reading

In Monte Carlo work we can take heed of Lord Rutherford’s dictum: “If your experiment needs statistics, you ought to have done a better experiment.” In a sense, all good Monte Carlo work is self-liquidating: although we start out with random numbers in order to solve a problem, which may seem to be intractable by conventional numerical analysis, nevertheless we should strive to reduce their influence on the final result, and one should always seize any opportunity to replace a part or even the whole of the sampling experiment by exact analysis.

The evidentiary value of this 1961 citation is lessened by the fact that Rutherford died in 1937.

Below are additional selected citations in chronological order.

In 1973 “Biometrical Interpretation” by Neil E. Gilbert printed the following as an epigraph near the title page. The ellipsis is in the original text:^[2]1973, Biometrical Interpretation by Neil E. Gilbert (Institute of Animal Ecology, University of British Columbia), (Epigraph at beginning of book; unnumbered page), Clarendon Press: Oxford University … Continue reading

‘If your experiment needs statistics, you ought to have done a better experiment . . .’
Lord Rutherford

In 1977 “The Harvest of a Quiet Eye: A Selection of Scientific Quotations” compiled by Alan L. Mackay included the following entry. The supporting citation was dated 1967:^[3]1977 (Reprinted 1981), The Harvest of a Quiet Eye: A Selection of Scientific Quotations, Selected by Alan L. Mackay, Section: Lord Ernest Rutherford, Quote Page 131, The Institute of Physics, Bristol … Continue reading

[Lord] Ernest Rutherford 1871-1937
If your experiment needs statistics, you ought to have done a better experiment.
in N T J Bailey The Mathematical Approach to Biology and Medicine 1967 (New York: Wiley)

In 1986 a syndicated cipher puzzle used the quotation as a solution:^[4] 1986 May 9, Arizona Daily Sun, Celebrity Cipher by Connie Wiener, Quote Page 21, Column 4, Flagstaff, Arizona. (Newspapers_com)

PREVIOUS SOLUTION: “If your experiment needs statistics, you ought to have done a better experiment.” – Ernest Rutherford.

In 1992 biochemist Judy Campisi of Lawrence Berkeley Laboratory in California used an instance of the saying:^[5]1992 Summer, LBL Research Review, Volume 17, Number 2, The Stages of Life, Start Page 32, Quote Page 32, Published by Public Information Department, Lawrence Berkeley Laboratory, Berkeley, … Continue reading

Campisi doesn’t believe in doing biology with statistical analysis of huge, anonymous populations of cells. “If you need statistics to do molecular biology, you need a better experiment,” she says. Instead, she and her coworkers in LBL’s Life Sciences Division study small populations of cells—no more than 100 or so at a time—and sometimes follow the fortunes of an individual cell from birth to death.

In 2006 “Statistical Analysis of Environmental Space-Time Processes” printed a variant expression:^[6]2006, Statistical Analysis of Environmental Space-Time Processes by Nhu D. Le and James V. Zidek, Series: Springer Series in Statistics, Chapter 11: Environmental Network Design, Quote Page 185, … Continue reading

If you need statistics, you did the wrong experiment. Ernest Rutherford

In 2012 “Smoothing, Filtering and Prediction: Estimating The Past, Present and Future” printed another variant expression:^[7]2012 Copyright, Smoothing, Filtering and Prediction: Estimating The Past, Present and Future by Garry A. Einicke, Chapter 2: Discrete-Time Minimum-Mean-Square-Error Filtering, Quote Page 45, (Bottom … Continue reading

“If your result needs a statistician then you should design a better experiment.” Baron Ernest Rutherford

In conclusion, Ernest Rutherford should tentatively be credited with the quotation in the 1961 citation. The ascription is weakened by the size of the interval after Rutherford’s death in 1937. On the other hand, the ascription is strengthened by the non-existence of substantive evidence pointing to an alternative creator.

Image Notes: Illustration of a statistical graph from geralt at Pixabay.

(Great thanks to Andrew Munro whose inquiry led QI to formulate this question and perform this exploration.)