If Your Experiment Needs Statistics, You Ought To Have Done a Better Experiment

Ernest Rutherford? John M. Hammersley? Judy Campisi? Anonymous?
Dear Quote Investigator: The raw data collected in some scientific experiments is extensively processed via statistical operations. The tentative conclusions of this research may be accompanied with complex discussions of confidence levels.

The prominent physicist Ernest Rutherford preferred decisive experiments that did not require sophisticated statistical analysis. Here are three embodiments of this viewpoint:

  • If you need statistics, you did the wrong experiment.
  • If you need statistics to do science, then it’s not science.
  • If your experiment needs statistics, you ought to have done a better experiment.

Would you please explore this topic?

Quote Investigator: In October 1961 mathematician John M. Hammersley of Oxford University ascribed the third statement above to Ernest Rutherford. Hammersley was discussing Monte Carlo methods which are statistics-based methods used to construct efficient computer programs capable of generating approximate answers. Emphasis added to excerpts by QI: 1

In Monte Carlo work we can take heed of Lord Rutherford’s dictum: “If your experiment needs statistics, you ought to have done a better experiment.” In a sense, all good Monte Carlo work is self-liquidating: although we start out with random numbers in order to solve a problem, which may seem to be intractable by conventional numerical analysis, nevertheless we should strive to reduce their influence on the final result, and one should always seize any opportunity to replace a part or even the whole of the sampling experiment by exact analysis.

The evidentiary value of this 1961 citation is lessened by the fact that Rutherford died in 1937.

Below are additional selected citations in chronological order.

Continue reading If Your Experiment Needs Statistics, You Ought To Have Done a Better Experiment

Notes:

  1. 1962 August, U. S. Army Research Office (Durham), Report No. 62-2, Proceedings of the Seventh Conference on the design of Experiments in Army Research Development and Testing, Sponsored by the Army Mathematics Steering Committee conducted at U. S. Army Signal Research & Development Laboratory, Fort Monmouth, New Jersey, Date: October 18-20, 1961, Article: Monte Carlo Methods, Author: J. M. Hammersley (Oxford University and Princeton University), Start Page 17, Quote Page 18 and 19, Published by: U. S. Army Research Office (Durham), Box CM, Duke Station, Durham, North Carolina. (HathiTrust Full View)