The "problem of induction" arises when we ask whether this form of reasoning can lead to apodeictic or "metaphysical" certainty about knowledge, as the Scholastics thought. Thomas Aquinas especially thought that certain knowledge can be built upon first principles, axioms, and deductive or logical reasoning. This certain knowledge does indeed exist, within a system of thought such as logic or mathematics. But it can prove nothing about the natural world.

Bacon understood logical deduction, but like some proto-empiricists among the Scholastics (notably John Duns Scotus and William of Ockham), Bacon argued in his Novum Organum that knowledge of nature comes from studying nature, not from reasoning in the ivory tower.

Bacon likely did not believe certainty can result from inductive reasoning, but his great contribution was to see that (empirical) knowledge gives us power over nature, by discovering what he called the form nature, the real causes underlying events.

It was of course David Hume who pointed out the lack of certainty or logical necessity in the method of inferring causality from observations of the regular succession of "causes and events." His great model of scientific thinking, Isaac Newton, had championed induction as the source of his ideas. As if his laws of motion were simply there in the data from Tycho Brahe's extensive observations and Johannes Kepler's orbital ellipses.

"Hypotheses non fingo," Newton famously said, denying the laws were his own ideas. Although since Newton it is obvious that the gravitational influence of the Sun causes the Earth and other planets to move around their orbits, Hume's skepticism led him to question whether we could really know, with certainty, anything about causality, when all we ever see in our inductive study is the regular succession of events.

Thus it was Hume who put forward the "problem of induction" that has bothered philosophers for centuries, spilling a great deal of philosophical ink. Hume's skepticism told him induction could never yield a logical proof. But Hume's mitigated skepticism saw a great deal of practical value gained by inferring a general rule from multiple occurrences, on the basis of what he saw as the uniformity of nature. What we have seen repeatedly in the past is likely to continue in the future.

While Hume was interested in causal sequences in time, his justification of induction also applies to modern statistical thinking. We infer the frequency of some property of an entire population from the statistics of an adequately large sample of that population.

The Information Philosopher's solution to this problem (more properly a "pseudo-problem," to use the terminology of twentieth-century logical positivists, logical empiricists, and linguistic analysts) is easily seen by examining the information involved in the three (or four) methods of reasoning - logical deduction, empirical induction, mathematical induction (actually a form of deduction), and what Charles Sanders Peirce called "abduction," to complete one of his many philosophical triads.

Mathematical induction is a method of proving some property of all the natural numbers by proving it for one number, then showing that if it is true for the number n, it must also be true for n + 1. In both deduction and mathematical induction, the information content of the conclusion is often no more than that already in the premises. To be sure, the growth of our systems of thought such as logic, mathematics, and perhaps especially geometry, has generated vast amounts of new knowledge, new information, when surprising new theorems are proved within the system. And much of this information has turned out to be isomorphic with information structures in the universe. But the existence of an isomorphism is an empirical, not a logical, finding.

The principal role of deduction in science is to derive, logically or mathematically, predictable consequences of the new theory that might be tested by suitable experiments. This step simply draws out information already present in the hypothesis. Theory, including deductions and predictions, is all done in the realm of ideas, pure information.

Abduction is the formation of new hypotheses, one step (rarely the first) in what some philosophers of science in the twentieth century described as the scientific method - the hypothetico-deductive-observational method. It can be described more simply as the combination of theories and experiments. Observations are very often the spur to theory formation, as the old inductive method emphasized. A scientist forms a hypothesis about possible causes for what is observed.

Although the hypothesis is an immaterial idea, pure information, the abduction of a hypothesis creates new information in the universe, albeit in the minds of the scientists.

By contrast, an experiment is a material and energetic interaction with the world that produces new information structures to be compared with theoretical predictions. Experiments are Baconian accumulations of data that can never "prove" a theory (or hypothesis). But confirmation of any theory consists entirely of finding that the statistical outcomes of experiments match the theory's predictions, within reasonable experimental "error bars." The best confirmation of any scientific theory is when it predicts a phenomenon never before seen, such that when an experiment looks, that phenomenon is found to exist.

These "surprising" results of great theories shows the extent to which science is not a mere "economic summary of the facts," as claimed by Ernst Mach, the primary creator of logical positivism in science.

Mach had a great influence on the young Albert Einstein, who employed Mach's idea in discovering his special theory of relativity. The positivists insisted on limiting science to "observable" facts. Atoms were not (yet) observable, so despite the great chemical theories of Dalton explaining molecules, the great statistical mechanical work of James Clerk Maxwell and Ludwig Boltzmann explaining thermodynamics, it remained for Einstein to predict the observable effects of atomic and molecular motions on the motions of visible particles like pollen seeds in a liquid.

The experimental measurements of those visible motions, with exactly the extent of motion predicted by Einstein, confirmed the reality of atoms. The motions had been observed, almost eighty years earlier, by Robert Brown. Einstein's 1905 hypothesis - a "free creation of the human mind," as he called it and his other extraordinary theories, together with the deduction of mathematically exact predictions from the theory, and followed by the 1908 experiments by Jean Perrin, gives us a paradigmatic example of the scientific method.

In information philosophy terms, the abstract immaterial information in the Einstein theory of Brownian motion, was found to be isomorphic to material and energetic information structures in the universe.

In his early years, Einstein thought himself a disciple of Mach, a positivist. He limited his theories to observable facts. Special relativity grew from the fact that absolute motions are not observable.

But later when he realized the source of his greatest works were his own mental inventions, he changed his views. Here is Einstein in 1936,

We now realize, with special clarity, how much in error are those theorists who believe that theory comes inductively from experience. Even the great Newton could not free himself from this error ("Hypotheses non fingo")...

There is no inductive method which could lead to the fundamental concepts of physics. Failure to understand this fact constituted the basic philosophical error of so many investigators of the nineteenth century. It was probably the reason why the molecular theory and Maxwell's theory were able to establish themselves only at a relatively late date. Logical thinking is necessarily deductive; it is based upon hypothetical concepts and axioms. How can we expect to choose the latter so that we might hope for a confirmation of the consequences derived from them?

The most satisfactory situation is evidently to be found in cases where the new fundamental hypotheses are suggested by the world of experience itself. The hypothesis of the non-existence of perpetual motion as a basis for thermodynamics affords such an example of a fundamental hypothesis suggested by experience; the same holds for Galileo's principle of inertia. In the same category, moreover, we find the fundamental hypotheses of the theory of relativity, which theory has led to an unexpected expansion and broadening of the field theory, and to the superseding of the foundations of classical mechanics.

(Physics and Reality," Journal of the Franklin Institute, Vol.221, No.3, March, 1936. pp. 301, 307)

And here, Einstein wrote in his 1949 autobiography,

I have learned something else from the theory of gravitation: No ever so inclusive collection of empirical facts can ever lead to the setting up of such complicated equations. A theory can be tested by experience, but there is no way from experience to the setting up of a theory. Equations of such complexity as are the equations of the gravitational field can be found only through the discovery of a logically simple mathematical condition which determines the equations completely or [at least] almost completely.

("Autobiographical Notes," in Albert Einstein: Philosopher-Scientist, Ed. Paul Arthur Schilpp, 1949, p.89)

Werner Heisenberg told Einstein in 1926 that his new quantum mechanics was based only on "observables," following the example of Einstein's relativity theory that was based on the fact that absolute motion is not observable. For Heisenberg, the orbital path of an electron in an atom is not an observable. Heisenberg said of his first meeting with Einstein,

Einstein himself discovered the transition probabilities between states in the Bohr atom, ten years before this conversation with Heisenberg

I defended myself to begin with by justifying in detail the necessity for abandoning the path concept within the interior of the atom. I pointed out that we cannot, in fact, observe such a path; what we actually record are frequencies of the light radiated by the atom, intensities and transition-probabilities, but no actual path. And since it is but rational to introduce into a theory only such quantities as can be directly observed, the concept of electron paths ought not, in fact, to figure in the theory.

To my astonishment, Einstein was not at all satisfied with this argument. He thought that every theory in fact contains unobservable quantities. The principle of employing only observable quantities simply cannot be consistently carried out. And when I objected that in this I had merely been applying the type of philosophy that he, too, had made the basis of his special theory of relativity, he answered simply "Perhaps I did use such philosophy earlier, and also wrote it, but it is nonsense all the same." Thus Einstein had meanwhile revised his philosophical position on this point. He pointed out to me that the very concept of observation was itself already problematic. Every observation, so he argued, presupposes that there is an unambiguous connection known to us, between the phenomenon to be observed and the sensation which eventually penetrates into our consciousness. But we can only be sure of this connection, if we know the natural laws by which it is determined. If however, as is obviously the case in modern atomic physics, these laws have to be called in question, then even the concept of "observation" loses its clear meaning. In that case it is theory which first determines what can be observed. These considerations were quite new to me, and made a deep impression on me at the time; they also played an important part later in my own work, and have proved extraordinarily fruitful in the development of the new physics.

(Encounters with Einstein, 1983, pp.113-4)

Since philosophy has made the "linguistic turn" to abstract propositions, the problem of induction for today's philosophers is subtly different from the one faced by David Hume. It has become an epistemological problem of "justifying true beliefs" about propositions and thus lost the connection to "natural philosophy" it had in Hume's day. Information philosophy hopes to restore at least the "metaphysical" elements of natural philosophy to the domain of philosophy proper.

In contemporary logic, epistemology, and the philosophy of science, there is now the problem of "enumerative induction" or universal inference, an inference from particular statements to general statements. For example, the inference from propositions

to the general inference that

This is clearly a purely linguistic version of the original problem. Divorcing the problem of induction from nature empties it of the great underlying principle in Hume, Mill, and other philosophers, namely the assumption of the uniformity of nature, which alone can justify our "true?" belief that the sun will come up tomorrow.

In information terms, the problem of induction has been reduced, even impoverished, to become only relations between ideas. Perhaps "ideas" is too strong, much of philosophy has become merely logical relations between statements or propositions. Because of the inherent ambiguity of language, sometimes philosophy appears to have become merely a game played using our ability to make arbitrary meaningless statements, then critically analyze the resulting conceptual paradoxes.

Karl Popper famously reprimanded Ludwig Wittgenstein's claim that there are no real philosophical problems, only puzzles and language games.

On a close examination, it appears current philosophical practice has reduced the problem of induction to one of permissible linguistic deductions.

Consider these examples from the Stanford Encyclopedia of Philosophy:

1. Inductions with general premises and particular conclusions:
   • All observed emeralds have been green.
   • Therefore, the next emerald to be observed will be green.

2. Valid deductions with particular premises and general conclusions:
   • New York is east of the Mississippi.
   • Delaware is east of the Mississippi.
   • Therefore, everything that is either New York or Delaware is east of the Mississippi.

The second example is purely deductive, appropriate for philosophical puzzles, which are merely ideas, today merely relations between words. But it does not belong to the historical problem of induction. It is the problem of philosophers redefining the terms of debate!