In the book, Wiener helped define the new quantitative concept of information coming out of the work of John von Neumann, Alan Turing, and Claude Shannon on computers and communications.

As Leo Szilard had done twenty years earlier, Wiener emphasized the information gained in choices between two equiprobable alternatives, which produce "bits" (binary digits) of information.

One and all, time series [of experimental data] and the apparatus to deal with them, whether in the computing laboratory or in the telephone circuit, have to deal with the recording, preservation, transmission, and use of information. What is this information, and how is it measured? One of the simplest, most unitary forms of information is the recording of a choice between two equally probable simple alternatives, one or the other of which is bound to happen — a choice, for example, between heads and tails in the tossing of a coin. We shall call a single choice of this sort a decision. If then we ask for the amount of information in the perfectly precise measurement of a quantity known to lie between A and B, which may with uniform a priori probability lie anywhere in this range...

We may conceive this in the following way: we know a priori that a variable lies between 0 and 1, and a posteriori that it lies on the interval (a, b) inside (0, 1). Then the amount of information we have from our a posteriori knowledge is

-log₂ (measure of (a, b) / measure of (0, 1))

Wiener's negative of the entropy led Leon Brillouin to coin the term negentropy

The quantity we here define as amount of information is the negative of the quantity usually defined as entropy in similar situations. The definition here given is not the one given by R. A. Fisher for statistical problems, although it is a statistical definition; and can be used to replace Fisher's definition in the technique of statistics.

(Cybernetics, 2nd edition, pp.61-62)

As a final remark, let me point out that a large computing machine, whether in the form of mechanical or electric apparatus or in the form of the brain itself, uses up a considerable amount of power, all of which is wasted and dissipated in heat. The blood leaving the brain is a fraction of a degree warmer than that entering it. No other computing machine approaches the economy of energy of the brain. In a large apparatus like the Eniac or Edvac, the filaments of the tubes consume a quantity of energy which may well be measured in kilowatts, and unless adequate ventilating and cooling apparatus is provided, the system will suffer from what is the mechanical equivalent of pyrexia, until the constants of the machine are radically changed by the heat, and its performance breaks down. Nevertheless, the energy spent per individual operation is almost vanishingly small, and does not even begin to form an adequate measure of the performance of the apparatus. The mechanical brain does not secrete thought "as the liver does bile," as the earlier materialists claimed, nor does it put it out in the form of energy, as the muscle puts out its activity. Information is information, not matter or energy. No materialism which does not admit this can survive at the present day.

(Cybernetics, 2nd edition, p.132)

In his book "The Human Use of Human Beings," the Cybernetics founder saw the Devil himself increasing entropy everywhere. But he also saw the flow of negative entropy from the Sun that is the source of all life and mind on the Earth.

The Maxwell demon

(The Human Use of Human Beings, p.44)

We are immersed in a life in which the world as a whole obeys the second law of thermodynamics: confusion increases and order decreases. Yet, as we have seen, the second law of thermodynamics, while it may be a valid statement about the whole of a closed system, is definitely not valid concerning a non-isolated part of it.

Processes that decrease the entropy locally we call "ergodic"

There are local and temporary islands of decreasing entropy in a world in which the entropy as a whole tends to increase, and the existence of these islands enables some of us to assert the existence of progress. What can we say about the general direction of the battle between progress and increasing entropy in the world immediately about us?

(The Human Use of Human Beings, p.44)

In physics, the idea of progress opposes that of entropy, although there is no absolute contradiction between the two. In the forms of physics directly dependent on the work of Newton, the information which contributes to progress and is directed against the increase of entropy may be carried by extremely small quantities of energy, or perhaps even by no energy at all. This view has been altered in the present century by the innovation in physics known as quantum theory.

Quantum theory has led, for our purposes, to a new association of energy and information. A crude form of this association occurs in the theories of line noise in a telephone circuit or an amplifier. Such background noise may be shown to be unavoidable, as it depends on the discrete character of the electrons which carry the current; and yet it has a definite power of destroying information. The circuit therefore demands a certain amountof communication power in order that the message may not be swamped by its own energy. More fundamental than this example is the fact that light itself has an atomic structure, and that light of a given frequency is radiated in lumps which are known as light quanta, which have a determined energy dependent on that frequency.

Thus there can be no radiation of less energy than a single light quantum. The transfer of information cannot take place without a certain expenditure of energy, so that there is no sharp boundary between energetic coupling and informational coupling. Nevertheless, for most practical purposes, a light quantum is a very small thing; and the amount of energy transfer which is necessary for an effective informational coupling is quite small. It follows that in considering such a local process as the growth of a tree or of a human being, which depends directly or indirectly on radiation from the sun, an enormous local decrease in entropy may be associated with quite a moderate energy transfer. This is one of the fundamental facts of biology; and in particular of the theory of photosynthesis, or of the chemical process by which a plant is enabled to use the sun's rays to form starch, and other complicated chemicals necessary for life, out of the water and the carbon dioxide of the air.

Thus the question of whether to interpret the second law of thermodynamics pessimistically or not depends on the importance we give to the universe at large, on the one hand, and to the islands of locally decreasing entropy which we find in it, on the other. Remember that we ourselves constitute such an island of decreasing entropy, and that we live among other such islands. The result is that the normal prospective difference between the near and the remote leads us to give far greater importance to the regions of decreasing entropy and increasing order than to the universe at large. For example, it may very well be that life is a rare phenomenon in the universe; confined perhaps to the solar system, or even, if we consider life on any level comparable to that in which we are principally interested, to the earth alone. Nevertheless, we live on this earth, and the possible absence of life elsewhere in the universe is of no great concern to us, and certainly of no concern proportionate to the overwhelming size of the remainder of the universe.

(The Human Use of Human Beings, p.44)

What I say about the need for faith in science is equally true for a purely causative world and for one in which probability rules. No amount of purely objective and disconnected observation can show that probability is a valid notion. To put the same statement in other language, the laws of induction in logic cannot be established inductively. Inductive logic, the logic of Bacon, is rather something on which we can act than something which we can prove, and to act on it is a supreme assertion of faith. It is in this connection that I must say that Einstein's dictum concerning the directness of God is itself a statement of faith. Science is a way of life which can only flourish when men are free to have faith. A faith which we follow upon orders imposed from outside is no faith, and a community which puts its dependence upon such a pseudo-faith is ultimately bound to ruin itself because of the paralysis which the lack of a healthily growing science imposes upon it

(The Human Use of Human Beings, pp.263-4)

The succession of names Maxwell-Boltzmann-Gibbs represents a progressive reduction of thermodynamics to statistical mechanics: that is, a reduction of the phenomena concerning heat and temperature to phenomena in which a Newtonian mechanics is applied to a situation in which we deal not with a single dynamical system but with a statistical distribution of dynamical systems; and in which our conclusions concern not all such systems but an overwhelming majority of them. About the year 1900, it became apparent that there was something seriously" wrong with thermodynamics, particularly where it concerned radiation. The ether showed much less power to absorb radiations of high frequency—as shown by the law of Planck—than any existing mechanization of radiation theory had allowed. Planck gave a quasi-atomic theory of radiation—the quantum theory—which accounted satisfactorily enough for these phenomena, but which was at odds with the whole remainder of physics; and Niels Bohr followed this up with a similarly ad hoc theory of the atom. Thus Newton and Planck-Bohr formed, respectively, the thesis and antithesis of a Hegelian antinomy. The synthesis is the statistical theory discovered by Heisenberg in 1925, in which the statistical Newtonian dynamics of Gibbs is replaced by a statistical theory very similar to that of Newton and Gibbs for large-scale phenomena, but in which the complete collection of data for the present and the past is not sufficient to predict the future more than statistically. It is thus not too much to say that not only the Newtonian astronomy but even the Newtonian physics has become a picture of the average results of a statistical situation, and hence an account of an evolutionary process. This transition from a Newtonian, reversible time to a Gibbsian, irreversible time has, had its philosophical echoes. Bergson emphasized the difference between the reversible time of physics, in which nothing new happens, and the irreversible time of evolution and biology, in which there is always something new. The realization that the Newtonian physics was not the proper frame for biology was perhaps the central point in the old controversy between vitalism and mechanism; although this was complicated by the desire to conserve in some form or other at least the shadows of the soul and of God against the inroads of materialism. In the end, as we have seen, the vitalist proved too much. Instead of building a wall between the claims of life and those of physics, the wall has been erected to surround so wide a compass that both matter and life find themselves inside it.

Newton is deterministic, Gibbs is statistical, but still deterministic, just unpredictable.
Ananke (necessity) and Tyche (chance) are the standard argument against free will.

It is true that the matter of the newer physics is not the matter of Newton, but it is something quite as remote from the anthropomorphizing desires of the vitalists. The chance of the quantum theoretician is not the ethical freedom of the Augustinian, and Tyche is as relentless a mistress as Ananke.

(Cybernetics, 2nd edition, p.38)

"Progress and Entropy," chapter 2 from The Human Use of Human Beings