Einstein in his later years grew pessimistic about the possibilities for deterministic continuous field theories, by comparison with indeterministic and statistical discontinuous particle theories like those of quantum mechanics.
Although Einstein initially was a strong critic of quantum theory and its implications for indeterminism and a statistical nature of reality, from the 1930’s on he never said that quantum mechanics is “incorrect” – as far as it goes – only that something else would likely be added to quantum physics in the future, making it “complete.”
As early as 1930, Einstein marveled at the logical strength of the theory, especially its formulation by Paul Dirac, “to whom, in my opinion, we owe the most perfect exposition, logically, of this [quantum] theory.”
To Leopold Infeld he wrote in 1941,
“I tend more and more to the opinion that one cannot come further with a continuum theory.”
In his 1949 autobiography (he called it his obituary) for his Schilpp volume he wrote an extensive analysis of his criticism of the quantum theory, repeating the concerns he had first developed in 1935. It is worth looking at them completely here.
I must take a stand with reference to the most successful physical theory of our period, viz., the statistical quantum theory which, about twenty-five years ago, took on a consistent logical form (Schrödinger, Heisenberg, Dirac, Born). This is the only theory at present which permits a unitary grasp of experiences concerning the quantum character of micro-mechanical events. This theory, on the one hand, and the theory of relativity on the other, are both considered correct in a certain sense, although their combination has resisted all efforts up to now. This is probably the reason why among contemporary theoretical physicists there exist entirely differing opinions concerning the question as to how the theoretical foundation of the physics of the future will appear.
Will it be a field theory; will it be in essence a statistical theory? I shall briefly indicate my own thoughts on this point.
Physics is an attempt conceptually to grasp reality as it is thought independently of its being observed. In this sense one speaks of “physical reality.” In pre-quantum physics there was no doubt as to how this was to be understood. In Newton’s theory reality was determined by a material point in space and time; in Maxwell’s theory, by the field in space and time. In quantum mechanics it is not so easily seen.
(“Autobiographical Notes,” in Albert Einstein: Philosopher-Scientist,Ed, Paul Arthur Schilpp, 1949, p.1, in German and English)
Einstein’s dream of a continuous field theory was fading fast.
Einstein wrote his friend Michele Besso 1954 to express his lost hopes for a continuous field theory like that of electromagnetism or gravitation…
“I consider it quite possible that physics cannot be based on the field concept, i.e:, on continuous structures. In that case, nothing remains of my entire castle in the air, gravitation theory included, [and of] the rest of modern physics.”
quoted in Subtle is the Lord…, Abraham Pais, 1982, p.467
The fifth edition of The Meaning of Relativity included a new appendix on Einstein’s field theory of gravitation. In the final paragraphs of this work, his last, published posthumously in 1956, Einstein wrote:
Is it conceivable that a field theory permits one to understand the atomistic and quantum structure of reality ? Almost everybody will answer this question with “no”…One can give good reasons why reality cannot at all be represented by a continuous field. From the quantum phenomena it appears to follow with certainty that a finite system of finite energy can be completely described by a finite set of numbers (quantum numbers). This does not seem to be in accordance with a continuum theory, and must lead to an attempt to find a purely algebraic theory for the description of reality. But nobody knows how to obtain the basis of such a theory.
The Meaning of Relativity, 1956, pp.165-66