General Relativity and MTW's Three Tests for Viability

At well over twelve hundred pages, "Gravitation" (1973, ISBN 9780716703440) by Charles Misner, Kip Thorne and John Wheeler, collectively known as "MTW" for short, is one of the thicker textbooks on general relativity.

MTW's section on gravitational testing (~p1066) suggests that there's not really much doubt over the correctness of general relativity, but that we're supposed to put up alternatives for the sake of scientific procedure, so that we can make comparisons and conclude objectively that the theory is wonderful. MTW then give three conditions that a theory has to meet in order to be considered credible enough to warrant being compared with Einstein's wonderful theory.

There are a couple of problems here: the first is that the main competitor class of theory to GR1916 seems to be the "Cliffordian" acoustic metric class which doesn't reduce to flat spacetime and special relativity, and is therefore not included in MTW's scheme before we even look at the three conditions. This means that the main class of theory that could hypothetically whup GR's arse is already excluded from the comparison, because its metric is too sophisticated to fit the "clunky" SR-based defintions that act as a foundation for a lot of work on GR. All we're supposed to compare with GR is other similar SR-reducing theories, which means, essentially, other variations on the existing GR1916/60 theme.
The assessment is basically "skewed and screwed"  before we even begin.

The second problem is that even though the three critical tests seem to have been designed to make the 1916 theory look good, GR1916 still manages to fail at least one out the three, from the perspective of someone in 2015 it fails at least two out of the three, and when we look at the original theory before its  1960 reboot, that version arguably fails all three tests.

This is not good.



MTW, page 1066:

" Not all theories of gravitation are created equal. Very few, among the multitude in the literature, are sufficiently viable to be worth comparison with general relativity or with future experiments. The "worthy" theories are those are those which satisfy three criteria for viability: self-consistency, completeness, and agreement with past experiment. "

Let's examine these criteria:

1: SELF-CONSISTENCY

We know (and MTW presumably also knew in 1973) that Einstein's 1916 theory had already been found in 1960 to have failed the test of internal self-consistency. That's when the theory had a crisis in which it was discovered that the principle of equivalence, arguably the foundation of the theory, appeared to be fundamentally irreconcilable with special relativity. Einstein had warned in 1950 about potential issues related to his original "pragmatic" decision to include SR as a limiting case in 1916, subsequently arguing that this wasn't obviously a legitimate feature of a general theory. He died in 1955 before the 1960 crisis vindicated his concerns:

Alfred Schild, "Equivalence Principle and Red-Shift Measurements" Am. J. Phys. 28, 778 (1960):

" ... special relativity and the equivalence principle do not form a consistent theoretical system. "

If the principles of general relativity ruled out the inclusion of SR, we'd lose both major theories of relativity and have to rewrite a new single-stage general theory to replace both previous layers. Since the "total rewrite" option was considered unacceptable, we instead rejected the very idea that SR could be wrong, and declared that SR was an unavoidable part of any credible gravitational model. The principle of equivalence therefore had to be suspended every time it was about to collide with SR and crash the theory, because any process that crashed the theory was by definition, not a correct process under that theory.

So GR1916(original) never was a self-consistent theory, and the 1960 "reimagining" that tried to fix this cannot be said to be meaningfully self-consistent, because it only achieves a more limited form of consistency by setting up protocols for coping with failures in an orderly and consistent way. This is like the difference between a self-driving car that never crashes, and a car that crashes repeatedly, but comes with seatbelts and airbags and crumple zones, and instructions for when to override the autopilot. GR1960 does failure management rather than failure avoidance.

2: COMPLETENESS

MTW's second criterion is that

" ... it must mesh with and incorporate a consistent set of laws for electromagnetism, quantum mechanics, and all other laws of physics. ... "

At the time those words were written, it probably seemed that GR1916/60 met all those requirements, but since the theoretical discovery of black hole radiation in the Seventies, we've realised that textbook GR very much does NOT mesh with quantum mechanics.

Kip Thorne, "Black Holes and Timewarps" (1994) , p237:

" ... looking at the laws of general relativity and the laws of quantum mechanics, it was obvious that one or the other or both must be changed to make them mesh logically. "

So, assuming that current QM is basically correct, GR (past and present) also currently fails MTW test number 2. We can try to  hypothesise a larger theoretical structure – a theory of quantum gravity – that somehow contains both theories, and this is what the classical physics guys have been holding out for ... but forty years later, nobody's managed to come up with one that works without modifying GR. Current GR and QM seem to have fundamentally incompatible causal structures and definitions that would make their predictions irreconcilable, so if QM is right then we'd seem to be using the wrong general theory of relativity.

3 AGREEMENT WITH PAST EXPERIMENT

This is an odd one. It's natural to ask that a theory agree with experiment (at least reasonably well) because we need it to agree with reality. So why use the word "past"? Aren't all experiments past? Do current experiments not matter?

Setting aside that one ambiguous word, GR obviously doesn't obviously agree with all current, recent gravitational experimental data. MTW mentions "the expansion of the universe" as one of the things that  gravitational theory has to get right, and textbook GR gets expansion characteristics wrong unless we invent a new thing, "dark energy", specifically to make up the shortfall between what GR predicts and what our hardware reports.

Similarly, GR currently "under-predicts" the cohesiveness of large systems such as galaxies – the rotation curve of spiral galaxies seems to be wrong, and suggests that either the gravitational attraction within a galaxy is somehow greater, or the interaction across intergalactic voids is weaker. We'd get the second effect (and a stronger expansion characteristic) if GR was the wrong theory, and the real theory was more aggressively nonlinear, so both these things are arguably "warning flags" for the theory being faulty. Instead, we prefer to explain the extra cohesiveness by inventing a whole new form of matter ("dark matter") which obeys different rules to the rest of the universe, is conveniently invisible and unreactive (except gravitationally), and which has no theoretical basis or  reason to exist other than to help us to balance the books.

With dark matter and dark energy, we can't currently say that GR agrees well with experiment because we can't currently demonstrate that these things are real and not just ad-hoc devices that let us write "blank cheques" for any mismatch between GR and reality. GR enthusiasts might interpret the situation as meaning that once dark matter and energy are added, the theory matches the data excellently ... but they can't dispute the fact that textbook GR is currently functionally indistinguishable from a theory that fails to agree with the available data. General relativity cannot be shown to pass test #3, without making so many "creative" external ad hoc adjustments that nominally "passing" the test carries no real significance.


MTW 1973:

" Among all bodies of physical law none has ever been found that is simpler or more beautiful than Einstein's geometric theory of gravity ... as experiment after experiment has been performed , Einstein's theory has stood firm. No purported inconsistency between experiment and Einstein's laws of gravity has ever surmounted the tests of time. "

We're no longer in a position to make this statement.

CONCLUSIONS

According to MTW, failing any one of these conditions means that a theory is to be regarded as not worth pursuing and not worth testing. A sceptic could argue that the 1916 theory technically appears to fail all three tests, and with even the most optimistic and most generous interpretation of MTW's criteria, where the theory "only" fails test #2, we'd still be obliged to write off the current general theory as not being a credible theory of gravitation.

General Relativity is Screwed Up

With Einstein's general theory of relativity, one of the theory's harshest critics was probably Einstein himself. This was partly a matter of personal discipline, and partly – like the joke about sausages – because it's sometimes easier to like a thing if you don't know the gruesome details of how it was actually made. Einstein found it easy to be sceptical about the design decisions that had gone into his general theory, because he was the guy who'd made them. It had been the best general theory that had been possible at the time, said Einstein, but with the benefit of hindsight ... perhaps its construction wasn't entirely trustworthy.
The "iffy" aspects of C20th GR are difficult to see from within the theory, because – where the lower-level design decisions have forced a fudge or bodge – from the inside, these things seem to be completely valid, derived (and quite necessary) features. It's not until we look at the structure from the outside, with a designer's eye, that we see the arbitrary design decisions and short-term fudges that went into making the theory work the way it does.

Sure, the surface math looks pretty (with no obvious free variables or adjustable parameters), but that's because, as part of the theory's development, all the ugliness necessarily got moved down to the definitional and procedural structures that sit below the math. Change those underlying structures, and the surface mathematics break and reform into a different network that looks similarly unavoidable. So even though the current system looks like the simplest possible theory when viewed from the inside, we can't invest too much significance in this, because if the shape and structure was different, that'd look like the simplest possible theory, too.

To see how the theory might have been, we need to look at the subject's protomathematics, the bones and muscles and guts of the theory that dictate its overall shape, and which don't necessarily have a polite set of matching mathematical symbols.

Here are two interlinked examples of decisions that we made in general relativity that weren't necessarily correct:

Problem #1: Gravitational dragging, velocity-dependent gravitomagnetic effects

As Fizeau demonstrated back in ~1849 with water molecules, moving bodies drag light. General relativity describes explicit gravitomagnetic dragging effects for accelerating and rotating masses, and logic pretty much then forces it to describe similar effects for relative velocity, too. When you're buffeted by the surrounding gravitational field of a passing star, the impact gives you some of the star's momentum – momentum exchange means that the interaction of the two gravitational fields acts as a sort of proxy collision, and the coupling effect speeds you up a little, and slows down the star, by a correspondingly tiny amount.

For a rotating star, GR915 also agrees you're pulled preferentially to the receding side – there's an explicit velocity component to gravitomagnetism (v-gm). Even quantum mechanics seems to agree. And we can use this effect to calculate the existence of the slingshot effect, which is not just theory, but established engineering.

But v-gm effects appear to conflict with Newton's First Law of Motion: If all the background stars dragged light according to their velocity, then as you moved at speed with respect to the background starfield, the receding stars would pull on you a little bit stronger than the others, slowing you down. There'd be a preferred state of rest, that'd correspond to the state in which the averaged background starfield was stationary (ish). This doesn't agree with experience.

So the v-gm effect gets edited out of current GR, and when we do slingshot calculations, we tend to use Newtonian mechanics and model them in the time domain, instead. We compartmentalise.

Summary:

Argument: The omission of v-gm effects from general relativity seems to be arbitrary and logically at odds with the rest of the theory, but it seems to be “required” to force agreement with reality … otherwise “moving” bodies would show anomalous deceleration.

I'd consider this a fairly blatant fudge, but GR people would tend to refer to it as essential derived behaviour (based on the condition that the theory has to agree with reality).

Problem #2: Gravitational Aberration

If signals move at a finite speed, the apparent positions of their sources get distorted by relative motion. We "see" a source to be pretty much in the direction it was when it emitted the signal, with a position and distance that's out of date, thanks to the signal timelag.

If gravitational and optical signals both move at about the same speed, "c", (ignoring nonlinear complications), then we expect to "feel" the gravitational signal of a body to be coming from the same position that the object is seen to occupy. Which is kinda helpful.

But it seems that under current GR, the apparent "gravitational" position of a body gets assigned to its instantaneous position, as if the speed of gravity was infinite. We say that the speed of gravity isn't actually infinite, but that moving bodies somehow "project" their field forwards and then sideways so that it looks infinite as far as the observer's measurements are concerned. In other words, it seems that under current GR, there's no such thing as gravitational aberration.

This is a bit like the sound of fingernails scratching down a blackboard. It means that there's no longer the concept of a body having a single observed position, and we get separate definitions of "apparent position" for EM and gravity. This badly weakens the theory, because it means that mismatches between the two that that we might normally look out for to show us that we've made a mistake somewhere, are the theory's default behaviour. We lose a method of testing or falsifying the model.

So why do we do it?

We...ell, the usual argument involves planetary orbits and the apparent position of the Sun as seen by an observer on a rotating planet. But that argument's complicated and perhaps still a bit unconvincing, so … the simpler argument is that if gravitational aberration existed, it'd again seem to screw up Newton's First Law. When an astronaut travels through the universe at high speed, the background stars appear to bunch together in front of them (e.g. Scott and van Driel, Am.J.Phys 38 971-977 (1970) ), and if the gravitational effect of all those stars was shifted to the front as well, then we'd expect the astronaut to be pulled towards the region of highest apparent mass-density … forwards … and this'd further increase their forward speed, making the aberration effect even worse, which'd then create an even stronger forward pull.

So again, we manually edit the effect out, say that it's known not to exist, and then do whatever we have to do with math and language to stop the theory contradicting us.

Summary:

Argument: Losing gravitational aberration seems to be arbitrary and logically at odds with the rest of the theory, but seems to be "required" to force agreement with reality … otherwise "moving" bodies would show anomalous acceleration.

---

Put these two arguments together, and you should immediately begin to see the problem:

If we'd resisted the "urge to fudge", it looks as if our two problems would have eventually canceled each other out anyway, without our having to get involved. They seem to have the same characteristic and magnitude, but different signs. One produces anomalous acceleration, the other anomalous deceleration. Put them together and the moving astronaut doesn't accelerate or decelerate, because the stronger rearward pull of the fewer redshifted stars behind them is balanced by the increased number of stars ahead, which are blueshifted and individually weakened. Instead of our imposing N1L-compliance on general relativity as a necessary initial condition, the theory works out N1L all by itself, as an emergent property of curved spacetime.

So in these two cases, we seem to have corrupted the "deep structure" of the current general theory of relativity not once but twice, by trying to solve problems sequentially rather than letting the geometry generate the solutions for us, organically. Both "deleted" effects turn out to be necessary for a "purist" general theory … but once we'd fudged the theory once to eliminate one of them, we had to go back and fudge the theory a second time to eliminate the second effect that would otherwise have balanced it out.

And in doing that, we didn't just "double-fudge" a few details of the theory, we broke important parts of the structure that should have allowed it to expand and blossom into a larger, more tightly integrated, more strictly falsifiable system that could have embraced quantum mechanics and dealt with properly with cosmological issues. General relativity should have been a tough block of dense, totally interlocking theory, with independent multiply-redundant derivations of every feature, rather than the thing we have now.

---

The fudging of these two issues also changed some of the theory's physical predictions:

Losing gravitational aberration gave us a different set of observerspace definitions that altered the behaviour of horizons. Losing v-gm meant that we got different equations of motion, once again a different behaviour for black holes, and no way of applying the theory properly to cosmology without generating further cascading layers of manual corrections reminiscent of the old epicycle approach to astronomy. It also created a statistical incompatibility with quantum mechanics.

So general relativity in its current form seems to be pretty much screwed. GR1915 was fine as an initial prototype, but it should really have been replaced half a century ago – in 2010, it's an ugly, crippled, mutated, limited form of what the theory could, and should have been by now. But because people fixate on the math rather than on the structure, they can't see the possibility of change, or the beauty of what general relativity always had the potential to become. And that's why the subject's been almost stalled for pretty much the last fifty years, it's because Einstein died, and too many of the surviving physics people who did this stuff couldn't see past the mathematical and linguistic maze that'd developed around the subject, they didn't "get" the design principles and the dependencies between the choice of initial design decisions and the characteristics of the resulting model, and they didn't appreciate the design aesthetics.

And I find that sad on so many levels.

My Website Sucks

I know why it sucks ... it's not because I don't know how to write a proper website ... I do ... it's because it's a personal site, and I kinda tinker with it and add things from time to time, and experiment ... and because I've been using HTML for too long.

I was designing the site for someone else, I'd be less indulgent and more brutal with it. I'd insist that the owners had a clear brief of exactly what they wanted the site to do, and how to judge success. It'd be focused and lean and mean. I'd decide a visual theme, and a hierarchy, and apply it strictly. But when it's your own site, the tendency is to drift and add things and sections and use the pages as a sandbox for playing with different techniques until you end up with an indulgent hodge-podge of themes and style ideas that don't really gel.

If it was someone else's site, I'd tell them to delete the whole thing and start again. Don't just fiddle with the layout, start with a blank sheet of paper and a pen, doodle a brand new layout based on CSS, set up some default templates and rebuild the site from the ground up.

---

When you drift and add bits and pieces haphazardly, you end up slipping into old habits. I started writing webpages before we even had html tables. My first site (Erk's Relativity Pages) was a 300-page monster written entirely in Windows Notepad, and back then, a site designer had to learn all sorts of odd layout tricks (like using invisible GIFs as spacers) to produce efficient layouts. When tables were implemented by Netscape (and then by MS), we redesigned our pages to suit, with nice orderly auto-resizing panels – they were a pain to begin with, but the quirks and incompatibilities smoothed out with time, and we ended up using them everywhere. Tables became the answer to everything, from navigation panels to equation-setting. Then there was a craze for breaking a page up into sections and writing those sections as separate webpages embedded in frames. I managed to avoid that one (since I could see the long-term search-engine problems), but for a few years, using frames everywhere was supposed to be the mark of a "pro" designer. And then a couple of years later, the importance of search-engine optimisation became obvious, the fashion swung into reverse, and any frame-based sites began to look terribly dated.

Back in the 1980's and 1990's, the way to produce a flashy (but legible) site was to use a dark background with light text. The old CRT monitors tended to be strongly curved, with a display area that didn't extend quite to the edges, so a dark background made your page appear larger. With low-res CRT displays, "inverted" light-on-dark text was often easier to read, because the the outward blurring of light from the letters produced a sort of natural antialiasing effect. With dark text on pale backgrounds, the surrounding light tended to bleed over the characters, making them more difficult to read. Adding background patterning made the pages look more exciting, made the screen defects less distracting, and helped the user forget that they were staring at a fairly nasty little computer screen.

In 2010, things have flipped. Legibility isn't a problem on modern LCD displays, and because the screens are now flat, stark white rectangles actually look good. The monitor glass is thinner, so "snow blindness" due to light-scattering from large bright areas isn't so much of a problem, and you no longer need to add a faint background texture to pale or white backgrounds to disguise the "bitty" red, green and blue phosphor dots of a low-res CRT screen.

Nowadays, we practically squander space. On large screens, we use column layouts that waste most of the screen display, so that the central vertical column corresponds to what the user sees if they try to view the site from an iPhone. The web in the 1980s was content-starved, and you'd try to impress visitors with how much you had on your site and how much you could cram onto a small screen. In 2010 the visitor is spoilt for choice, so now designers try to keep things minimal and direct their visitors as quickly as possible to the information they actually want, otherwise they'll just click back to Google and try somewhere else.

After tables and frames, we now have Cascading Style Sheets. CSS is genuinely cool, and I really ought to rip up the existing pages and redo all their elderly table-based layout completely using CSS. Trouble is, it'll require a certain amount of work, and the immediate result will be that certain existing things (like same-height panels) won't work so well. There are bodges and workarounds, CSS isn't quite perfect yet.

The site's "look" also badly needs an overhaul. It was originally going to just be a few pages supporting the book, with a navy blue block across the top and down the left side referring to the book cover art (front and spine). On the subsequent pages, that morphed into a "program window" theme, with a title bar and an icon in the top left corner. I never quite worked out what to do with the spine. It's now an inconsistent mess, with pages on almost unrelated subjects like fractals, and should really be torn down and rebuilt.

---

Relativity theory is in a similar mess. A number of themes have come and gone, and left their mark on the subject. There are artefacts and traditions in the way that theory is presented that don't really make sense in the new context, and older methods that aren't compatible with newer principles. We teach special relativity as having destroyed aether theory, but we still teach SR using the length-contraction idea, which was an old aether theory concept borrowed from Lorentzian electrodynamics.

In theoretical physics, we probably have a feeling deep down that we know that we really ought to be tearing up the current system and starting again. But it'd require a lot of work without an immediate payoff, and some of the things we currently do would stop working for a while as the new system found its feet. The current system is bodgy and patched and held together with string and duct tape, but we know how to use it, and over time the bugs and fudges have started to feel like old friends. We invested a lot of time in special relativity (like website designers spent a lot of time learning the quirks of HTML tables), and now that we know that system, we tend to use it everywhere. With special relativity, we've gone further and actually redefined some key parts of relativity theory in such a way as to make SR inevitable and unavoidable, and this lock-in frees us from having to make awkward upgrade decisions.

So while it may seem that I'm sometimes a bit harsh on the theoretical physics community for being welded to obsolete and archaic systems that don't really make sense in the C21st, I do actually sympathise and empathise with their problem. They ought to rip up their SR-based structure and redesign, just as I ought to rip up my table-based webpage layout and redesign. But there's a difference between knowing that you ought to do something, and actually rolling up your sleeves and starting work, especially when there's no external deadline forcing your hand, and you always seem to have other more pressing things demanding your time.

So to help the theoretical physics community, here's a time-point. The book came out in late 2007, and sketches out the principles and the rough shape of the suggested next-generation replacement for our current general theory of relativity. This is early 2010, and the book's now been out for two years. That book is the roadmap to what comes next. So perhaps we can have a concerted start on plotting out at least a rough preliminary schedule for a replacement to general relativity, some time in 2010?

Meanwhile, I'll try to think of a way of cleaning up the website.

Projective Cosmology, and the topological failure of Einstein's General Theory

The graphic above is from my old, defunct, 1990s website, and I also borrowed it for chapter 12 of the book.

It shows a rather fun observerspace projection: if we assume that the universe is (hyper-) spherical, but we colour it in as it's seen to be rather than how we deduce it to be, expansion and Hubble shift result in a description in which things are more redshifted towards the universe's farside. Free-falling objects recede from us faster towards the apparent farside-point, as if they were falling towards some hugely massive object at the opposite end of the universe, and as if there was a corresponding gravitational field centred on the farside. At a certain distance between us and where this (apparent) gravitational field would be expected to go singular, there's a horizon (the cosmological horizon) censoring the extrapolated Big Bang singularity from view, and that looks gravitational, too.

And, funnily, enough, this "warped" worldview turns out to be defensible (as an observer-specific description) using the available optical evidence. Since we reckon that the universe is expanding, and we're seeing older epochs of the universe's history as we look further away, we're seeing those distant objects as they were in the distant past, when the universe was smaller and denser and the background gravitational field-density was greater than it is now.

Our perspective view is showing us an angled slice through space and time that really does include a gravitational gradient – between "there-and-then" and "here-and-now". The apparent gravitational differential is physically real within our observerspace projection, and viewed end-on, the projection describes a globular universe with a great big black hole at the opposite end to wherever the observer happens to be.

This projection is fascinating: it means that we end up describing cosmological-curvature effects with gravitational-curvature language, and it cuts down on the number of separate things that our universe model has to contain. If we take this topological projection seriously, some physics descriptions need to be unified. If we can agree on a single definition of relative velocity, the projection means that cosmological shifts (as a function of cosmological recession velocity) have to follow the same law as gravitational shifts (as a function of gravitational terminal velocity) ... and then, since gravitational shifts can be calculated from their associated terminal velocities as conventional motion shifts, we have have three different effects (cosmological, gravitational and velocity shifts) all demanding to be topologically transformed into one another, and all needing to obey the same laws.

---

This all sounds great, and at this point someone who hasn't done advanced gravitational physics will probably be anticipating the punchline – that when we work out what this unified set of laws would have to be, we find that they're the set given by Einstein's special and general theories, QED.

Except that they aren't. We don't believe that cosmological shifts obey the relationship between recession velocity and redshift supplied by special relativity.

We dealt with this by ignoring the offending geometry. Since cosmological horizons had to be leaky, and GR1915 told us (wrongly) that gravitational horizons had to give off zero radiation, we figured that these had to be two physically-irreconcilable cases, and that any approach that unified the two descriptions was therefore misguided. Since a topological re-projection couldn't be "wrong", it had to be "inappropriate". Instead of listening to the geometry and going for unification, we stuck with the current implementation of general relativity, and suspended the usual rules of topology to force a fit.

But then Stephen Hawking used quantum mechanics to argue that gravitational horizons should emit indirect radiation after all, as the projection predicts. So we'd broken geometrical laws (in a geometrical theory!) to protect an unverified physical outcome that turned out to be wrong. Where we should have been able to predict Hawking radiation across a gravitational horizon from simple topological arguments in maybe the 1930's, by using the closed-universe model and topology, we instead stuck with existing theory and had to wait until the 1970's for QM to tap us on the shoulder and point out that statistical mechanics said that we'd screwed up somewhere.

If we look at this projection, and consider the consequences, it suggests that the structure of current general relativity theory, when applied to a closed universe, doesn't give a geometrically consistent theory ... or at least, that the current theory is only "consistent" if we use the condition of internal consistency to demand that any logical or geometrical arguments that would otherwise crash the theory be suspended (making the concept almost worthless).
It basically tells us that current classical theory is a screw-up. And that's why you probably won't see this projection given in a C20th textbook on general relativity.

The Riemann Projection and General Relativity

The Riemann projection is associated with the mathematician Bernhard Riemann (1826-1866), and gives a method of projecting the contents of a finite spherical surface onto an infinite flat plane.

We place the sphere onto the plane, so that its South Pole is touching the surface, and then we draw lines from the North Pole to the plane. After leaving N, each line intersects one (and only one) point on the spherical surface, and one (and only one) point on the spherical plane. Every point on one on one of the two surfaces has its corresponding point on the other. As long as we don't mind making a vanishingly-small pinprick in the spherical surface at its North Pole, the two surfaces are topologically identical … we can take our pin-mark, stretch it to a finite-sized hole, and then stretch the resulting bowl-shaped surface to cover the full infinite plane.

We can also imagine this as a simple optical projection – if the sphere is a hollow transparent surface and we place a lightsource at N, then anything drawn on the sphere will project shadows onto the plane.

Einstein used the projection in his "Geometry and Experience" lecture, as an aid to visualising the idea of a closed finite universe:There's also a nice Riemann Sphere animation on YouTube, courtesy of the American Mathematical Society, and a nice image at Encyclopaedia Britannica.

---

Now although we don't usually want to make this sort of projection (unless we're working on something a bit abstract, like Moebius transformations), the "Riemann Sphere" projection was psychologically important for physics, because the thing was fairly easy to visualise, and because it had such far-reaching implications for geometrical physics.

Thanks to the projection, we know that any physics described in sphereland has to have an exact counterpart description in flatland, as long as we scale all our definitions to match. When we lay rulers over the surface of the sphere, rulers near the North Pole have projections onto the plane that tend towards becoming infinitely large, so the plane's surface appears (to its occupants) to be finite, just like the sphere. Similarly, a constant-speed light-pulse travelling around the sphere has a "shadow" on the plane whose speed tends to infinity as the corresponding position on the sphere approaches N . If we take objects and structures whose internal equilibrium is maintained by signals travelling at the speed of light, then as we move these objects away from S, they enlarge. So it takes us the same number of tiles to pave the infinite plane as the sphere. And to the plane's inhabitants, there's no obvious way of telling which tile is the central tile – the internal physics of the plane and sphere are precisely the same.

But the intrinsic geometry of a blank plane, on its own, is not the same as that of a sphere. We need to add something – a density-map. In order to recreate the sphere's properties , we need to either project a helpful scaling grid from the sphere onto the plane to describe how scalings need to vary across the plane's surface, or attach a value to each point on on the plane to describe the local scaling. This "density" parameter varies smoothly over the surface, so we're entitled to describe it as a field. We can then say that it's this density-field that deflects light and matter in the plane towards the region of highest density (S), by Huygens' principle. But as Newton and Einstein both pointed out, a variation in the density of an underlying medium, and the associated variation in the speed of light, can both be considered as expressions of the action of a gravitational field.

As a crude first approximation, we can say that the unscaled plane description includes a gravitational field that doesn't exist in the sphere description – and yet both descriptions are equivalent.

---

So ... the implication of the Riemann projection is that gravitational fields aren't absolute. We can take a physical description that works, and stretch and squash our reference-grid in weird and silly ways, and as long as we invent compensating gravitational fields that vary in sympathy with our fictitious distortions (causing space's contents to nominally stretch and squash and distort to fill exactly the same region as before), the final predictions should be identical, regardless of which grid we use.
Within a space defined by that grid, these fields are physically real. And, said Einstein, we could also run the process backwards. We can place an observer in a genuine gravitational field, and allow them freefall acceleration, and for them, that field will no longer exist in their local physics ("a freefalling observer feels no gravity"). If Eötvös' Principle (that everything falls at the same rate in a gravitational field) was right, and gravity affected everything equally, then we had to be able to produce a geometrical description of gravitational effects ... and by allowing space to be warped, we could then eliminate gravitational fields from our description as a separate effect. The background gravitational field was simply space(-time), and what we normally thought of as conventional gravity was simply the result of curvature, and of curvature-related variations in projected density.

In practice, things were a little more difficult than this: Riemann and co couldn't get their curved-space models to work using curvature in just three dimensions, so a geometrical theory of gravity had to wait until Einstein had noticed the argument for gravitational time dilation, and that it led to curvature in four dimensions.
Einstein also decided to use a "frame-based" approach, which led to some simplified geometries being cross-mapped and projected that sometimes didn't correspond to actual physics, or to the shapes that more general principles said ought to be there.

I'll deal with the topological failure of the current default version of the general theory of relativity in a future post (or two). If anyone can't wait, it's in the book.

Einstein (1950) - Special Relativity is Not Fundamental

Albert Einstein, 1950 (Scientific American):

"[re: the general principle of relativity] ... without this ... it would be practically impossible for anybody to hit on the gravitational equations, not even by using the principle of special relativity ...

... This is why all attempts to obtain a deeper knowledge of the foundations of physics seem doomed to me unless the basic concepts are in accordance with general relativity from the beginning. This situation makes it difficult for us to use our empirical knowledge, however comprehensive, in looking for the fundamental concepts and relations of physics, and it forces us to apply free speculation to a much greater extent than is presently assumed by most physicists.

I do not see any reason to assume that the heuristic significance of the principle of general relativity is restricted to gravitation and that the rest of physics can be dealt with separately on the basis of special relativity, with the hope that later on the whole may be fitted consistently into a general relativistic scheme. I do not think that such an attitude, although historically understandable, can be objectively justified. The comparative smallness of what we know today as gravitational effects is not a conclusive reason for ignoring the principle of relativity in theoretical investigations of a fundamental character. In other words, I do not believe that it is justifiable to ask: what would physics look like without gravitation? "

---

Some people really don't like the implications of what Einstein seemed to be saying here about his own theories. Some online physics people in the past have insisted that Einstein couldn't possibly have said such a thing, but I've checked paperback reprints and the original 1950 SciAm publication and there it is, in black and white. That's what he said.

Einstein's point is entirely logical, and supported by the historical record. If we look at the background to the development of relativity theory, notably the Newtonian Catastrophe, we find that the version of general relativity that we ended up with, with special relativity providing an underlying flat-spacetime layer, was not just not inevitable, it actually seemed to depend on a rather unlikely-looking chain of historical accidents. Probablistically, it really shouldn't have happened like this.
We should have understood that gravity slowed time a century before Einstein eventually noticed, shortly after John Michell had predicted gravitational shifts way back in 1783. With the benefit of that missing piece of information, Gauss and Riemann and Clifford and friends ought to have been able to complete their curved-space projects (allowing curvature in four dimensions rather than just three) and should have been able to produce a general theory of relativity in the Nineteenth Century, before special relativity had been thought of. Einstein's special theory was partly a reaction against the aether models that dominated in the absence of a proper curvature-based description, and when Einstein went on to try derive his more ambitious general theory a few years later, he naturally wanted it to incorporate his earlier theory.

But special relativity assumed inertia without gravity, and energy-concentration without curvature. Its founding geometrical principles are fundamentally incompatible with key results that arise from the general principle of relativity. Like Einstein said, historically understandable, but not objectively justifiable with the benefit of hindsight.

General Relativity and Nonlinearity

One of the difficulties set up by the structure of Einstein's general theory of relativity is the tension between GR's requirement that there be no prior geometry, and the assumption that the geometry must necessarily reduce to the flat fixed geometry of special relativity's Minkowski metric as a limiting case over small regions.

Although it's not news that GR shouldn't presume a prior geometry (GR's fields are not superimposed on a background metric, they define the metric), this is one of those irritating principles that's easier to agree with in principle than it is to actually implement.
It's only human nature when attacking a problem to want to start off with some sort of fixed point or known property that everything else can be defined in relation to. It's like starting a jigsaw by identifying the four corner pieces first. We tend to start off by imagining the shape of the environment and then imagining placing a test object within it ... but the act of placing an observer itself modifies the shape and characteristics of the metric, and means that the signals that the observer intercepts might have different characteristics to those that we might otherwise expect to have passed though the particle's track, if the particle hadn't actually been there, or if it had been moving differently. Although the basic concept of a perfect "test particle" isn't especially valid under relativity theory, we like to assume that the shape of spacetime is largely fixed by large background masses, and that the tiny contribution of our observer-particle won't change things all that much (we like to assume that our solutions are insensitive to small linear "perturbations" of the background field).

Unfortunately, this assumption isn't always valid. Even though the distortion caused by adding (say) a single atom with a particular state of motion to a solar system may well be vanishingly small, and limited to a vanishingly-tiny region of spacetime compared to the larger region being looked at, every observation that the atom and solar system make of each other will be based on the properties of exchanged signals that all have to pass through that teensy-weensy distorted region. So if we build a theory on mutual observation and the principle of relativity, even a particle-distortion or gravitomagnetic distortion that's only significant in over a vanishingly small speck of spacetime surrounding the atom still has the potential to dramatically change what the atom sees, and how outsiders see the atom. It changes the properties of how they interact, and by doing that, it also changes the characteristics of the physics. Although a star isn't going to care much whether an individual distant atom makes a tiny distortion in spacetime or not, our decision as to whether to model that distortion or not can change the functional characteristics of our theory, and change the way that we end up modelling the star, and some of the predictions that we make for it. It also has the potential to wreck the validity of the frame-based approach that people often use with general relativity – if we take nonlinearity seriously, we should probably be talking about the relativity of object views, rather than the relativity of "frames".

Field components aren't always guaranteed to combine linearly, they can twist and impact and writhe around each other in fascinating ways, and generate new classes of effect that didn't exist in any of the individual components. For instance, if we take a bowling ball and a trampoline, and place the ball on the trampoline, their combined height is less than the sum of the two individual heights, and the trampoline geometry has some new properties that aren't compatible with its original Euclidean surface. The surface distorts and the rules change. [Ball+ Trampoline] <> [Ball] + [Trampoline].
Or, place a single bowling ball on an infinite trampoline surface and it settles down and then stays put. But place two bowling balls on the surface, reasonably near to each other, and the elastic surface will push them towards each other in an attempt to minimise its stresses and surface area, producing relative motion. A one-ball model is static, a two-ball model is dynamic, so the rules just changed again.
The result of assuming a background field and simply overlaying particles isn't guaranteed to be the same as a more realistic model in which the particles are intrinsically part of the background field. Nonlinear behaviour generates effects that often can't be generated by simple overlay superimpositions.

Einstein's special theory of relativity rejects the idea of any such interaction between a particle and its surrounding spacetime, so this class of nonlinear effect is incompatible at the particle level with our current general theory of relativity (which is engineered to reduce to SR). While we understand that perhaps a fully integrated model of physics can't be broken up into self-consistent self-contained pieces that can be modelled individually and then assembled into a whole, we try it anyway, because it's easier to tackle smaller bite-size theories than to try to create the full Theory of Everything from scratch. And when we work on these isolated theories, and try to make them internally consistent without taking into account external factors, we end up with a series of theoretical building blocks built on different principles that don't fit together properly.

For Einstein's general theory of relativity, we say that the theory must reduce to the flat-spacetime physics of special relativity over small regions, which makes the theory pretty much incompatible with attempts to model particle-particle interactions as curvature effects. But if what we understand as "physics" is the result of particle-observers communicating through an intermediate medium, and the geometrical properties of those particles on the metric is an intrinsic part of how they interact – if physics is about nonlinear interactions between geometrical features – then by committing to special relativity as a full subset of GR, we might have guaranteed that our general theory can never describe the problem correctly, because any solution with a chance of being right will be ruled out for being in conflict with special relativity. Since deep nonlinearity (which GR1915 doesn't have) seems to be the key to reproducing QM behaviour in a classically-based model, it's not surprising that serious attempts to try to find a way to combine GR and QM have tended to run into the nonlinearity issue:

Albert Einstein, 1954

at the present time the opinion prevails that a field theory must first, by "quantization", be transformed into a statistical theory of field probabilities ... I see in this method only an attempt to describe relationships of an essentially nonlinear character by linear methods.

Roger Penrose, 1976, quoted by Ashtekar:

... if we remove life from Einstein's beautiful theory by steam-rollering it first to flatness and linearity, then we shall learn nothing from attempting to wave the magic wand of quantum theory over the resulting corpse.

Some GR researchers did try to move general relativity beyond a reliance on a fixed initial geometry and dimensionality (see John Wheeler's work on pregeometry), but the QM guys were better at analysing where their "perturbative" and "nonperturbative" approaches differed than the GR guys were at identifying the artefacts that special relativity might have introduced into their model.

In order to work out what parts of current GR might be artefacts of our approach, it's helpful to look at non-SR solutions to the general principle of relativity, and compare the results with those of the usual SR-based version.
The two approaches give two different sorts of metric. If we embrace nonlinearity, we get a relativistic acoustic metric and a general theory that supports Hawking radiation, classically. The second approach (where we start by assuming that a particle's own distortion is negligible and doesn't play a role in what the particle sees) gives us standard classical theory, Minkowski spacetime, the current version of general relativity, and a deep incompatibility with Hawking radiation and quantum mechanics.

So I'd suggest that perhaps we shouldn't be trying to reconcile "current GR" with quantum theory ... we should instead be trying to replace our current crippled version of general relativity with something more serious, that didn't rely on that additional SR layer. There seem to have been two different routes available to us to construct a general theory of relativity, and it's possible that we might have chosen the wrong one.