The cosmological constant problem is one of the biggest crises in physics Premium

The calculated dark energy content of the universe, based on theory, is bizarrely off the mark. In the simplest estimate, there should be enough energy in a cube with sides of length 10^-21 cm to unbind the entire Milky Way, yet in reality there appears to be much, much less. Nobody knows a convincing way to get around this. This is to say that while the universe is observed to be incredibly big, physicists calculate that it must be tinier than a proton. This, in a nutshell, is the cosmological constant problem.

Can you get rid of all the energy in the room in which you are reading this? Move the room far from the earth’s gravity, then toss out everything made of matter so you can cross off mass, kinetic, and potential energies. Also pump out the air, the cosmic rays constantly streaming in, and the fog of neutrinos that were made during the Big Bang. Next, kill the energy in photons: darken the room completely, and clear away the microwave radiation left over from the infant universe.

Now all that’s left in your room is empty space, and no energy – right? No! Your room still has “dark energy”. In a patch of space the size of your room, dark energy is so scant that it is nearly impossible to detect. But across the cosmos, which is full of space, it contributes a titanic 70% of all energy. Matter, in the form of stars, gas, and the mysterious dark matter, supplies most of the other 30%, while radiation in the form of photons and neutrinos chips in 0.01%.

The basic truth about space that Albert Einstein taught us is that it is not a state of ‘nothingness’. Instead, it is a bendable, stretchable medium that we occupy, much like water is for fish. Add energy uniformly across a patch of space and that patch will expand (or shrink, if the energy is negative). Each form of energy tells the universe how to expand in its own way. This is much like how you can inflate a balloon with air, water or sand, and in each case it will have a characteristic look and feel.

Since dark energy dominates the energy budget of the universe, it also dictates the rate at which space expands. We can reverse-engineer this fact to estimate how much dark energy is present in any volume of space, by considering the size and age of the universe. Add too much positive energy and the cosmos would expand too fast: galaxies would fly away from us faster than light, so that only the regions of the universe nearest to us will be visible. Effectively, this “observable universe” would appear to shrink. Add too much negative energy and the universe would actually shrink to a tiny point. The greater this negative energy, the sooner this event.

Everyone can agree that the universe is larger than India and older than the Indus Valley Civilisation. These facts alone restrict the density of dark energy to the caloric content of a pinch of sugar in a cubic metre. In reality, the universe is wider than billions of lightyears and older than 10 billion years, so the dark energy is actually as dilute as one sugar crystal in a cubic kilometre.

And here is the crisis: the calculated dark energy content of the universe, based on theory – a bread-and-butter particle physics calculation – is bizarrely off the mark. In the simplest estimate, there should be enough energy in a cube with sides of length 10-21 cm to unbind the entire Milky Way, yet in reality there appears to be much, much less. Nobody knows a convincing way to get around this. This is to say that while the universe is observed to be incredibly big, physicists calculate that it must be tinier than a proton. This, in a nutshell, is the cosmological constant problem, and it has come to be called rightly as “the worst theoretical prediction in the history of physics”.

How is the amount of dark energy predicted from just theory? To begin with, particle theorists have a pretty sharp notion of what dark energy is composed of. (This situation is different from that of dark matter, whose identity is a total mystery.) There are three unavoidable quantities that behave exactly like dark energy.