Simple terms - a hunk of metal (or other thing) starts out at ambient temperature. It's exposed to other another temperature that warms (or cools) the surface. As time goes on, the head (or cold) evens out throughout the metal and the thing is "heat soaked". If you take away the source of the heat (or cold), the part will remain hot (or cold) for quite some time as it gives off the heat (or cold) that has "soaked" in the metal.
In terms of intercoolers, as the compressed air is compressed, it gets warmer. As it passes through the intercooler, the air gives off heat to the intercooler. And the intercooler gives off that air to the outside air that passes over the intercooler. After a while, the intercooler "heat soaks" and can't absorb as much heat from the compressed air to cool it. The temperature of the compressed air to the engine goes up and that increases the chance of detonation in the cylinder.
Sitting idle (with no air passing over the cooler) can also raise the "heat soak" causing loss of performance (or detonation), but it will reduce the "heat soak" as the car is moving with fresh (relatively) cool air passing over the intercooler again.
The same thing happens with cooling systems. The combustion chamber temperatures can get quite high, and the coolant passing through the cylinder head carries away the heat. But when you turn off the engine, there is still a high amount of residual heat in the chamber material that has not been removed. That heat "soaks" the head (and the coolant) and makes localized hot spots - the normal coolant temperature may be 195 degrees, but that local hot spot can easily get to be 20 - 50 degrees hotter. That's why many cars have a fan that will kick on after the car is parked if it detect that the engine is too hot. The Elise/Exige has a small pump that is turned on and cycles the coolant in the head through the heater circuit to dissipate the heat.