A headphone jack typically has a mechanical switch that breaks when you insert a plug, this is to do exactly what you'd think - it breaks the audio path to the final output drive for the speakers and routes it to the headphones. Depending on how the jack was made and how sloppy it is, you can get the plug to make connection with the audio path and not break the switch to the speaker drive. There are some devices that have better parts or rely on a more electronic method like impedance of the circuit connected to determine what's in use and to turn off the appropriate output.