250 million years ago, the land that is now the Guadalupe Mountains was under the water of a shallow sea. The mountains were created when countless organisms, such as seashells, algae and sponges, died and fell to the bottom where the limy parts of their bodies were "cemented" together forming a huge underwater reef. Fast forward to 10-12 million years ago -- the sea was long gone, but the reef, now buried under layers of sediment, was uplifted and the sediments eroded to reveal it once again. Guadalupe Peak, just visible in this image as the lighter peak directly behind the dark foreground peak, is the highest point in Texas (8749 feet).
Carlsbad Caverns also developed from this reef. As the reef was pushed upwards, rainwater mixed with hydrogen sulfide-rich water from oil and gas fields in the area to form sulfuric acid, which ate away the limestone of the reef and created the caverns we see today.
Once the caves were formed, drops of water infused with carbon dioxide (a weak acid) picked up minute amounts of calcite from the limestone and, as it trickled through the rock into the caverns, deposited this calcite to form both the stalactites (the formations hanging from the ceiling of the cave), and, from the solution that dripped off the end-points of the stalactites, the stalagmites (the formations rising towards the stalactites). Billions and billions of drops later we see these rock formations that have now taken the shape of cones, pillars, and slender, fragile straws.
Here are more images from inside Carlsbad Caverns: