Was just recalling one of the worst calls I ever got in IT...

Many years ago we had a single rack for all of our servers, network and storage (pre virtualization too!).

We had a new security system installed in the building and the facilities manager let the guy into the server room to run all the sensor cables in because that is where they wanted their panel... the guy was too lazy to get up on the roof and in the attic repeatedly so after he checked it out he went around every where and drilled a hole straight up where he wanted the sensor wire to go... well the server room was not under an attic space... when he found he had drilled through to the out side... HE FILLED IT WITH EXPANDING FOAM.... the membrane on the roof was damaged... that night it rained... I got a call at 4 am that systems were acting funky and I went in... when I opened the door it was literally raining through the corners of the drop ceiling onto the rack... An excellent DR plan saved our asses but the situation cost the vendor's insurance company $30k in dead equipment and another $10k in emergency labor. Good thing for him we had so little equipment in that room back in.

Moral of the story... always have a good DR plan... you never know when it will rain in the server room.... :)