XLR connectors are used primarily for balanced signals.  Pin 2 is + (or normal phase), Pin 3 is - (or inverted phase) and pin 1 is signal ground.  There are two identical signals being used here, 180 degrees out of phase with each other.  The major benefit of balanced is Common Mode Noise Reduction (CMNR) and the added benefit of not relying on a ground since the two signals created the voltage potential due to thier phase differential (you won't see many ground loop noise problems).

RCA connectors are used primarily for single ended (non-balanced) signals.  The center pin is the signal and the outer connector shell is signal ground.  There is one discrete signal being used here, referenced to ground.  Single ended can be cheaper to implement some of the time.