I've had both the 4" XD Service and 3" sub-compact and eventually went with the Service modle for concealed carry use.
My reasons for this were thus;
The extra 1" of barrel length is pretty much not noticed when carrying but makes a bit of difference in accuracy and there is a much greater difference in terminal ballistics (what happens at the target) than one would think with an inch difference in barrel length. The big problem area for concealment is in the grip area and this is where I made my decision. The grip on the subcompact came with a finger extension which is great from a controllability and accuracy standpoint as you can use all your fingers on the grip which helps to shoot the gun tremendously. The problem is, it extends the grip length out to the same as with the Service model which makes concealability a little more difficult at the cost of several rounds. Replacing with a flat base resolved the length issue but left the little finger dangling in the wind. So, I dumped the sub for the Service and find this to be a very good choice for me. For those times I feel concealment is more important than controlability, I carry a Kel-Tec in either 40 S&W or 9mm. It is smaller than most paperback books and still carries 10 rounds with a magazine extension. It does have a very heavy trigger, the sights are crude, and it is no long range shooter but I can hide it anywhere.
I conceal my Service XD under most types of clothing ranging from shorts and an untucked T-shirt to dress clothes. Wear slightly loose fitting clothes in medium colors and the gun should hide fairly well. I find that one can't wear white often as a dark gun butt will show through unless the material is fairly thick and coarse. the wearing of a white T-shirt underneath can make the gun show up even more if the shirt material is the least bit shear. Real dark colors like black, dark blue, and burgandy allow the gun to print a bit more but again, if loose fitting, few pick up on it.
I wear an inside waistband (IWB) holster the most frequently and have several I switch off between depending on the circumstances. The three I use the most are from Galco, C-Tac, and Blackhawk. They each have a couple of models that one can choose from.
As for caliber, I had both in 40 S&W but 9mm is effective, easier to shoot, and cheaper to shoot. With good personal defense ammo, it would be just fine as a defensive round. The 40 S&W has a fair bit of twist and recoil to it which can make it a bit harder to shoot decently, another reason I like the larger model.
