Just out of curiousity, how come olympic sailing teams don't wear lifejackets? Is it not mandatory? Or do they think they'll never capsize or fall off the boat? You can see from the pictures here:
Top level sailors often wear a PFD under an outer layer. I just looked at some video of the 2008 British Yngling team where they are wearing PFDs underneath.
As an aside: an Yngling is a keelboat and there are a lot of rescue and other boats on the water at the Olympics.
Finally, sane people do wear a PFD on keelboats when help wouldn't be immediately available and when the conditions get a bit rough.
photos I've seen show dinghy racers all wearing PFD'S, usually , as was pointed out, worn under a tight fitting shirt. This I suspect minimizes the chances of anything in the rigging snagging on the PFD. To go out in any boat and not wear a PFD is idiocy, what if you hit the water unconcious, injured or the water is very cold? If you can't be rescued within minutes your history.