My thought is a breeze above 5 mph will get you going. 8 mph might be better. A Sunfish will sail in very light wind, but below about 8 the skill required actually goes up. For someone new to Sunfish 8 to 12 is a good weather window to look for.
If there is a current where you are planning to sail, watch out for that. When the wind is low, a counter current can sweep you backwards
Same for tides....
PS: I have found myself racing in 0-5 mph winds, but that's hardly fun.