I have similar feelings about working with a female, but only for one segment of our training.
We periodically have an instructor visit who will teach us some ground work. I'm personally not too happy about being on the ground with my head stuck between ANYONE'S knees while they have me in a joint lock but if I have a choice, I prefer to do this stuff with a woman. From what I gather, SOME men in our dojo prefer working with a man for groundwork. Of course, the opposite can be true too!
Other than this example, I'd have to say that it mostly makes no difference to me whether it's a male or female instructor.
Also, I hope that as younger people continue to enter the MA's they bring their belief in equality with them.