I have found that stories of this nature are often times just propaganda to inflate what ever style it makes look good. A prominent MA of the time is stated to have come to whatever school to fight, loses, and goes home with his tail between his legs. The school which "beat" him, uses this as a motivating tool for students and a draw to pull new students in. Of course when the confrontations happen, it was a private match between these two where no one else was admitted due to it being a private matter between these two rivals. Sorry, not convinced.
One instructor that I had trained with was supposedly good at fighting because of the many fights he had as a street thug where he perfected his art, some were to the death.
There are so many similar stories floating around that it is hard to know what to believe but I have a difficult time believing that Mas Oyama couldn't lay a finger on someone???? And that now disgusted from his attempts, he admits defeat??? Yeah, that coincides with every thing I have ever read about the man. He sounds like someone who would quit a fight and give up out of frustration...I don't think so.