schools that are well-established or doing excellent martial arts aren't going to get many challenges. It's usually the "newbies" on the block that have challengers dropping in, or the occasional "ronin" that runs around dropping challenges in an area until somebody turns them into a pretzel.
When I had a school open on my own, we had a few "challengers" that dropped by and left with their tails between their legs and humble, but once word got out that we did traditional martial arts and weren't trying to steal any of the McDojo's students, it ran its course. We had several schools in the same association that made some bad comments about us, but it was personal animosity that caused it, not anything we did to cause it.
We loved training at, and with other schools, and did it all the time. Once the instructors figure out you aren't trying to snatch their students, they usually get "friendlier"...