games of soccer etc are commonplace to non ma minded people, while a sword contest is a bit out of the ordinary,
they may have been interested in what you were doing and wish to try it themselves someday, - why would it matter if you are being watched?
the bottom line is that in a public area you stand a very good chance to run into members of the public