If you want to be fit there are many alternatives. Of course the martial arts are a very good way to stay in shape, most of the time, but self defense as mentioned is really not exactly related to being fit.
Of course being physically able is necessary to defend yourself but self defense is mostly a mindset.
If you want to achieve a capability to defend yourself through martial arts you will need to find a good teacher that specifically suits you and put in a good deal of commitment - something you wouldn't achieve if you just take it as a 2-3 times a week thing to stay fit.