If the engine is not cranking:
I just had a similar situation. Firsthing I did was bypass the solenoid with a jumping cable and that seemed to work for about 20 consecutive starts and then that stopped working. So I went to remove the starter from the engine and found that the positive post on the starter was loose. it built up carbon between the post and the nut and the wire. so a quick wirebrushing and tightening of the post nut, wirebrushing the contact of the wire and I was back in business.
Since that happened, I had severl starters that I had replaced for neighbors sitting in my junk pile, I took each one, opened them up, cleaned up real good tightened that post nut up and now 3 out of 5 are working starters again. 2 of them need new "brushes".. So it seems to be an issue more often then not.. at least 3 out of 5 times from last season..