okay I tried the screwdriver and still doesn't fire it right off also as a side note I have a solenoid with two red wires (cables) on the positive post and the black ground on the other postI suspect you have a Briggs Single cylinder engine with an inertia type starter. You probably having a voltage drop problem at the starter. Would suspect the remote starter solenoid of having burned internal contacts.
You can try jumping the two large posts of the remote starter solenoid with a junk screw driver and see if the starter engages every time. If it does then replacing the solenoid may very well fix the problem.