sorry im late to the party, but did you remove the second set screw? the one that is there to prevent the pulley from sliding off of the crankshaft?
the key slot is tapered and gets shallower at the bottom so the key cannot move down and out
also measure and record exactly how far the plate is above the engine case or your drive pulley wont get the correct traction when you reassemble it. a 64th of an inch difference can be all it takes to cause the mower drive to slip doing up an incline