None should be noticeable, but with tapered roller bearings any wear will start showing up in up and down motion in the shaft. Since there is no way of adjusting the end play like wheel bearings on a car, you either have to replace them or live with them until they get worse and with proper grease may take years.
Slight up and down should be fine, any side to side motion should be replaced.