no pulley ever has to be destroyed, if you are willing to spend the time.
Worst case is you have to heat it red hot to break the rust bond.
Like water that gets into cracks in rocks, freezes then splits the rock, water gets into the cavity between the pulley & the shaft then it causes rust.
Rust occupies more space than steel , which is why you get rust "bubbles" forming under paint , so it locks the shaft on to the pulley.