Any time I go to install anything that does not say specifically "64 bit", Windows won't allow me to install it, and aborts.
I've only heard of this for like one other person, but it doesn't make any sense, especially for FF4. You have tried the latest version? I've only had issues with one or two 32bit programs that have not installed properly on W7 x64 and they were like 10 years old. Make sure you run all setups as Administrator, you could even run in compatibility mode, see if that helps.
If you try to install and the program just disappears after extraction, then you can use 7Zip to unzip the installation files to the Desktop (or wherever). You can then run the setup from there as normal.
You should not need to do any of that, but maybe your installation of Windows was screwed up from the start or got corrupted, a reformat would probably take care of it, because if you check microsofts site, FF 32 bit is supported on 64 bit systems. I have been running 64bit Windows from XP, Vista, and now 7 and have not had any issues (I have 2 personal computers, one laptop, one media center, and a work computer all running x64.)
Again, if you can't do it for any odd reason, make sure you download the latest IE in x64 format, I think its pretty good (
Internet Explorer downloads - Microsoft Windows)