
And to answer your questions: applications contain either one or the other, not both. Some are 32-bit, some are 64-bit. For most uses the benefits of 64-bit are limited (have you seen a unit that needs more than 2^32 MB (4GB) of RAM before, other than on bigadv?). However in the scenarios where 4GB or more of RAM is needed it will be necessary to have 64-bit versions.
for reference, 64-bit OSes I use (none of which have displayed the phenomenon you describe):
Vista Home Premium x64
7 Home Premium x64
7 Professional x64
Ubuntu/Kubuntu 9.04 x64