There isn't a 120gb 5th gen, so either you have a 6th gen or it's an already-modded device. 5th and 6th gen are very different from rockbox's point-of-view. The problem you describe seems more plausible in a 6th gen, I don't really see how this could happen with a 5th gen. The installation with a 6th gen is more complicated.
I don't understand what you mean when you say 'the tool installs the firmware' and 'the tool won't find my device'. Surely in order to install the firmware it must have found the device? Are you saying it loses the ability to see the ipod once the firmware is installed? What does windows explorer (assuming you are using windows, you don't say which OS you have) see? And likewise device manager?
Once you write rockbox to the ipod, do you reboot in rockbox mode? What happens if you write rockbox directory but then reboot in of mode? I can't see any reason why the ipod would become undetectable by the computer just because you wrote the rockbox directory to the drive.
What format are you using for your card? (also 8gb seems quite a small capacity to 'upgrade' to, is this just a test, or is that all you require?)