Yes, it's true. I also have a 5.5G iPod video (30GB), and I usually can net around 6-7 hours on one full charge. This is taking into account volume, codecs, and if I have Shuffle or Repeat on, and if I'm using the Database.
I use Ogg Vorbis as my primary codec, but I also have a lot of MP3s. I have some albums in FLAC, and I have the High Voltage SID Collection, a medium-sized collection of NSF files, and some Super NES soundtracks in SPC format. The SIDs and NSFs put the least strain on my battery, but most of my MP3s will drain the battery faster (they're mostly VBR; high quality).
Also note that videos will drain your battery even faster, since the hard drive is working more at that point. But it's not just the video and audio. Certain parts of the hardware are not put to "sleep" at this point, so until the Rockbox team figures out how to do that, we just have to live with the fact that our batteries will drain quicker.
You can help, however, if you can figure a way of optimizing Rockbox and learning how to "sleep" certain parts of the iPod hardware. One example: how to shut off the dock connector when it's not needed.