Mine has always done that (maybe not quite as long as 20 seconds, but definitely a noticeable time, maybe 5 or more seconds?).
It's not rebuilding the database, its just reading it, I think.
However, I have always assumed its because the aging hardware is creaking slightly with the number of tracks I have in the database (approx 70,000). Doesn't sound right if its happening with far fewer tracks. Is it a hard-disk pod? If so, could the drive be having problems?