Welcome to the Rockbox Technical Forums!
Load Last Bookmark is probably set to "Yes", so have a look at that setting.
Bookmark On Stop will probably help you tremendously with the "How long will this be stopped?" dilemma - though you will have to get into the habit of stopping (not pausing) playback.
I don't want to load a bookmark every single time...
Saving my position when I let the player power off works fine until I enable bookmarking then save one.
Page created in 0.095 seconds with 15 queries.