Yeah, all of those batteries are usually well past their prime. I've had someone I know replace a few for me on some different cartridges, I didn't trust my own soldering skills!

I do know the original game boy cartridges have a battery that you can carefully replace to save gameplay. 
Usually that battery is completely drained so it doesnt save anything anymore. 

Satosora

Yeah it's a frustrating holdover from the quarter eating arcade machines as well as trying to get as much gameplay as possible from the limited cartridge space! I do like using emulators for some games for that very reason. I'll use a few save states at times so I can put the game down for the day without erasing all the progress I've made.

astro_penguin

gaming

What do you use to play retro/vintage games?

Yes, they are just total time sucks. 
You could also save the game and leave for a while. 
This feature seems to be non existant anymore. 
You have to play until the game finishes. 