The only thing I'll add is in response to your battery question.
The EverDrive v3 and X7 have a battery that supplies power to the volatile memory that stores your save when you turn off your console until you power it on again, at which point it'll be written to the SD card. It also powers the real-time clock (RTC).
Here's an example of how it plays out:
1. You're playing a game with cartridge based saving (i.e. Ocarina of Time, Banjo-Kazooie)
2. When you're done, you save in game as you usually would..
3. You turn off your console - at this point the battery kicks in to power the volatile memory where you cartridge save is currently stored.
4. Later that day you fire up your EverDrive again - at this point, the EverDrive will copy the save from the volatile memory onto your SD card.
Note: As ozidual mentioned, any games that require a controller pak still require one, the EverDrive does not replace the need for a controller pak.In short, the save from the last played/stored game is only written to the SD card when you reboot and enter the EverDrive menu again.
This is one of the biggest differences between the EverDrive v2.5 and V3/X7. The V2.5 does not have a battery (therefore no RTC either), and after you're done playing, rather than powering down your console, you HAVE to hit Reset to get back into the EverDrive menu so it can store your save onto the SD card. Powering down a V2.5 without hitting reset would result in a lost save.
In terms of how long it'll last... well I'm sure it'll last a while (haven't had to replace mine yet), but the EverDrive will give an error when the battery is low/drained. But a dead battery won't have the same impact as a stock cartridge (i.e. you won't lose all your saves, as they're written to the SD), the only issue would be if you had a dead battery and powered off your console, then you'd lose THAT particular save.
Back on to the OS, depending on the version you got, download the latest OS:
EverDrive-64 X7:
http://krikzz.com/pub/support/everdrive-64/x-series/OS/EverDrive-64 v3 or v2.x:
http://krikzz.com/pub/support/everdrive-64/v2x-v3x/os-bin/Finally... in all honesty, you can't beat the stock controllers. 3rd party ones just don't come close and don't have a proper dead-zone so they end up being hyper-sensitive.
I've actually just ordered some new gears, bowl, and stick from Kitsch-Bent to restore some old worn-out controllers. Several folks on Reddit have had good experience with the parts, so I figured I'd give them a go.
However, I just tried out the Retro-Bit Tribute controller (looks similar to the Hori), and it not too bad, slightly more sensitive than stock, but better than most 3rd party.
Enjoy the EverDrive! The community here is great if you run into any issues, and krikzz is awesome if you run into any hardware issues.