http://www.amazon.com/dp/B00065ANYC/
Sandisk Ultra II 2 gb.
This card is on sale pretty often nowadays for $25 online or at Staples/Circuit City/etc. I like to use these memory cards to avoid any problems that might arise from taking long videos or many photos in a quick sequence. Even if you don't use the extra speed or notice a performance difference while using the camera, you'll definitely notice the difference the first time you attempt to transfer a full card's worth of photos to your computer.
Of course the battery it comes with is good as it was designed to match these devices.
There's a tradeoff that comes with these types of rechargeable batteries when you compare them to larger cameras that take AAs... If you go on vacation and you're out somewhere, you're not going to be happy if your juice runs out and you don't have a spare.
The huge advantage and main reason for these batteries is that allow for minimum camera size and allow for a decent number of shots before they run out of juice.