Don't get a motherboard that has an AGP slot. AGP is very obsolete now. PCIe is now mainstream.
If you have onboard video as well as a PCIe slot, you can run with onboard video, which may be sufficient forever and if not, you can pick up a PCIe card for as little as $30 when you feel the need. Onboard sound should be fine IF you can find out that it's compatible with Linux. The newer High Definition Audio chips can be very problematic with Linux. I've got SigmaTel HDA CODEC audio on my new laptop and I can't get sound to come out of the speakers, and Web searches show me that I have plenty of company.
I would not settle for less than a 64-bit dual core processor and 2 gigs of RAM. 64-bit dual core is the future, and dual core should give you a performance boost even with a 32-bit OS, as the cores can share the load (one handling one program that's running, another core handling something else, etc.). I should have said dual core is the *present* because quad core is now going mainstream. With 2 gigs of RAM you have the power for virtual machines, which are a terrific option. In fact, if something turns out to have serious hardware problems with Linux but works with Windows 2000/XP/Vista, you can run a virtual machine on Windows, install Linux in it, and that incompatible hardware will work just fine because the VM "fools" the guest OS into thinking the hardware is stuff with which the guest OS is compatible and the VM takes care of translating between the virtualized hardware and the actual hardware. That's the only way I've been able to run VectorLinux on my laptop, which has seemingly unsolvable problems with wireless and sound on the actual VL partition but runs like a champ in the VM. With enough RAM and CPU power, the system in the virtual machine runs just about as well as it would on a real partition.
Power supply depends on the video card. High-powered video cards need more power (duh!), so I'd go with 500 watts if I were buying. I have 350 watts in my Athlon desktop, but my PCIe video card there is low-end and not very demanding.
DDR2 is standard memory now. Even really cheap motherboards are using it. You have to use the kind of RAM required by your motherboard. I've never had a hard drive or CD or DVD burner that wouldn't work with Linux. My current hard drives are two Western Digital UDMAs, a Seagate UDMA PATA, and a Maxtor SATA, plus the Seagate (apparently) in the laptop. I also have four external hard drives (one FireWire and three USB 2) which I made with regular Ultra ATA hard drives that I put in a case. The drives are two Western Digitals, one Maxtor, and one Seagate (I think), but it may be another WD. No Linux problems with any of them.
Have fun! Building a computer is always a memorable experience.