An often overlooked source of noise from the PC is the power supply unit (PSU). Most PSU’s have internal fans which are sometimes quite noisy, so your options are usually limited to replacing the PSU with a quiter model. One option is a NoiseKiller kit from Antec ($8.49 from Newegg). This puts a little silicon buffer between the PCU and the case chassis, helping to reduce noise related to vibration. Another option would be to replace the PSU with a fanless model. These models dissipate heat through the use of large heat sinks. They are a bit more money, but not usually enough to break the bank.

On a related note, if you are buying a new PSU, make sure you buy a model which has the ATX 12V 4-pin connector which is required for most modern PCs. Also, pay attention to whether you will need the 24-pin to 20-pin converter to plug into the ATX power connection for the motherboard.