If you're feeling brave, it's also possible to entirely replace your computer's central processing unit (CPU). You can go for a faster version of your current processor, or get one with more cores to enable your computer to better handle multitasking. Due to differences in architectures and numbers of processor cores, comparing raw GHz numbers between different manufacturer's CPUs, or even different models from the same manufacturer, doesn't always tell you what CPU will be faster. But an upgrade to a newer CPU should greatly increase your computer's speed and performance. This upgrade is especially helpful for people who work with processor-intensive applications for things like audio and video encoding, or even gaming.
As with any hardware upgrade, there are potential compatibility issues. The easiest route is to see what CPUs are supported by your current motherboard and pick a faster compatible CPU. If you want something newer and better that just isn't supported, you have to upgrade the motherboard, too. Replacing the motherboard generally requires replacing the heatsink and cooling fan, and could change the type of RAM your computer needs, so you have to do some research to see what components you will need to buy in this case. Lone CPUs run from less than $50 into the hundreds, and motherboard and CPU combos start at less than $100 and similarly go up. But if your plans require replacing the whole kit and caboodle, you'll need to add everything up and weigh the cost against purchasing a new computer that has the performance you're seeking.
Replacing your computer's brain is not for the faint of heart. You will need to read manuals or online tutorials, make sure you have the proper tools (possibly including thermal grease), and carefully follow safety precautions so that you don't hurt yourself or fry components with static electricity. But if you install a brand spanking new processor, it could vastly improve performance, and give you the feeling of accomplishment that is sure to follow such a lofty undertaking.