We also load because of the knowledge. I've found out more about guns and shooting by reloading than by anything else (except shooting, I suppose). The first order of business is to get a reloading manual, preferably from the company that makes your preferred components.
I run a pretty simple setup, and I reload for 3 different cartridges right now. Two of them are rifle cartridges, and the third one is a rifle/handgun cartridge that I just started loading. Once you get the bulk of your tools, it starts to get cheaper. I built up my reloading stuff slowly, finding out what I need as I go along. I'd say the bare minimums are a simple press, the appropriate dies, calipers, and a scale (in addition to the manual). Other VERY handy things are reloading trays, a tumbler, a deburring tool, and a case trimmer. That's about the simplest set you'll want, and it can be thrown together for not a huge amount of money. You'll probably find other tools that work well for you and improve your reloading experience. As you start reloading different cartridges (which you will, once you get the hang of it), all you'll need is dies.
Welcome to the wonderful world of rolling your own.