After running 25 years without a single major injury, I've been fairly injury prone since turning 40. As a result, I've gotten into biking, and more reluctantly, swimming. Typically, I have a goal race in spring. After that, I do a triathlon in late summer. Therefore, I give my body a break by doing a lot more swimming and cycling for three or four months and only running 2-3 times per week max.
If you want to cycle, start by getting an entry level road bike. Shopping at a local bike store is a good idea, but REI is also an option. Just be prepared to take your bike to a LBS to get it tuned if you buy it at REI. I would say the most affordable road bike options that are high quality are Specialized, Cannondale and Trek. For example, you could get Specialized's entry level bike, which comes in at $1000:
If you shop around, you might find it for $800 or $900. Craigslist is a good option as well if you can't afford that. I agree with previous poster that dropped handlebars are the way to go.
If you can swim, sign up for a "super sprint" triathlon and you can get a casual chance to race.
I am a pretty good runner but suck at biking. I'm not sure why, honestly. For that reason, I still prefer running. But biking has been growing on me. I love that I've never had a whisper of injury. I've had plantar fasciitis and IT band problems and a calf pull that stop me from running but I can cycle completely pain free. I love that I can cover more ground and see more stuff while biking. I like the feeling of free wheeling downhill at high speed. (With that said, be careful. E = 1/2 * m * V^2 ~ severity of injury.)
I live in a place with nasty winter weather and have invested in a trainer (Kickr Core but there are lots of options) and a Zwift membership. This is a downside from running - I run outside in basically all weather, but because of the wind, inclement weather is much more difficult to handle on a bike.
It's worth it to invest in good bike bibs. I bought a bunch of cheap ones and eventually shelled out the big bucks for a good pair that changed the game for me and eliminated chafing. (Another thing I've never had to worry about running.) Also, if your butt hurts, get yourself a decent bike seat. At the end of a ride, your legs, not your butt, should hurt the most.