For a scientific setting, I highly recommend learning Python first. It's free, it runs on any platform (including Windows, OSX, Linux, and many more platforms you've probably never even heard of) and there's a wealth of libraries available. The SciPy and NumPy are very widely used in the scientific commmunity. There's a nice list of other packages on the SciPy website (http://www.scipy.org/Topical_Software)
Honestly, the best way to learn programming is to jump in and just do it. I'd suggest getting an introductory book for whatever language you plan on learning and start working with it. If you go with Python as I recommend, you can even download a good beginner book ('A Byte of Python') for free:
http://www.swaroopch.com/notes/Python
There are other more advanced Python books for free like this
http://mindview.net/Books/Python/ThinkingInPython.html
(Someone might follow up to this post saying Python is interpreted, so it's slow, blah, blah, blah. But they're wrong. You can always code critical sections in C and use SWIG to generate Python wrappers. Don't worry if you don't understand what I just said... you probably won't ever need to do this anyway.)
If you want to look at some general books on writing computer programs, there are some classics like:
- Structure and Interpretation of Computer Progams (again, free at
http://mitpress.mit.edu/sicp/full-text/book/book.html
)
- Jon Bentley's "Programming Pearls"