Heptathlon and Decathlon are the hardest because they require an athlete to prepare for several events with various disciplines.
However, the single events are bit tougher to pick.
For the field I would say Pole Vault because it requires power, strength, flexibility, speed, timing and courage; six metres is pretty high.
On the track it has to be the Steeple (400m hurdles not far behind) because it just takes so much out of you. I think most people would agree that even an easy steeple run is pretty tough on the body, whereas all other events - even the 10,000 metres - pose very few problems for anybody who is fit (except maybe boredom of course).
I