It's all a trade-off: the more real fear, danger and technical challenge, the greater the rush and sense of achievement...what you have to work out is where your own personal balance lies - i.e. what you will risk for the rush.
Skill and experience come into play, but from personal experience all that really means is the better you get, the faster you are willing to hit things and the bigger the jumps and drop-offs you are willing to take on become.