Did I marry the wrong person?

When I told friends and family that I would not be inviting my dad to my wedding, many taught I had gone too far. Some tried to persuade me to change my mind, but I was adamant. Why should I honour a person who abandoned me? Where was he when I needed him most? Well, I didn’t want him on my big day, and he was not missed.
That happened years ago, I still think critically, and I always try not to allow my sentiments and emotional blackmail from other people into my decision making. I am about to take a decision that may sound so foolish to people. I have decided to quit my job as a banker for something I have always love to do. Don't get me wrong, I am not having problems at work, neither am I underpaid. I have made good progress in the banking industry since I joined, but I am not fulfilled. I feel that I can be more useful to humanity if I pursue my passion. I am getting bored in the boardroom.
I am seeking advice or encouragement because of the risk factor. I presently have a steady but boring job; the salary is good because I still have some left to save after I have attended to my personal and other needs. If I stay at my job, I will be playing safe but unhappy. And, if I retire to pursue my dream, I will be fulfilled and happy but uncertain of steady incomes.
Please, let me know what your decision would be and why, if you were in my shoes.