After having a some light and some intensive discussions on the differences between designers and product managers, I might have found the core difference.
Designers care much more about people, feelings, emotions, and the design process, and much less about the result. The result is important, but secondary. It is something which will happen, given the process, and even if it does not happen, well at least you learned about the people, and you grew during the process.
Product Managers (or business-people in general), have directly opposite priorities. The result is what they want, and the process is secondary. They won’t mind getting a great result (and this will be measured with hard numbers), without a good process, maybe even better because why waste time on process when you can get a working result. They like to get shit done. To many of them, reflection is often a waste of time – they can learn directly from the world by making mistakes, failing fast, and moving forward.
I know these are just musings, and some sort of caricatures. We are of course much more complex than that. But this solves some personal dilemmas for me. I want to be someone who can design well, but people who know me think of me more as a product/business person. Because most of the time I am concerned about the result and impact. I follow the process, but the process is the means to an end. But to ‘true’ designers, the process itself seems to be the end.