Wow.
Hun, I do not know this man but based on what you wrote, he is a complete selfish fool. Out of the natural love for his child he shouldn't leave. Just for that in itself. Not to mention ***.Y.O.U***!!!! He told you that a paycheck is more important than anything.....Does that not give you the hint to leave this guy? I know love is a powerful force that can blind you and make you stay in crazy situations, but he has proved himself to definetely not be worthy of you nor your child. No man that loved a woman and his child would do this, period. On top of all that, itsounds like a "get rich quick scheme", totally.
Look at this whole scenario. Place yourself outside of it. If a friend came to you with your situation, what would you say and suggest to her? It is flat out 100% not right to leave you both. Do not go with him, your best where you are. He should be sensitive of his wife and babies needs, not his. He is selfish and undeserving of you.
I wish you the best of luck and may you find guidance soon.