All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
If it is confirmed to you that going to perform Hajj and leaving your child will not affect him/ her, you should perform Hajj. The scholars stated that the order to breast-feed the child for two years is a desirable order and not an obligatory one. Allah Says (what means): {The mothers shall give suck to their offspring for two whole years for him who desires to complete the term).} [Quran 2:233] So, the saying of Allah (which means): (Who desires to complete the term) means that the demand is not compulsory. Therefore, it becomes clear that the reward of Hajj is greater and so it should be given priority over completing the breast-feeding period, especially if one knows that discontinuing breast-feeding a child for few days does not mean weaning him completely.
This is in case your Hajj is optional [i.e. you have already done your obligatory Hajj] but if your Hajj is obligatory [i.e. you have not performed your Hajj before] then you have no right to fail to perform it unless you fear for your child by interrupting the breastfeeding or that you cannot find someone to look after him during your absence.
Allah knows best.