All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger. We ask Allaah to exalt his mention as well as that of his family and all his companions.
If the condom does not cause any harm then it takes the ruling of coitus interruption (withdrawing before ejaculation); we have already discussed it in Fatwa 86946, so please refer to it.
Therefore, what you are doing is permitted if you did not intend to completely prevent pregnancy as it is forbidden to completely prevent pregnancy, unless it is confirmed by a trustworthy physician that pregnancy is permanently dangerous for the life of the mother.
For more benefit, please refer to Fataawa 92181 and 87304.
Allaah Knows best.