All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad is His slave and Messenger. {C}
The established rule according to the scholars of Fiqh is that the fundamental principle is the non existence of something (Al-Asl Al-'Adam). This rule falls under a greater rule, that certainty is not overruled by doubt.
Therefore, if your husband does not remember what he actually uttered, then in principle he did not utter a conditional divorce.
The Hanbali scholar Ar-Ruhaybaani said in Mataalib Uli an-Nuha, "There is no divorce because of doubt about it, or because of doubting the condition upon which he conditioned the divorce."
Based on this, divorce does not take place, whether you did what he prevented you from doing or not. The bond of marriage between the two of you is still valid until the contrary is proven; so do not take any doubts or illusions into consideration.
Allah knows best.