All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah and that Muhammad is His slave and Messenger.
What happened between you and them is not very clear to us. However, we say: It is clear to us that if there is a text mentioning kufr or shirk without further specification, then what is meant by them is minor kufr and minor shirk, and these do not take a person out of the fold of Islam. This is the meaning that Imaam Al-Bukhari wished to establish in the mentioned chapter to respond to some innovated sects who consider that a person goes out of the fold of Islam by committing an act of disobedience, such as the Khawaarij.
Ibn Hajar said in Fat-h al-Baari:
“In brief, because he had previously said that sins are called metaphorically kufr, referring to arrogance toward the favors of Allaah (kufr ni'mah) and not to real disbelief in Allaah (kufr jahd), he wanted to clarify that it is a kufr that does not take a person out of the fold of Islam, contrary to the sect of Khawaarij, who consider that a person becomes a disbeliever by committing sins, while the text of the Quran refutes their claim through the saying of Allaah: {but He forgives what is less than that for whom He wills...} [Quran 4:48]; so Allaah considered what is less than shirk subject to the possibility of being forgiven…”
This is exactly the meaning that Ibn al-‘Arabi wanted to establish by saying, “The purpose is to clarify that just as acts of obedience are called belief (imaan), likewise, sins are called disbelief (kufr). But where they are called disbelief, it is not meant with this the disbelief that takes a person out of the fold of Islam.”
Finally, having an excuse of lack of knowledge and committing a sin by mistake and so forth are among the impediments from declaring someone as a disbeliever; this is established in the following fataawa: 87963 and 8106, so please refer to them.
Allaah knows best.