He lived peacefully with his family on a large estate in the rolling green hills of southern Uganda and rarely quarrelled with his family.
But that all changed when he and several Christian colleagues began a radical cult that is today blamed for the deaths of nearly 900 of its followers, half of them burnt alive in a small church earlier this month.
``This is what I can't imagine,'' Juvenal Rugambwa, Kwibetere's 36-year-old son, told Reuters in an interview.
``A father who loved people, who loved God. Now today you can't imagine a father who kills.'' Family members say the sect, the Movement for the Restoration of the Ten Commandments of God, evolved after Kwibetere visited the central Ugandan town of Mbarara in 1989.
There, he met a woman named Gredonia Mwerinda, a self-styled visionary who claimed to communicate directly with the Virgin Mary, and her sister.
Kwibetere's wife Theresa, who separated from him in 1992 after a 32-year marriage, described him as a ``loving man, cheerful, a man of good character.''
``But when he joined that group, he changed completely,'' she told Reuters on their family's farm in Kabumba.
Theresa and her children joined the cult, but she soon fell out with the newcomers, who called her a sinner, burning her clothes in what they said was a ``fire from heaven.'' ``He was influenced by those women that what I'm doing is wrong and what they're doing is right,'' she said, adding that the sect soon took priority over the family in her husband's eyes.
``They (our children) were mistreated and they were not given food, they used to starve. They nearly beat one of my daughters to death,'' she said.
``I was really surprised because such a thing couldn't have been done in people worshipping God. They pretended to be holy, but they weren't.'' The sect's odd regulations -- no soap, no speaking and no medical care -- alienated members of Joseph's own family, and five of his own sons and daughters ran away.
``The women claimed to be visionaries. They claimed to be seeing the Virgin Mary and they told Joseph what was to be done,'' Rugambwa said. ``But his followers mistreated us. They started beating us, and whenever they did they said it was the Virgin Mary beating us.''
Rugambwa led a drive to free the farm of sect members, including his own father, with sticks and stones in 1992. By then, 250 followers were living on the estate.
The last time the family saw Joseph was in 1995, when he came for the funeral of one of their sons who died of natural causes.
Today, nobody can say for sure if Joseph Kwibetere is dead or alive, but his family is not optimistic. His wife believes he died in the fire at Kanungu earlier this month. His son visited the scene and found a ring on one of the fingers that resembled one his father used to wear.
Kibwetere's wife and family are now guarded by a lone armed soldier posted at the house to prevent possible revenge attacks by relatives who lost members of their own families.
Despite everything, Theresa remains strongly Catholic.
Her house is adorned with colourful framed pictures of Jesus and Mary, as well as black and white photos of Joseph Kibwetere with Catholic bishops in better times.