It is an extension of the saying "hanging the habits", adding to the fig tree as a reference to the myth about the suicide of Judas, who would have hanged himself on a branch of that tree after betraying Jesus. The phrase is then interpreted as a "betrayal of the Church by abandoning the clerical state."
