Give the chestnut is bothering to another or annoy him, use must come from the use of chestnuts or better yet castanets made of the wood of the chestnut tree which can be a pretty nasty noise.
* Only one "like" per meaning and day, the more "likes" the meaning will appear higher in the list