Your holly may have a fungus called Sphaeropsis tumefaciens, sometimes called Sphaeropsis Gall. It is commonly called witches broom. Symptoms include swollen and enlarged portions of diseased stems and branches, usually starting in the lower part of the tree and working upward. The name witches broom is used to describe the abundance of shoots and branches that sprout from diseased portions of branches. Branchlets originating from diseased tissue turn upward, with branch tips growing nearly vertical. Infected branches typically exhibit leaf dropping and decline, thus giving the tree an overall unhealthy appearance. Branch dieback can result in an unsightly tree and in some cases the disease can kill the tree.

Infested branches and limbs can be removed, but should be pruned 12 inches into clean, non-diseased wood. Sanitation of tools is critical in order to avoid spreading the disease throughout the tree or to other susceptible plants such as Oleander, Carissa or Bottle Brush.Dip tools in a 10% solution of bleach between each cutting and before pruning on another plant. This disease is frequently spread by pruning tools and by cuttings taken from diseased plants. Fungicides are ineffective in controlling Sphaeropsis. 

Additional Resource
http://mrec.ifas.ufl.edu/jos/Sphaeropsis.htm