What you are describing sounds like growths of lichen. These are rough, moss-like organisms that grow naturally on the bark and branches of trees and shrubs. Their body, which can be gray, green, reddish, or yellow, is a combination of an alga and a fungus growing together symbiotically, an association advantageous to both organisms. They are completely harmless and neither attack nor kill plants on which they grow. They develop rapidly when exposed to bright light as frequently occurs when a tree’s canopy thins out.
If a plant is declining and covered with lichen, there is some other cause for the stress - possibly drought, disease, or insects. An overall evaluation of the tree’s condition should be conducted to determine what is causing the decline. Since lichens are not considered pests, there are no chemicals which can be legally sprayed to control them. They are simply a natural part of tree trunks.
Additional Resources
http://alachua.ifas.ufl.edu/lawn_and_garden/fact_sheets/lichens.shtml
The article has been updated successfully.