Mangos can drop some to many of the immature fruits for several reasons. Some varieties tend to drop the majority of fruits that set, carrying as few as one fertilized flower out of 150 to fully mature fruit. Insufficient soil moisture, some diseases and also low (below 40 degrees F.) temperatures at the time of bloom can also result in fruit drop.

For more information on growing mangos see this fact sheet,

Mango Growing in the Florida Home Landscape: http://edis.ifas.ufl.edu/mg216