The plant you described is most likely the golden dew drop, Duranta erecta. The origin of this small evergreen tree is subject to debate. Some sources claim it is native to the West Indies and Central America, while others claim it is native to the Florida Keys. It currently can be found in many states within the sun belt of the United States, including Florida, Louisiana, Texas, Arizona, California, and Hawaii.
Golden dewdrop grows best in full sunlight but will tolerate partial shade, and can reach heights of up to 18 feet. It works well as a specimen shrub or even a beautiful hedge. Simple and oppositely arranged leaves grow from 1 to 3 inches long, are ovate to obovate in shape, and have serrate margins. The leaves are yellow-green, glabrous (or hairless) and thin, and each one possesses a thorn on the bottom side along the rachis (stem). The bark of the golden dewdrop tree is light brown and slightly furrowed.
Small, showy, tubular flowers, ranging from purple and white to violet or blue, bloom in the spring and hang in 6-inch racemes. Bright yellow spherical fruits about ½ inch in diameter hang in long, trailing clusters in the summer which is attracting your children.
It is critical to note the fruits of golden dewdrop are poisonous to humans so it will be important to either remove the shrub or keep your children away. If you decided to keep the shrub, then be sure to remove the fruit when they emerge.
Butterflies enjoy nectar from the flowers, and although the fruit is poisonous to humans, birds use the berries as a food source. For more information check out the University of Florida publication: http://edis.ifas.ufl.edu/fr327