Natural enemies have emerged as important mediators of species diversity in tropical forests through density-dependent regulation of plant populations. In response, plants have evolved an array of chemical defenses. While much attention has been given to the allocation of these defenses in protecting leaf tissues from invertebrate herbivores, many classes of defense compounds that repel herbivores also show anti-microbial activity. Seeds are among the most highly defended plant tissues and are susceptible to both pre- and post-dispersal seed predators and pathogens in the soil. Defenses that have evolved to protect against seed predators or herbivores may therefore play a role in determining susceptibility to pathogens and vice-versa. In this study, we examined whether the seeds of congeneric tree species differ in their susceptibility to seed-infecting fungi consistent with the level of host-specificity required to generate density-dependent effects. Surface-sterilized seeds of Cecropia insignis, C. longipes, C. peltata, and Jacaranda copaia were buried in mesh bags in five common gardens in the understory of lowland forest on Barro Colorado Island, Panama, and retrieved at intervals of 1 to 30 months. We assessed viability of seeds and then isolated and characterized seed-infecting fungi via DNA barcoding. We found that a large fraction of seeds were infected by fungi after burial. Rather than burial duration or burial site, tree species identity was the strongest determinant of fungal community composition. Germination assays showed that fungi often had host-specific impacts on seed germination and viability. Evidence of strong host-dependent differentiation of fungal communities among species buried at the same sites, coupled with functional specificity of these fungi in their impacts on seeds of different tree species, supports pathogen-mediated coexistence of related tree species at the seed stage, preceding the demographic stages (seedlings, saplings) most often studied by tropical ecologists. A next step is to determine how seed chemistry associated with susceptibility to fungal infection relates to the chemistry and anti-herbivore defenses of above-ground plant tissues.