New kinship and $F_{ST}$ estimates reveal higher levels of differentiation in the global human population


Kinship coefficients and $F_{ST}$, which measure genetic relatedness and the overall population structure, respectively, have important biomedical applications. However, existing estimators are only accurate under restrictive conditions that most natural population structures do not satisfy. We recently derived new kinship and FST estimators for arbitrary population structures. Our estimates on human datasets reveal a complex population structure driven by founder effects due to dispersal from Africa and admixture. Notably, our new approach estimates larger $F_{ST}$ values of 26% for native worldwide human populations and 23% for admixed Hispanic individuals, whereas the existing approach estimates 9.8% and 2.6%, respectively. While previous work correctly measured $F_{ST}$ between subpopulation pairs, our generalized $F_{ST}$ measures genetic distances among all individuals and their most recent common ancestor (MRCA) population, revealing that genetic differentiation is greater than previously appreciated. This analysis demonstrates that estimating kinship and $F_{ST}$ under more realistic assumptions is important for modern population genetic analysis.

bioRxiv, doi:10.1101/653279