It is well known that children’s academic performances are affected by both their family backgrounds and contextual or structural factors such as the urban–rural difference and regional variation. This article evaluates the relative importance of family background versus structural factors in determining children’s academic achievements across three different societies: China, the United States of America, and Germany, analyzing data from five large-scale, high-quality, and nationally representative data sets. The results reveal two main findings: (a) family socioeconomic status exerts much stronger positive effects on children’s academic achievement in the USA and Germany than in China; and (b) structural factors (such as those measured by location and urban/rural residence) play much smaller roles in the USA and Germany than in China.