We investigated the familial clustering of different classes of voluntary regular exercise behavior in extended twin-family pedigrees. In contrast to the earlier work based on twin data only, this allowed us to estimate the contributions of shared household effects (C), additive (A), and non-additive (D) genetic effects on voluntary exercise behavior. To test whether shared household effects were inflated by assortative mating we examined the causes of spousal resemblance. For adolescent and adult participants (aged 16 to 65) in the Netherlands Twin Register we constructed 19,543 pedigrees which specified all relations among nuclear family members and larger families in the register (N = 50,690 individuals). Data were available on total weekly MET minutes spent on leisure time exercise, and on total weekly MET minutes spent on exercise activities in team-based, solitary, competitive, non-competitive, externally paced and internally paced exercise. We analyzed the data in the Mendel software package (Lange et al. in Bioinformatics 29(12):1568-1570, 2013) under multiple definitions of household sharing and used data from spouses of twins to test phenotypic assortment, social homogamy, and marital interaction as potential sources of spousal resemblance. Results confirmed the influence of genetic factors on the total volume of weekly exercise behavior throughout the life span. Broad sense heritability ranged from 34 to 41% (19-26% A, 12-21% D), and did not depend on the definition for household sharing. Engaging in team-based, competitive, externally paced activities (e.g., soccer) was ~ 13% more heritable than engaging in non-competitive, solitary activities (e.g., jogging). Having shared a household as siblings explained 4-8% of the variance in adult exercise behavior, whereas sharing a household by spouses yielded higher C estimates (20-24%), as it incorporates spousal resemblance. Spousal resemblance was explained by both social homogamy and marital interaction, with little evidence for phenotypic assortment. We conclude that both the amount of voluntary exercise behavior and the preference for specific classes of exercise activities in adults is explained by additive and non-additive genetic factors and unique environmental influences that include correlated exercise behavior of spouses.