Agricultural production and household food security are hypothesized to play a critical role connecting climate change to downstream effects on women’s health, especially in communities dependent on rainfed agriculture. Seasonal variability in agriculture strains food and income resources and makes it a challenging time for households to manage a pregnancy or afford a new child. Yet, there are few direct assessments of the role locally varying agricultural quality plays on women’s health, especially reproductive health. In this paper we build on and integrate ideas from past studies focused on climate change and growing season quality in low-income countries with those on reproductive health to examine how variation in local seasonal agricultural quality relates to childbearing goals and family planning use in three countries in sub-Saharan Africa—Burkina Faso, Kenya, and Uganda. We use rich, spatially referenced data from the Performance Monitoring for Action (PMA) individual surveys with detailed information on childbearing preferences and family planning decisions. Building on recent advances in remote monitoring of seasonal agriculture, we construct multiple vegetation measures capturing different dimensions of growing season conditions across varying time frames. Results for the Kenya sample indicate that if the recent growing season is better a woman is more likely to want a child in the future. In Uganda, when the growing season conditions are better, women prefer to shorten the time until their next birth and are also more likely to discontinue using family planning. Additional analyses reveal the importance of education and birth spacing in moderating these findings. Overall, our findings suggest that, in some settings, women strategically respond to growing season conditions by adjusting fertility aspirations or family planning use. This study also highlights the importance of operationalizing agriculture in nuanced ways that align with women’s lives to better understand how women are impacted by and respond to seasonal climate conditions.