Genetic variation alone may not account for common chronic disease susceptibility. Rather, an interaction between genetic and environmental factors may clarify the underlying disease mechanism. Hence, we tested whether body mass index (BMI) modified the genetic association of the apolipoprotein E ε4 allele with cognitive decline. The data came from a longitudinal population-based sample of 4055 participants interviewed at 3-year intervals from 1993 to 2012. Cognitive function was assessed using a standardized global cognitive score and BMI was assessed at baseline and classified as normal, overweight, and obese. There were 1374 (34%) participants with the ε4 allele. In normal BMI participants, cognitive decline was 0.048 units/y without the ε4 allele, and increased by an additional 0.031 units/y with the ε4 allele. In overweight participants, cognitive decline was 0.038 units/y without the ε4 allele, and increased by an additional 0.026 units/y with the ε4 allele. Finally, in obese participants, cognitive decline was 0.038 units/y without the ε4 allele, and increased by an additional 0.014 units/y with the ε4 allele. The association of ε4 allele with cognitive decline was significantly lower in obese participants compared with normal BMI participants (P=0.003), thereby suggesting significant gene-environment interaction on cognitive decline.