Genetic polymorphisms in the APOE ɛ and TOMM40 '523' poly-T repeat gene loci have been associated with significantly increased risk of Alzheimer's disease. This study investigated the independent effects of these polymorphisms on human cognitive ageing, and the extent to which nominally significant associations with cognitive ageing were mediated by previously reported genetic associations with brain white matter tract integrity in this sample. Most participants in the Lothian Birth Cohort 1936 completed a reasoning-type intelligence test at age 11 years, and detailed cognitive/physical assessments and structural diffusion tensor brain magnetic resonance imaging at a mean age of 72.70 years (s.d.=0.74). Participants were genotyped for APOE ɛ2/ɛ3/ɛ4 status and TOMM40 523 poly-T repeat length. Data were available from 758-814 subjects for cognitive analysis, and 522-543 for mediation analysis with brain imaging data. APOE genotype was significantly associated with performance on several different tests of cognitive ability, including general factors of intelligence, information processing speed and memory (raw P-values all<0.05), independently of childhood IQ and vascular disease history. Formal tests of mediation showed that several significant APOE-cognitive ageing associations--particularly those related to tests of information processing speed--were partially mediated by white matter tract integrity. TOMM40 523 genotype was not associated with cognitive ageing. A range of brain phenotypes are likely to form the anatomical basis for significant associations between APOE genotype and cognitive ageing, including white matter tract microstructural integrity.