Olfactory impairments are present in common neurodegenerative disorders and predict conversion to dementia in non-demented individuals with cognitive impairment. In cognitively intact elderly, evidence is sparse regarding the role of olfactory deficits in predicting cognitive impairment. The present study investigated predictors of 5-year prospective decline in the Mini-Mental State Examination (MMSE) in a large (n = 501), population-based sample of elderly (65-90 years) individuals. All participants were genotyped for the ApoE gene, assessed for health factors, and were non-demented at the baseline assessment. After partialling out the influences of demographic and health-factors at baseline and dementia at follow-up, poor odor identification ability in combination with older age and the ApoE-epsilon4 allele predicted larger prospective global cognitive decline. This effect could not be produced by a vocabulary test. In sum, the findings suggest that an olfactory deficit can dissociate between benign and malign global cognitive development in non-demented, very old epsilon4-carriers, who are at high risk of developing dementia.