High circulating cholesterol and its deregulated homeostasis may facilitate prostate cancer progression. Genetic polymorphism in Apolipoprotein (Apo) E, a key cholesterol regulatory protein may effect changes in systemic cholesterol levels. In this investigation, we determined whether variants of the Apo E gene can trigger defective intracellular cholesterol efflux, which could promote aggressive prostate cancer. ApoE genotypes of weakly (non-aggressive), moderate and highly tumorigenic (aggressive) prostate cancer cell lines were characterized, and we explored whether the ApoE variants were associated with tumor aggressiveness generated by intra-cellular cholesterol imbalance, using the expression of caveolin-1 (cav-1), a pro-malignancy surrogate of cholesterol overload. Restriction isotyping of ApoE isoforms revealed that the non-aggressive cell lines carried ApoE ε3/ε3 or ε3/ε4 alleles, while the aggressive cell lines carried the Apoε2/ε4 alleles. Our data suggest a contrast between the non-aggressive and the aggressive prostate cancer cell lines in the pattern of cholesterol efflux and cav-1 expression. Our exploratory results suggest a relationship between prostate aggressiveness, ApoE isoforms and cholesterol imbalance. Further investigation of this relationship may elucidate the molecular basis for considering cholesterol as a risk factor of aggressive prostate tumors, and underscore the potential of the dysfunctional ApoE2/E4 isoform as a biomarker of aggressive disease.