Leukemia cells highly expressing chemokine receptor CXCR4 can actively response to stroma derived factor 1α (CXCL12), trafficking and homing to the marrow microenvironment, which causes poor prognosis and relapse. Here we demonstrate that a novel designed peptide (E5) targeting CXCR4 inhibits CXCL12- and stroma-induced activation in multiple acute myelocytic leukemia (AML) cell lines and displays anti-AML activity. We show that E5 has high affinity to multiple AML cells with high CXCR4 level in a concentration dependent manner. E5 significantly inhibits CXCL12- or murine stromal cell (MS-5)-induced migration of leukemia cells and prevents the cells from adhering to stromal cells. Mechanistic studies demonstrate that E5 down-regulates CXCL12-induced phosphorylation of Akt, Erk, and p38, which affects the cytoskeleton F-actin organization and ultimately results in the inhibition of CXCL12- and stroma-mediated leukemia cell responses. E5 can induce concentration-dependent apoptosis in the four AML cell lines tested while did not affect the viability of MS-5 or human umbilical vein cell (ea.hy926) even at 80 µM, both of which have a low level of CXCR4. In vivo experimental results show that immunocompromised mice transplanted with HL-60 cells survived longer when treated with E5 twice a week in comparison to those treated with cyclophosphamide.