Eph/ephrin signaling system plays a very important role in the tumorigenesis and the formation of blood vessel. However, the function of EphB4 and its ligand ephrin B2 in the carcinogenesis of esophageal squamous cell carcinoma (ESCC) is not fully understood. Here, it was found that the expression of EphB4 was up-regulated in ESCC tissues compared with the paired normal tissues, while ephrin B2 was down-regulated in ESCC samples. Phosphorylation of EphB4 induced by its ligand ephrin B2-Fc inhibited the growth, migration and colony formation of ESCC cells. Moreover, over-expression of EphB4 or EphB4 kinase dead mutant (EphB4 KD) in ESCC cells promoted cell growth and migration, suggesting EphB4 promoted cell growth and migration independent of its kinase activity. Furthermore, we found that EphB4 interacted with the adaptor protein RACK1 and RACK1 decreased the phosphorylation level of EphB4. Taken together, our study revealed the important function and regulation of EphB4 in the progression of ESCC and suggested EphB4 as a novel target for the treatment of ESCC.