Endosomes and endosomal vesicles (EVs) rapidly move along cytoskeletal filaments allowing them to exchange proteins and lipids between different endosomal compartments, lysosomes, the trans-Golgi network (TGN), and the plasma membrane. The precise mechanisms that connect membrane traffic between the TGN and perinuclear endosomal compartments with motor-protein driven transport have largely remained elusive. Here we show that Gadkin (also termed gamma-BAR), a peripheral membrane protein localized to the TGN and to TGN-derived EVs, directly associates with the clathrin adaptor AP-1 and with the motor protein kinesin KIF5, thereby potentially regulating EV dynamics. Gadkin overexpression induced the dispersion of transferrin (Tf)- and Rab4-positive EVs to the cell periphery, whereas KIF5B-depleted cells displayed a perinuclear concentration. Functional experiments suggest that the role of Gadkin as a regulator of endosomal membrane traffic critically depends on complex formation with both AP-1 and KIF5. Our data thus provide a direct molecular link between TGN-derived EVs and the microtubule-based cytoskeleton.