Phagocytosis in macrophages is receptor mediated and relies on actin polymerization coordinated with the focal delivery of intracellular membranes that is necessary for optimal phagocytosis of large particles. Here we show that phagocytosis by various receptors was inhibited in primary human macrophages infected with wild-type HIV-1 but not with a nef-deleted virus. We observed no major perturbation of F-actin accumulation, but adaptor protein 1 (AP1)-positive endosome recruitment was inhibited in HIV-1-infected cells. Expression of negative factor (Nef) was sufficient to inhibit phagocytosis, and myristoylation as well as the LL and DD motifs involved in association of Nef with AP complexes were important for this inhibition. We observed that Nef interferes with AP1 in association with membranes and/or with a cleaved regulatory form of AP1. Finally, an alteration of the recruitment of vesicle-associated membrane protein (VAMP3)- and tumor necrosis factor-alpha (TNFalpha)-positive recycling endosomes regulated by AP1, but not of VAMP7-positive late endosomes, was observed in phagocytic cups of HIV-1-infected macrophages. We conclude that HIV-1 impairs optimal phagosome formation through Nef-dependent perturbation of the endosomal remodeling relying on AP1. We therefore identified a mechanism of macrophage function down-regulation in infected cells.