Philip had been a member of his world's Avengers before it was destroyed. He was then was recruited by Proctor to serve as one of his Gatherers. Swordsman joined,unaware that Proctor himself was responsible for the damage as part of his mad campaign to murder every existing counterpart of the Eternal Sersi.
When Proctor's forces reached Earth-616 Earth, Javert and another Gatherer, Magdalene, began to doubt their leader. They eventually turned against him and assisted the Avengers. Proctor retaliated to their betrayal by having the Anti-Vision attack and severely wound Javert.
After recovering, he and Magdalene continued to serve alongside the Avengers. Javert took a particular interest in Hawkeye (Clint Barton), who had been a good friend and teammate on his world.
Immortus's forces attacked Javert and attempted to replace him with a Space Phantom after taking him to what appeared to be the Cotati garden.
After assisting the Avengers on several missions, Javert and Magdalene left Earth with the help of Iron Man, seeking to find a new home for themselves in another dimension.