You make a mistake to think there is no overlap between PvE and PvP players.
If all the players who never do PvP would leave the game, the rest of the playerbase that does both PvE and PvP will happily continue playing without any problems*.
Kind regards, a PvE and PvP player
*The only minor problem would be that there are less targets to shoot at