Modded and Vanilla

So I like modded Terraria, but for some reason, even if I turn off all mods, I still cant join servers with TModLoader installed. Is there a way to have both a Vanilla Version (allows multiplayer), and a Modded Version (allows advanced singleplayer) on my computer simultaneously without having to switch the files every time I want to switch between Modded and Vanilla?

If this is not possible, then is there a way to access Vanilla Servers with TModLoader on my computer?