Rubber banding with multiple clients

When running 2-10 clients I have encountered pretty serious rubber banding lasting several minutes. Even just mining, its to the point where my strip miners go offline because my ore hold fills up- before I see anything in the hold. This also happened when I had 2 clients in a fleet fight. Any suggestions to reduce this? Thanks!

All in potato mode with voice, game center, and shadowplay off. 50/50mbps network and ~70ms ping. Very beefy setup.