Okay, so there are two different windowed modes (fixed window and windowed mode). Windowed mode should allow you to manually resize the window. There is also a settings file that you can try editing. Not sure where it will be on a mac, so try doing a file search.
And if you do figure this one out, please come back and tell us how you did it. Might help the next guy with the same problem.