Sounds like a very valid response! I don't think it's crazy at all! Someone who violates a boundary while staring in your eyes is trying to assert dominance and make you feel that you don't matter. You have to stand firm!