First, people sometimes think that this is political correctness which has “run amok” and makes it impossible to speak about any challenging issue. The key idea could appear here is to taking the “micro” part of “micro-aggression” to heart, arguing that they are not such a big deal compared to the more serious racism or other “-isms” that we recognize the problem.
Second, people may argue that even if the micro-aggression causes harm, it is everyone’s right under the free speech to say it anyway. They imagine that people fighting for social justice are literally telling people that they can’t say certain things, which they consider an assault on free speech.