I still hear people talk about "reverse racism" and it is very frustrating. They will literally quote the exact definition of racism, which, by that standard, one could potentially make this argument, but that's because the dictionary defintion is flawed - it is only half of a definition. What it fails to define is the power dynamic, as you say.
When a power dynamic allows one race to exploit another, the reverse, because of the power imbalance, cannot be true. Meaning, reverse racism cannot exist. Honestly, I think dictionaries need to be revised to include the power dynamic in their definitions of racism so white people stop making this dumb argument. (Although, if they're so determined not to recognize this, they'll just come up with some other nonsense.)