I found the images or elements with mw-no-invert class are still slightly color-altered. I looked into it and found the two hue-rotate(180deg) did not exactly return to the original color. invert() does not influence this.
Topic on Extension talk:DarkMode
"filter: unset" should return it to normal. Definitely should be a configuration option though.
filter: unset equals to filter: none. It does not remove filters applied by parent elements.
Apologies; I meant to apply "filter: unset" on the root element where the original filter is set. Then you can implement your own class (.client-darkmode .invertible) on elements you want to invert - if you were wanting any to be inverted in the first place.
I ended up just ditching the built-in resources/ext.DarkMode.less file altogether and making my own for that very reason, but also to maintain more control (example).