I do a lot of reference improvement on Wikipedia. Our cite templates put the last name of an authors first, followed by a comma, and the given name (e.g., "Bloggs, Joe"). One very common task among people who improve references is to look for existing articles for reference authors so that authorlinks can be added. This was conveniently done for instance in Firefox by just highlighting the name and using the right click menu to search on Wikipedia as I have it setup to do. The old search was pretty amazing at finding the correct author article even when the search had the names backwards. The new search engine is hit or miss and overall I would say it is pretty terrible at it. For example a search for "Gettleman, Jeffrey" on the English Wikipedia completely misses the existing article "Jeffrey Gettleman". I'd estimate that the new search "completely misses" the obvious intended page about a quarter of the time when using this search format. About half the time the intended target is within about the top 5 results. And the remaining quarter of the time it actually finds the target. This is compared to the old search which is almost always spot on.
I don't know how the internals of the old search work, but I would venture a guess that when a search term like "X, Y" is given, it does a search for "X, Y" and "Y X". Perhaps that is missing in the new search. When I said above that the old search was pretty amazing, I meant it. Very often, it returns the intended result first even when initials are used for the first and/or middle name. It's also possible that it was taking into account redirects and stuff to figure out the intended target.
I've been using the new search (the beta implementation) for a while now. My overall impression is that except for the above issue, where the new search is inferior, I haven't noticed any significant change in quality of the returned results. They are about equally good and I would have trouble noticing which engine I was using if I had to guess.