Here’s a field report about some negative aspects of a particular federated search user experience, not certain by whom, sorry.
I think some of those specific issues cited have actually been dealt with pretty well in David Walker’s Xerxes design. For instance, it gets the user to full text much more reliably (although still not perfectly) than most broadcast search interfaces.
I think Xerxes does a lot better than any commercial out of the box broadcast search software I’ve seen. But there are limits to broadcast search. There’s a point at which incremental improvements take increasing development. There are a few more tweaks that can be made on the technology level, that vendors have, are, or could work on: Faster resonse time, better machine analysis (relevancy ranking, de-duping, clustering).
I’ve suggested before that you can get a lot better interface out of a local index solution than a broadcast search solution, if you can manage to get a robust local index of all our licensed scholarly content. Which is not easy, but if you really want a good cross-search interface, is easier than trying to make broadcast search do magic.
I think the Serial Solutions Summon service is awfully interesting; providing a centralized licenseable index of cross-vendor article-level metadata. That is, letting the user search a large database of scholary material (like Google Scholar), that is (not like G.Sch.) balanced accross disciplines, predictable in content, normalized and structured (useful for better link resolver integration and much more), and available to customers through an full-featured API.
If they can succeed in providing access to nearly as much and diverse content as broadcast searching, can provide a reliably robust and quick-running service, and can price it in the general order of magnitude of existing broadcast search services — I think this could be the future of scholarly meta-search.