I also have the exact same issue. I would like to use a more granular general return value than userfriendlyException. You might look at AjaxResponse object in ABP framework. I think that is what you mean. But I don't know what would be the side effect of using it in xAppService classes. Because ABP and Zero, do many things automatically.
I'm not an AspNet Zero customer yet. I'm evaluating the ABP to see if it meets my requirements and how could I achieve my goals with it. (if possible) Therefore I started reading the documentations, tutorials and looking at the code of the Module Zero. But couldn't find any reference about an application with a lot of custom API's which are not part of the UI and should only be used by other applications or mobile apps. I understand from the website that the AspNet Zero as a product is a complete implementations of the ABP and Module Zero. But to evaluate the framework I only have access to the Module Zero.