I find this take off

How would you know the agent is correct in domain expertise if you’re not possessing domain expertise yourself?

Shallow generic domain knowledge is not really a helpful domain expertise, is it. Lets take banks - knowing how banks work in general means little for that one specific bank who is paying you, which has different portfolio than most, different processes because everything is homebrew over decades of doing business and they differentiate from rest of the market in many ways and thats their key proposition, different data infra, set of internal apps, protocols and so on. There is no general llm knowledge you can distill in few minutes that would help you much. You need to know your specific company, no way around it. In fact, to be proficient in those deliveries, you need to know those internal politics, processes, their quirks and so on, 100x more than some code fast delivery since this is majority of any bigger project. You need some reputation to get things done effectively. This, for s…

> I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions And you will not notice the obvious errors in its answers to even basic questions, because you are no expert and should have really consulted with one. When talking about their own area of expertise, almost everybody immediately notices the glaring faults. But then they are very impressed and take LLMs as gospel when talking about anything else. I do not get it.