We tried this for a while and stopped because it seemed like a dead end
We tried this for a while and stopped because it seemed like a dead end. Our use case was/is a text-to-sql bot that would provide domain-specific output. Our domain is very complex so any sort of off-the-shelf AI SQL helper is a joke to us at best. My thinking goes something like "If your schema is so simple that you don't need to think about FTing the model, why do you need its help writing SQL in the first place? Your answers are probably on google somewhere." The perspective we have now is that the LLM is a probabilistic text transformation engine that must always be contextualized by some external means. Clearly, if we could just include all of our context & it fits & it's cheap, we should just do this. But, the reason anyone is thinking about FT in the first place is because you can't fit the whole damn business in the prompt (or because it's too expensive to do it at scale). The approach we are looking into now is a classification front end that attempts to discover the relevant business context being referred to by the user's initial prompt. …
Original discussionFinetuning Large Language Models70 source comments · at import