ontogpt may cache LLM's "errors"/inconsistent output

[leokim-l]

Sometimes calls to GPT did not explicitly fail, but returned empty, which for practical purposes is a failed call, see image below. Thanks to

malco/src/malco/run/search_ppkts.py

Lines 29 to 31 in 653291d

	if terms:
	# ONLY if terms is non-empty, it was successful
	files.append(label)

they would be considered as unsuccessful, because there was no grounding. Rerunning the same phenopacket again, however, meant trying to ground the same empty reply over and over, since the output of the LLM had been cached. Solved by manually removing .litellm_cache.

Again, more of a reference for me, but @caufieldjh you may be interested in ontogpt UX

[caufieldjh]

Hrm, in practice this shouldn't matter except in cases where the phenopacket doesn't parse for stochastic reasons, because any change in the way the prompt is submitted will cache a new response. But I see what you mean. Perhaps the extracted output needs its own error flag.

There are cases where a lack of grounding is expected or desirable, but we can definitely avoid caching responses with no content.

[leokim-l]

Hmm, I am not 100% sure I understand.

So, just to make sure we understand each other: I am inclined to think the prompt we sent out made sense, but the LLM at hand returned an empty list. When ontogpt gets an empty list to ground, it obviously fails. When running the same prompt again, we get again the same cached empty list (funny that the numbers, such as "1. \n 2. ... " are there, which technically makes it a non empty reply) , then pass that on to ontogpt (sorry :P) and it obviously fails again. When removing .litellm_cache the effect was that the prompt would be sent again, this time returning a meaningful answer, which would then be forwarded to ontogpt for grounding.

[caufieldjh]

Ah, that's right - I forgot the initial prompt and the grounding were decoupled here. Either way, the empty responses don't need to be cached, and that's something I can fix in ontogpt.