No errors thrown even if there is no response

[vishalvibes]

Description

Service: Groq
Model: Llama 3 70B
Issue: No errors thrown, status shows 200, even when no response is received

Effect: I am not able to understand why sometimes an empty response is coming. And whether it's an issue with Groq or with Vercel AI SDK

Code example

server route handler

const openai = new OpenAI();
const groq = createOpenAI({
	apiKey: process.env.GROQ_API_KEY,
	baseURL: "https://api.groq.com/openai/v1",
});

export async function POST(req) {
  const result = await streamText({
		  model,
		  messages,
		  maxTokens: 1500,
		  temperature: 0.5,
		  topP: 1,
		  frequencyPenalty: 1,
	  });
  
  // Respond with a streaming response
  return result.toAIStreamResponse();
}

Client Component

import { useChat } from "ai/react";

const {
		messages,
		input,
		handleInputChange,
		handleSubmit,
		isLoading,
                error
	} = useChat({
		api: "/api/chat"
	});

Additional context

No response

[vishalvibes]

Seemingly there are some issues going on with the Groq servers. Yet, we should be getting an error 503 instead of 200 here.

[lgrammel]

Can you try the streamText console example for groq?

https://github.com/vercel/ai/blob/main/examples/ai-core/src/stream-text/groq.ts

One potential issue could be the use of temperature, topP, and frequencyPenalty. Maybe try to add them step by step to the example and see if they are related to the error that you are seeing.

[lgrammel]

@vishalvibes is this still an issue? if so, please re-open