-
Notifications
You must be signed in to change notification settings - Fork 379
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: capture token counts from google vertex in langchain-python #2032
Comments
Is this specific to vertex? Could this be the default for all models? I have noticed that the langchain integration does not generally use the token counts that come back with a response, and, the token counts in the response are different than what is displayed in the dashboard. Was digging into the python SDK code today to try and debug. It seems like the llamaIndex integration does this already. More than happy to try and contribute towards this. |
When using
generate
theGenerationChunk
object containsusage_metadata
. It'd be useful to capture the token counts from there.Example:
Docs: https://python.langchain.com/v0.1/docs/integrations/llms/google_vertex_ai_palm/
The text was updated successfully, but these errors were encountered: