Skip to content

Commit

Permalink
Fix notebook example (langchain-ai#3142)
Browse files Browse the repository at this point in the history
The following calls were throwing an exception:


https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L192


https://github.com/hwchase17/langchain/blob/575b717d108984676e25afd0910ccccfdaf9693d/docs/use_cases/evaluation/agent_vectordb_sota_pg.ipynb?short_path=4b3386c#L239

Exception:

```
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[14], line 1
----> 1 chain_sota = RetrievalQA.from_chain_type(llm=OpenAI(temperature=0), chain_type="stuff", retriever=vectorstore_sota, input_key="question")

File ~/github/langchain/venv/lib/python3.9/site-packages/langchain/chains/retrieval_qa/base.py:89, in BaseRetrievalQA.from_chain_type(cls, llm, chain_type, chain_type_kwargs, **kwargs)
     85 _chain_type_kwargs = chain_type_kwargs or {}
     86 combine_documents_chain = load_qa_chain(
     87     llm, chain_type=chain_type, **_chain_type_kwargs
     88 )
---> 89 return cls(combine_documents_chain=combine_documents_chain, **kwargs)

File ~/github/langchain/venv/lib/python3.9/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for RetrievalQA
retriever
  instance of BaseRetriever expected (type=type_error.arbitrary_type; expected_arbitrary_type=BaseRetriever)
```

The vectorstores had to be converted to retrievers:
`vectorstore_sota.as_retriever()` and `vectorstore_pg.as_retriever()`.

The PR also:
- adds the file `paul_graham_essay.txt` referenced by this notebook
- adds to gitignore *.pkl and *.bin files that are generated by this
notebook

Interestingly enough, the performance of the prediction greatly
increased (new version of langchain or ne version of OpenAI models since
the last run of the notebook): from 19/33 correct to 28/33 correct!
  • Loading branch information
qpleple authored Apr 19, 2023
1 parent 599e17c commit 126d7f1
Show file tree
Hide file tree
Showing 3 changed files with 403 additions and 59 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,6 @@ wandb/
# asdf tool versions
.tool-versions
/.ruff_cache/

*.pkl
*.bin
Loading

0 comments on commit 126d7f1

Please sign in to comment.