(Docs) Add support for Llama2, Claude, Palm, Cohere, Replicate (100+ LLMs)

[krrishdholakia]

Hey @AntonOsika,

Based on your feedback on this PR - #660,

My PR shows users how to call their custom LLM providers (bedrock, togetherai, huggingface tgi, replicate, ai21, cohere, ai21 etc.) by pointing the api base to a local proxy they can use for experimentation with GPT-Engineer.

This makes no code changes to GPT-Engineer.

Happy to add additional tests/documentation if the initial PR looks good.

[sweep-ai]

Apply Sweep Rules to your PR?

• Apply: Ensure all new functions and classes have very clear, concise and up-to-date docstrings. Take gpt_engineer/ai.py as a good example.
• Apply: Leftover TODOs in the code should be handled.
• Apply: All new business logic should have corresponding unit tests in the tests/ directory.
• Apply: Any clearly inefficient or repeated code should be optimized or refactored.

[haseeb-heaven]

Hi @krrishdholakia this is nice idea and great suggestion to add LiteLLM for new models out there i really like this, but i have one question that GPT-Engineer already is using LangChain which already supports all the models right away so why we need LiteLLM in this PR?
Can you describe what feature LangChain is missing that are covered in LiteLLM.

[krrishdholakia]

@haseeb-heaven can you show me how i could use anthropic with gpt-engineer without modifying the code?

[haseeb-heaven]

@haseeb-heaven can you show me how i could use anthropic with gpt-engineer without modifying the code?

I don't know much about this codebase as i have not yet contributed enough.
So i checked LangChain docs about Anthropic (https://python.langchain.com/docs/integrations/chat/anthropic)

So i asked GPT-4 About the codebase LangChain integration.

here is a summary of the steps to integrate the Anthropic model from Langchain into the GPT-Engineer codebase:

1. Import the ChatAnthropic class from langchain.chat_models and the necessary message classes from langchain.schema.
2. Create an instance of the ChatAnthropic class for interacting with the Anthropic model.
3. Modify the AI class in the gpt_engineer/core/ai.py file to use the ChatAnthropic instance instead of the current model.
4. Modify the AI class methods start, next, and advance to use the HumanMessage and AIMessage classes for creating and handling messages.
5. Test the integration by running the application and checking if the Anthropic model is being used correctly.

[krrishdholakia]

@haseeb-heaven this is docs for gpt-engineer 😄, not langchain. specific to this repo, i don't believe i can use non-openai/azure models without making code changes or creating a proxy (as indicated with these docs)

[haseeb-heaven]

@haseeb-heaven this is docs for gpt-engineer 😄, not langchain. specific to this repo, i don't believe i can use non-openai/azure models without making code changes or creating a proxy (as indicated with these docs)

LangChain supports Anthropic so you could use LangChain but since its using OpenAI directly then I guess you have to make more changes in codebase first for it to work

[TheoMcCabe]

These seems sensible to me - its just the readme being changed and its adding value to people that want to use an alternative or local model straight away without making any code changes. Shall we merge?

@krrishdholakia if you run pre-commit run --all-files locally it will fix the failing pre commit checks for you

pip install pre-commit if not installed already

[krrishdholakia]

@TheoMcCabe thanks! will do and update the PR

[TheoMcCabe]

There is already an example at the top of this document showing how to link with local models by updating the open ai base url variable. Theres no need to duplicate this information in the docs. Closing for this reason.

[ATheorell]

I agree that it is a duplication of the wizardcoder example. However, I think, in the section with the wizardcoder example, it could be useful to also have a short instruction on how to alternatively set up a liteLLM endpoint. The using local/open endpoints section could give a (very) brief overview of potential frameworks that can be used for hosting a local server (in particular containing links to the relevant tutorials) and then a unified gpt-engineer call for how to use them. Feel free to have a shot at this in a new PR @krrishdholakia