The chatstream package provides a Shiny for Python module for building AI chat applications. Please keep in mind that this is very much a work in progress, and the API is likely to change.
See this post on the Shiny blog to see some examples in action!
Chatstream currently supports the OpenAI API. To use this, you must have an OpenAI API key. You can get one from the OpenAI or from Azure's OpenAI Service. (Note that if you have use Azure, you will need to point the applications to the Azure endpoint instead of the default OpenAI endpoint.)
The chatstream package is not on PyPI, but can be installed with pip:
pip install chatstream@git+https://github.com/wch/chatstream.gitAlternatively, if you'd like to develop a local copy of the package, first clone the repository and then install it with pip:
cd chatstream
pip install -e .[dev]Before running any examples, you must set an environment variable named OPENAI_API_KEY with your OpenAI API key.
You can set the environment variable with the following command:
export OPENAI_API_KEY="<your_openai_api_key>"Then run:
shiny run examples/basic/app.py --launch-browserSome examples (like recipes) have a requirements.txt file. For those examples, first install the requirements, then run the application as normal:
pip install -r examples/recipes/requirements.txt
shiny run examples/recipes/app.py --launch-browser-
Does this work with Shinylive? It almost does. The
openaipackage has dependencies which do not install on Pyodide, but chatstream currently has anopenai_pyodideshim which uses the browser'sfetchAPI. However, there is one more hurdle: thetiktokenpackage (which counts the number of tokens used by a piece of text) needs to be built to run on Pyodide. -
Does this work with langchain? It currently does not. Note that most of the langchain interfaces do not support streaming responses, so instead of showing responses as each word comes in, there is a wait and then the entire response arrives at once.