- Lecture slides
- Video (in russian): lecture, practice
- Practice session: practice.ipynb
Applications mentioned in the lecture:
- the Gandalf game by Lakera.ai
- aidungeon.com - an LLM-generated role-playing game
- ora.ai - a tool to build your own chatbot with prompting
- this is not an exhaustive list: there's a million of various applications using LLMs
Open-source models mentioned in the lecture:
- LLaMA-2 - https://huggingface.co/meta-llama/Llama-2-70b
- Falcon - https://huggingface.co/tiiuae/falcon-180B
- BLOOM - https://huggingface.co/bigscience/bloom
Some of those models require you to apply for access, and model authors may take time to process your application. While you are waiting for your license to be processed, you may wish to browse the huggingface hub for alternative (e.g. quantized) versions of the same model that are available immediately with no application. For example, here's a LLaMA-2-70B quantized to 4-bit and available using the same transformers.AutoModelForCausalLM.from_pretrained syntax. Please note that, while the hub allows you to download and use those model versions without officially applying for access to LLaMA-2, the original model's license may restrict this kind of usage.
Extra materials:
- Glitch tokens (lecture mentions SolidGoldMagikarp) - blog post by Jessica Rumbelow, mwatkins
- "Sparks of AGI" - a controversial but influential paper about worrying LLM abilities
- BigBench - a benchmark of emergent LLM abilities mentioned in the slides
- Chain of thought papers: Few-shot: Wei et al. (2022) few-shot
- A guide to prompt injection and jailbreaking: https://learnprompting.org/docs/prompt_hacking/injection
- A repo with popular jailbreaks for GPTx models: https://github.com/0xk1h0/ChatGPT_DAN
- A ton of other cool stuff linked in the lecture slides (see the top of this readme)