This project provides a Fullmetal Agent written in Node.js that interacts with Fullmetal API.
The main workflow this application is as following:
- Register itself to Fullmetal API
- Receive prompts from Fullmetal API
- Generate a response to the received prompts using LLM running locally
- Send the response to Fullmetal API
- Clone the following repository
git clone https://github.com/m0dE/fullmetal-agent-example
- Install the project and set your configuration parameters
cd fullmetal-agent-example
npm install
-
Obtain your
FULLMETAL_API_KEY
by following instructions mentioned here -
Create
.env
file in root folder and add the following keys. Remember all these keys are important. If left blank then your agent may not work properly.
FULLMETAL_API_KEY=YOUR_FULLMETAL_API_KEY
#AGENT NAME
AGENT_NAME=
#MODEL INFORMATION
MODEL_NAME=
MODEL_FILE=
# -ngl N, --n-gpu-layers N number of layers to store in VRAM
NGL=28
Download a GGUF LLM file and save it in the models folder. For example, you can use https://huggingface.co/TheBloke/Wizard-Vicuna-7B-Uncensored-GGUF/resolve/main/Wizard-Vicuna-7B-Uncensored.Q2_K.gguf
mkdir models
cd models
wget https://huggingface.co/TheBloke/Wizard-Vicuna-7B-Uncensored-GGUF/resolve/main/Wizard-Vicuna-7B-Uncensored.Q2_K.gguf
Please make sure you have build-essentials
installed on your machine.
To install build-essentials
yum groupinstall "Development Tools"
To build node-llama-cpp with CUDA support please run this command inside of your project:
npx --no node-llama-cpp download --cuda
To run, type:
npm start