Language is All a Graph Needs

This repo presents implementation of the InstructGLM (Instruction-finetuned Graph Language Model) and provide a natural language interface for graph machine learning:

Paper: Language is All a Graph Needs

Paper link: https://arxiv.org/abs/2308.07134

Introduction

We introduce our proposed Instruction-finetuned Graph Language Model, i.e. InstructGLM, a framework utilizing natural language to describe both graph structure and node features to a generative large language model and further addresses graph-related problems by instruction-tuning, which provides a powerful natural language processing interface for graph machine learning.

Usage

0. Clone this repo

git clone https://github.com/agiresearch/InstructGLM.git

1. Download preprocessed data from Arxiv, Cora, PubMed. If you would like to preprocess your own data, please follow the data_preprocess folder. Requiured raw data files for preprocessing can be downloaded from this raw-Arxiv, raw-Cora, raw-PubMed.

2. Download Llama-7b pretrained checkpoint via this Google Drive link, it has been processed by the format conversion script of HuggingFace. Please then put the ./7B folder under the same path with ./scripts folder.

3. Multi-task Multi-prompt Instruction Tuning

bash scripts/train_llama_arxiv.sh 8

Here 8 means using 8 GPUs to conduct parallel instruction tuning with DDP.

4. Validation/ Inference

bash scripts/test_llama_arxiv.sh 8

5. Main key points are summarized in note.txt

Checkpoints

See: Google Drive link.

Citation

Please cite the following paper corresponding to the repository:

@article{ye2023language,
  title={Language is All a Graph Needs},
  author={Ye, Ruosong and Zhang, Caiqi and Wang, Runhui and Xu, Shuyuan and Zhang, Yongfeng},
  journal={EACL},
  year={2024}
}

Acknowledgements

TAPE, GIANT, OGB, P5, OpenP5, and Planetoid