Skip to content

Latest commit

 

History

History
130 lines (84 loc) · 3.1 KB

README.md

File metadata and controls

130 lines (84 loc) · 3.1 KB
Raw

Dria HNSW

Dria HNSW is an API that allows you to permissionlessly search knowledge uploaded to Dria. It works over values downloaded from Arweave to a Redis cache, and reads these values directly from Redis. It is written in Rust, and several functions respect the machine architecture for efficiency.

Setup

To run the server, you need to provide a contract ID along with a RocksDB path:

CONTRACT_ID=<contract-id> ROCKSDB_PATH="/path/to/rocksdb" cargo run

Dria HNSW is available as a container:

docker pull firstbatch/dria-hnsw

Tip

The docker image is cross-compiled & built for multiple architectures, so when you pull the image the most efficient code per your architecture will be downloaded!

To see the available endpoints, refer to this section below.

Endpoints

Dria is an Actix server with the following endpoints:

All endpoints return a response in the following format:

  • success: a boolean indicating the success of request
  • code: status code
  • data: response data

Tip

If success is false, the error message will be written in data as a string.

HEALTH

GET /health

A simple healthcheck to see if the server is up.

Response data:

  • A string "hello world!".

FETCH

POST /fetch

Given a list of ids, fetches their corresponding vectors.

Request body:

  • id: an array of integers

Response data:

  • An array of metadatas, index i corresponding to metadata of vector with ID id[i].

QUERY

POST /query

Given a list of ids, fetches their corresponding vectors.

Request body:

  • vector: an array of floats corresponding to the embedding vector
  • top_n: number of results to return
  • query: (optional) the text that belongs to given embedding, yields better results by looking for this text within the results
  • level: (optional) an integer value in range [0, 4] that defines the intensity of search, a larger values takes more time to complete but has higher recall

Response data:

  • An array of objects with the following keys:
    • id: id of the returned vector
    • score: relevance score
    • metadata: metadata of the vector

INSERT_VECTOR

POST /insert_vector

Insert a new vector to HNSW.

Request body:

  • vector: an array of floats corresponding to the embedding vector
  • metadata: (optional) a JSON object that represent metadata for this vector

Response data:

  • A string "Success".

Testing

We have several tests that you can run with:

cargo test

Some tests expect a RocksDB folder present at $HOME/.dria/data/WbcY2a-KfDpk7fsgumUtLC2bu4NQcVzNlXWi13fPMlU, which can easily be downloaded with the Dria CLI if you do not have it:

dria pull WbcY2a-KfDpk7fsgumUtLC2bu4NQcVzNlXWi13fPMlU

The said knowledge is a rather lightweight knowledge that is useful for testing.