Skip to content

Commit

Permalink
Update readme and docs
Browse files Browse the repository at this point in the history
  • Loading branch information
ogzhanolguncu committed Jan 16, 2024
1 parent b4d345f commit d46ff6a
Show file tree
Hide file tree
Showing 16 changed files with 430 additions and 233 deletions.
220 changes: 213 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,221 @@
# vector-sdk
# Upstash Vector Node.js Client

To install dependencies:
This is the official Node.js client for [Upstash](https://upstash.com/), written in TypeScript.

```bash
bun install
## Documentation

- [**Reference Documentation**](https://upstash.com/docs/vector/overall/getstarted)

## Installation

```
npm install @upstash/vector
pnpm add @upstash/vector
```

To run:
## Usage

### Initializing the client

There are two pieces of configuration required to use the Upstash vector client: an REST token and REST URL. These values can be passed using environment variables or in code through a configuration object. Find your configuration values in the console dashboard at [https://console.upstash.com/](https://console.upstash.com/).

#### Using environment variables

The environment variables used to configure the client are the following:

```bash
bun run index.ts
UPSTASH_VECTOR_REST_URL="your_rest_url"
UPSTASH_VECTOR_REST_TOKEN="your_rest_token"
```

When these environment variables are set, the client constructor does not require any additional arguments.

```typescript
import { fromEnv } from "@upstash/vector";

const index = fromEnv();
```

#### Using a configuration object

If you prefer to pass configuration in code, the constructor accepts a config object containing the `url` and `token` values. This
could be useful if your application needs to interact with multiple projects, each with a different configuration.

```typescript
import { Index } from "@upstash/vector";

const index = new Index({
url: "<UPSTASH_VECTOR_REST_URL>",
token: "<UPSTASH_VECTOR_REST_TOKEN>",
});
```

## Index operations

Upstash vector indexes support operations for working with vector data using operations such as upsert, query, fetch, and delete.

### Targeting an index

To perform data operations on an index, you target it using the `index` method.

```typescript
const index = new Index();

// Now perform index operations
await index.fetch([1, 2, 3], { includeMetadata: true, includeVectors: true });
```

### Targeting an index, with metadata typing

If you are storing metadata alongside your vector values, you can pass a type parameter to `index()` in order to get proper TypeScript typechecking.

```typescript
const index = new Index();

type Metadata = {
title: string,
genre: 'sci-fi' | 'fantasy' | 'horror' | 'action'
}

await index.upsert([{
id: '1234',
vector: [
.... // embedding values
],
metadata: {
title: 'Lord of The Rings',
genre: 'drama',
category: 'classic'
}
}])

const results = await index.query<Metadata>({
vector: [
... // query embedding
],
includeVectors: true,
topK: 1,
})

if (results[0].metadata) {
// Since we passed the Metadata type parameter above,
// we can interact with metadata fields without having to
// do any typecasting.
const { title, genre, category } = movie.metadata;
console.log(`The best match in fantasy was ${title}`)
}
```

This project was created using `bun init` in bun v1.0.4. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime.
### Upsert records

Upstash vector expects records inserted into indexes to have the following form:

```typescript
type UpstashRecord = {
id: number | string;
vector: number[];
metadata?: Record<string, unknown>;
};
```

To upsert some records, you can use the client like so:

```typescript
const index = new Index();

// Prepare your data. The length of each array
// of vector values must match the dimension of
// the index where you plan to store them.
const records = [
{
id: "1",
vector: [0.236, 0.971, 0.559],
},
{
id: "2",
vector: [0.685, 0.111, 0.857],
},
];

// Upsert the data into your index
await index.upsert(records);
```

### Querying

#### Querying with vector values

The query method accepts a large number of options. The dimension of the query vector must match the dimension of your index.

```typescript
type QueryOptions = {
vector: number[];
topK: number;
includeVectors?: boolean;
includeMetadata?: boolean;
};
```

For example, to query by vector values you would pass the `vector` param in the options configuration. For brevity sake this example query vector is tiny (dimension 2), but in a more realistic use case this query vector would be an embedding outputted by a model. Look at the [Example code](#example-code) to see more realistic examples of how to use `query`.

```typescript
> await index.query({ topK: 3, vector: [ 0.22, 0.66 ]})
{
matches: [
{
id: '6345',
score: 1.00000012,
vector: [],
metadata: undefined
},
{
id: '1233',
score: 1.00000012,
vector: [],
metadata: undefined
},
{
id: '4142',
score: 1.00000012,
vector: [],
metadata: undefined
}
],
namespace: ''
}
```

You include options to `includeMetadata: true` or `includeVectors: true` if you need this information. By default these are not returned to keep the response payload small.

### Update a record

You may want to update vector `vector` or `metadata`. Specify the id and the attribute value you want to update.

```typescript
await index.upsert({
id: "18593",
metadata: { genre: "romance" },
});
```

### Fetch records by their IDs

```typescript
const fetchResult = await index.fetch(["id-1", "id-2"]);
```

### Delete records

For convenience there are several delete-related options. You can verify the results of a delete operation by trying to `fetch()` a record.

#### Delete one

```typescript
await index.delete("id-to-delete");
```

#### Delete many by id

```typescript
await index.delete(["id-1", "id-2", "id-3"]);
```
10 changes: 5 additions & 5 deletions index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export type { Requester, UpstashRequest, UpstashResponse };
* Connection credentials for upstash vector.
* Get them from https://console.upstash.com/vector/<uuid>
*/
export type VectorConfig = {
export type IndexConfig = {
/**
* UPSTASH_VECTOR_REST_URL
*/
Expand All @@ -39,13 +39,13 @@ export class Index extends core.Index {
*
* @example
* ```typescript
* const vector = new Vector({
* const index = new Index({
* url: "<UPSTASH_VECTOR_REST_URL>",
* token: "<UPSTASH_VECTOR_REST_TOKEN>",
* });
* ```
*/
constructor(config: VectorConfig);
constructor(config: IndexConfig);

/**
* Create a new vector client by providing a custom `Requester` implementation
Expand All @@ -65,7 +65,7 @@ export class Index extends core.Index {
* ```
*/
constructor(requesters: Requester);
constructor(configOrRequester: VectorConfig | Requester) {
constructor(configOrRequester: IndexConfig | Requester) {
if ("request" in configOrRequester) {
super(configOrRequester);
return;
Expand Down Expand Up @@ -109,7 +109,7 @@ export class Index extends core.Index {
* This tries to load `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` from
* your environment using `process.env`.
*/
static fromEnv(config?: Omit<VectorConfig, "url" | "token">): Index {
static fromEnv(config?: Omit<IndexConfig, "url" | "token">): Index {
const url = process?.env.UPSTASH_VECTOR_REST_URL;
if (!url) {
throw new Error(
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"module": "./dist/index.mjs",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"version": "0.1.0",
"version": "v0.1.0-alpha",
"keywords": [
"vector",
"upstash",
Expand Down
21 changes: 17 additions & 4 deletions src/commands/client/delete/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ describe("DELETE", () => {
);
await Promise.all(upsertPromises);

const deletionResult = await new DeleteCommand({ ids: idsToUpsert }).exec(client);
const deletionResult = await new DeleteCommand(idsToUpsert).exec(client);
expect(deletionResult).toBeTruthy();
});

Expand All @@ -29,8 +29,21 @@ describe("DELETE", () => {
);
await Promise.all(upsertPromises);

await new DeleteCommand({ ids: idsToUpsert }).exec(client);
const res1 = await new DeleteCommand({ ids: idsToUpsert }).exec(client);
expect(res1).toBeNull();
await new DeleteCommand(idsToUpsert).exec(client);
const res1 = await new DeleteCommand(idsToUpsert).exec(client);
expect(res1).toEqual({
deleted: 0,
});
});

test("should delete single item", async () => {
const initialVector = [6.6, 7.7];
const id = randomID();
await new UpsertCommand({ id, vector: initialVector }).exec(client);

const res1 = await new DeleteCommand(id).exec(client);
expect(res1).toEqual({
deleted: 1,
});
});
});
39 changes: 9 additions & 30 deletions src/commands/client/delete/index.ts
Original file line number Diff line number Diff line change
@@ -1,34 +1,13 @@
import { Command } from "@commands/command";

/**
* Payload Type Definition for DeleteCommand
*
* This type defines the structure of the payload specifically used in the DeleteCommand.
*
* Properties:
* - ids: An array of numbers or strings representing the unique identifiers of the records to be deleted. These could be database IDs, unique keys, or any identifier used to uniquely refer to records in a specific context.
*
* Usage:
* This type is typically used in scenarios where a batch deletion of records is required. The `ids` array allows specifying multiple records for deletion in a single command, thereby facilitating efficient bulk operations.
*/
type Payload = {
ids: number[] | string[];
};

/**
* DeleteCommand Class
*
* This class extends the generic Command class to implement the deletion functionality.
*
* Example:
* ```
* const deletionIds = [123, 456, 789];
* const deleteCommand = new DeleteCommand({ ids: deletionIds });
* // Use deleteCommand to execute the deletion operation
* ```
*/
export class DeleteCommand extends Command<string> {
constructor(payload: Payload) {
super(payload.ids, "delete");
export class DeleteCommand extends Command<{ deleted: number }> {
constructor(id: (number[] | string[]) | number | string) {
const finalArr = [];
if (Array.isArray(id)) {
finalArr.push(...id);
} else {
finalArr.push(id);
}
super(finalArr, "delete");
}
}
Loading

0 comments on commit d46ff6a

Please sign in to comment.