.cursorrules

Below is a formatted system prompt consolidating all given instructions and guidelines, ensuring that all details—including those about the not-well-known “screenpipe” API—are preserved. This system prompt is intended for another AI to follow during its execution.

System Prompt:

always console log like this:

console.log("0xHypr", 'variableName', variable )

Assistant Guidelines:
	•	When i use ai models most likely i use Model: gpt-4o
	•	Development and Packaging:
	•	Use pnpm as the package manager.
	•	Maintain a monorepo structure.
	•	Follow kebab-case for all file and folder naming conventions.
	•	Utilize TypeScript and React throughout the project.
	•	Implement a “return early” coding pattern to enhance code readability and error handling.



Project Structure:
- Main app is in packages/desktop
- Frontend: packages/desktop/src/renderer
- Backend: packages/desktop/src/electron
- packages/landing-v0 containts landing page and in the future all account logins and signups




	UI and Styling:
	•	Integrate the shadcn/ui components library.
	•	Priority in UI design: Ease of Use > Aesthetics > Performance.
	•	Employ Tailwind CSS for utility-first styling and rapid UI development.

Recommended Libraries and Tools:
	1.	State Management:
	•	Use React Context API for simple state management needs.
	•	Use Zustand for lightweight, scalable state management compatible with React Server Components.
	2.	Form Handling:
	•	Employ React Hook Form for efficient, flexible form handling and built-in validation features.
	3.	Data Fetching:
	•	Use TanStack Query (React Query) for efficient data fetching, caching, and revalidation flows.
	4.	Authentication:
	•	Integrate authentication via Clerk.
	5.	Animations:
	•	Use Framer Motion for smooth animations and transitions.
	6.	Icons:
	•	Incorporate the Lucide React icon set for a wide array of open-source icons.

AI Integration with Vercel AI SDK:
	•	Utilize the Vercel AI SDK (TypeScript toolkit) to build AI-driven features in React/Next.js.
	•	Implement conversational UIs using the useChat hook, which manages chat states and streams AI responses.

Using Tools with useChat and streamText:
	•	Tool Types:
	•	Server-side tools auto-executed on the server.
	•	Client-side tools auto-executed on the client.
	•	User-interactive tools that require a confirmation dialog or user input.
	•	Workflow:
	1.	The user inputs a message in the chat UI.
	2.	The message is sent to the API route.
	3.	The language model may generate tool calls using streamText.
	4.	Tool calls are forwarded to the client.
	5.	Server-side tools execute server-side and return results to the client.
	6.	Client-side tools execute automatically via the onToolCall callback.
	7.	User-interactive tools display a confirmation dialog. The user’s choice is handled via toolInvocations.
	8.	After the user interacts, use addToolResult to incorporate the final result into the chat.
	9.	If tool calls exist in the last message and all results are now available, the client re-sends messages to the server for further processing.
	•	Note: Set maxSteps > 1 in useChat options to enable multiple iterations. By default, multiple iterations are disabled for compatibility reasons.

Example Implementation:
	•	API Route Example (app/api/chat/route.ts):

import { openai } from '@ai-sdk/openai';
import { streamText } from 'ai';
import { z } from 'zod';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const result = await streamText({
    model: openai('gpt-4o'),
    messages,
    tools: {
      // Server-side tool:
      getWeatherInformation: {
        description: 'Show the weather in a given city to the user.',
        parameters: z.object({ city: z.string() }),
        execute: async ({ city }: { city: string }) => {
          const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy'];
          return `${city} is currently ${weatherOptions[Math.floor(Math.random() * weatherOptions.length)]}.`;
        },
      },
      
      // Client-side, user-interactive tool:
      askForConfirmation: {
        description: 'Ask the user for confirmation.',
        parameters: z.object({
          message: z.string().describe('The message to ask for confirmation.'),
        }),
      },

      // Automatically executed client-side tool:
      getLocation: {
        description: 'Get the user location after confirmation.',
        parameters: z.object({}),
      },
    },
  });

  return result.toDataStreamResponse();
}


	•	generateObject() Usage Example:

import { openai } from '@ai-sdk/openai';
import { generateObject } from 'ai';
import { z } from 'zod';

const { object } = await generateObject({
  model: openai('gpt-4-turbo'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.string()),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a lasagna recipe.',
});

console.log(JSON.stringify(object, null, 2));


	•	Client-Side Page Example (app/page.tsx):

'use client';

import { ToolInvocation } from 'ai';
import { Message, useChat } from 'ai/react';

export default function Chat() {
  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    addToolResult,
  } = useChat({
    maxSteps: 5,
    async onToolCall({ toolCall }) {
      if (toolCall.toolName === 'getLocation') {
        const cities = ['New York', 'Los Angeles', 'Chicago', 'San Francisco'];
        return {
          city: cities[Math.floor(Math.random() * cities.length)],
        };
      }
    },
  });

  return (
    <>
      {messages?.map((m: Message) => (
        <div key={m.id}>
          <strong>{m.role}:</strong> {m.content}
          {m.toolInvocations?.map((toolInvocation: ToolInvocation) => {
            const toolCallId = toolInvocation.toolCallId;
            const addResult = (result: string) => addToolResult({ toolCallId, result });

            if (toolInvocation.toolName === 'askForConfirmation') {
              return (
                <div key={toolCallId}>
                  {toolInvocation.args.message}
                  <div>
                    {'result' in toolInvocation ? (
                      <b>{toolInvocation.result}</b>
                    ) : (
                      <>
                        <button onClick={() => addResult('Yes')}>Yes</button>
                        <button onClick={() => addResult('No')}>No</button>
                      </>
                    )}
                  </div>
                </div>
              );
            }

            return 'result' in toolInvocation ? (
              <div key={toolCallId}>
                <em>Tool ({toolInvocation.toolName}):</em> {toolInvocation.result}
              </div>
            ) : (
              <div key={toolCallId}>
                <em>Executing {toolInvocation.toolName}...</em>
              </div>
            );
          })}
          <br />
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Type your message..." />
      </form>
    </>
  );
}



Additional Notes:
	•	Ensure proper handling of all tool invocations to maintain a seamless user experience.
	•	Regularly update dependencies and libraries to their latest versions for improved performance, security, and stability.
	•	Thoroughly test the chatbot to handle edge cases and unexpected user inputs.

Access to Screenpipe:

This project uses a local service named screenpipe that watches screen and audio, storing data in a local database. The application (hyprsqrl) integrates with the screenpipe API (accessible at http://localhost:3030) to gather OCR data and other information, potentially using it to create tasks or calendar events.

Screenpipe API Reference:
	•	Search API:
	•	Endpoint: GET /search
	•	Description: Searches captured data (OCR text, audio transcription, UI elements) from the local database.
	•	Query Parameters:
	•	q (string, optional): A single-word search term.
	•	content_type (enum): One of ocr, audio, or ui.
	•	limit (int, default=20): Maximum results per page.
	•	offset (int): Pagination offset.
	•	start_time (timestamp, optional): Filter by start timestamp.
	•	end_time (timestamp, optional): Filter by end timestamp.
	•	app_name (string, optional): Filter by application name.
	•	window_name (string, optional): Filter by window name.
	•	include_frames (bool, optional): Include base64 encoded frame data.
	•	min_length (int, optional): Minimum content length.
	•	max_length (int, optional): Maximum content length.
	•	speaker_ids (int[], optional): Filter by speaker IDs.
Sample Request:

curl "http://localhost:3030/search?q=meeting&content_type=ocr&limit=10"

Sample Response:

{
  "data": [
    {
      "type": "OCR",
      "content": {
        "frame_id": 123,
        "text": "meeting notes",
        "timestamp": "2024-03-10T12:00:00Z",
        "file_path": "/frames/frame123.png",
        "offset_index": 0,
        "app_name": "chrome",
        "window_name": "meeting",
        "tags": ["meeting"],
        "frame": "base64_encoded_frame_data"
      }
    }
  ],
  "pagination": {
    "limit": 5,
    "offset": 0,
    "total": 100
  }
}


	•	Audio Devices API:
	•	Endpoint: GET /audio/list
	•	Description: Lists available audio input/output devices.
Sample Response:

[
  {
    "name": "built-in microphone",
    "is_default": true
  }
]


	•	Monitors (Vision) API:
	•	Endpoint: POST /vision/list
	•	Description: Lists available monitors/displays.
Sample Response:

[
  {
    "id": 1,
    "name": "built-in display",
    "width": 2560,
    "height": 1600,
    "is_default": true
  }
]


	•	Tags API:
	•	Endpoint (Add Tags): POST /tags/:content_type/:id
	•	Endpoint (Remove Tags): DELETE /tags/:content_type/:id
	•	Description: Manage tags for content items (vision or audio).
Add Tags Request Example:

{
  "tags": ["important", "meeting"]
}

Add Tags Response Example:

{
  "success": true
}


	•	Pipes API:
	•	Endpoints:
	•	GET /pipes/list (List pipes)
	•	POST /pipes/download (Download a pipe from a remote URL)
	•	POST /pipes/enable (Enable a specified pipe)
	•	POST /pipes/disable (Disable a specified pipe)
	•	POST /pipes/update (Update a pipe’s configuration)
Download Pipe Request Example:

{
  "url": "https://github.com/user/repo/pipe-example"
}

Enable Pipe Request Example:

{
  "pipe_id": "pipe-example"
}


	•	Speakers API:
	•	List Unnamed Speakers:
	•	Endpoint: GET /speakers/unnamed
	•	Query: limit, offset, speaker_ids
	•	Search Speakers:
	•	Endpoint: GET /speakers/search
	•	Query: name (string)
	•	Update Speaker:
	•	Endpoint: POST /speakers/update
	•	Request Body:

{
  "id": 123,
  "name": "john doe",
  "metadata": "{\"role\": \"engineer\"}"
}


	•	Delete Speaker:
	•	Endpoint: POST /speakers/delete
	•	Request Body:

{
  "id": 123
}


	•	Get Similar Speakers:
	•	Endpoint: GET /speakers/similar
	•	Query: speaker_id, limit
	•	Merge Speakers:
	•	Endpoint: POST /speakers/merge
	•	Request Body:

{
  "speaker_to_keep_id": 123,
  "speaker_to_merge_id": 456
}


	•	Mark as Hallucination:
	•	Endpoint: POST /speakers/hallucination
	•	Request Body:

{
  "speaker_id": 123
}


	•	Health API:
	•	Endpoint: GET /health
	•	Description: Retrieves system health status.
Sample Response:

{
  "status": "healthy",
  "last_frame_timestamp": "2024-03-10T12:00:00Z",
  "last_audio_timestamp": "2024-03-10T12:00:00Z",
  "last_ui_timestamp": "2024-03-10T12:00:00Z",
  "frame_status": "ok",
  "audio_status": "ok",
  "ui_status": "ok",
  "message": "all systems functioning normally"
}


	•	Stream Frames API:
	•	Endpoint: GET /stream/frames
	•	Description: Streams frames as Server-Sent Events (SSE).
	•	Query Parameters: start_time, end_time
Sample Request:

curl "http://localhost:3030/stream/frames?start_time=2024-03-10T12:00:00Z&end_time=2024-03-10T13:00:00Z"

Sample Event Data:

{
  "timestamp": "2024-03-10T12:00:00Z",
  "devices": [
    {
      "device_id": "screen-1",
      "frame": "base64_encoded_frame_data"
    }
  ]
}

Instructions for Other AI Agents:
	•	Strictly adhere to the above structure, styles, and naming conventions.
	•	Always begin text responses with 0xHypr.
	•	Respect the described architectural and stylistic guidelines.
	•	Fully utilize the provided APIs, including the screenpipe API, to integrate OCR and other data into the application as needed.
	•	When implementing tools, ensure their correct registration, invocation, and result handling based on the described workflow.
	•	Follow the “ease of use” first design principle to ensure the final UI and user experience are smooth and intuitive.
	•	Keep all details about the “screenpipe” API to ensure correct interaction with its endpoints and data formats.

End of System Prompt