Skip to content

⚡️ Transform AI/ML operations: Transparency, Control and Cost Optimization. ⚡️

License

Notifications You must be signed in to change notification settings

NumexaHQ/frugal

Repository files navigation

Numexa

🚀 An open-source Cost & Resource Optimization Platform for LLMs. Be Frugal! 💰

Introduction

Numexa is an AI-driven cost and resource optimization tool designed to enhance operational efficiency. It achieves this by leveraging contextual insights derived from usage metrics. Numexa employs cutting-edge techniques such as intelligent caching and data retrieval, harnessing the power of vector databases to streamline operations. Explore how Numexa can revolutionize your resource management and cost-saving endeavors.

Features

  • 📝 Model agnostic functionality records unlimited requests from various providers like OpenAI, Cohere, Anthropic and more.

  • 📋 Model management

  • 🔔 Alerting & Notification with predefined policies, like error rate, threshold, cost, etc.

  • 💾 Caching, Custom Rate Limits, and Retries,

  • 📊 Track costs and latencies by users, applications, and endpoints

  • 🔜 (Coming soon) Intellegient caching and data retrieval

  • 🔜 (Coming soon) Cost and resource optimization

Development Setup 💻

Prerequisites

Before you begin, ensure you have the following installed on your system:

Getting Started

  1. Clone the Repository:

    git clone <repository_url>
    cd <repository_directory>
    
  2. Build and Start the Services: Run the following commands to build and start the project services

    make all
    docker compose -f docker-compose.dev.yaml up -d
  3. Verify Services: After running the above commands, your project services should be up and running. You can verify this by checking the logs

Community 🤝

Join our #Discord or drop email at [email protected]