Cache wrapper

[BBC-Esq]

Read commit notes for details, but here are some brainstorms to evaluate if/when basic metrics are implemented:

Caching Strategy	Description
RotatingKVCache	Rotates cache entries based on a fixed schedule, discarding the oldest entries as new ones are added to maintain a constant cache size.
LRU (Least Recently Used)	Evicts the least recently accessed items first, ensuring that frequently accessed data remains in the cache while older, unused data is removed.
LFU (Least Frequently Used)	Removes items that are accessed least frequently, prioritizing entries that are used more often to stay in the cache longer.
FIFO (First In First Out)	Discards the oldest entries based on the order they were added, without considering how frequently or recently they have been accessed.
Time-based Expiration	Invalidates cache entries after a specified time period has elapsed since their last access or creation, ensuring that stale data is periodically removed.
Size-based Eviction	Limits the cache size by removing entries once a predefined size threshold is reached, maintaining the cache within memory constraints.
Custom Replacement Policies	Allows the implementation of user-defined rules for cache eviction based on specific application requirements, such as combining multiple criteria like recency and frequency.
Multi-Level Caching	Utilizes multiple layers of caching (e.g., in-memory, disk-based) to optimize performance and resource utilization by distributing data across different storage mediums.
Cache Preloading	Loads frequently used or common prompts into the cache ahead of time to reduce processing latency during runtime operations.
Dynamic Cache Sizing	Adjusts the cache size dynamically based on workload, memory availability, or access patterns to optimize resource utilization and maintain performance under varying conditions.
Automatic Cache Optimization Based on Usage Patterns	Analyzes cache usage metrics to automatically fine-tune caching strategies, such as adjusting cache size or replacement policies, to enhance cache efficiency and performance.

---