Promptable App

Appearance

Core AI Module ​

The packages/core/src/AI directory contains modules dedicated to integrating Artificial Intelligence capabilities, primarily focusing on Large Language Models (LLMs) and vector embeddings. The structure is designed to be extensible, particularly for supporting various LLM providers. Given its active development, the details below reflect its current state and established patterns.

Core Services ​

  • LLMService.ts: This service is the main orchestrator for server-side interactions with LLMs. It is designed with a provider model in mind, allowing different LLM backends to be used. Currently, it integrates an OpenAIProvider. Key functionalities include:

    • Fetching standard chat completions.
    • Fetching structured chat completions, where the LLM's output is expected to conform to a dynamically generated Zod schema. This promotes reliable and predictable data structures from LLM responses.
    • It employs a lazy initialisation pattern for LLM providers, ensuring that API keys and other environment-specific configurations are loaded only when needed.
    • A singleton-like access pattern is provided via the getLLMService function.

  • VectorService.ts: This service manages operations related to vector embeddings, which are crucial for tasks like semantic search and Retrieval Augmented Generation (RAG). Its responsibilities include:

    • Storing document chunks alongside their vector embeddings in a Supabase database.
    • A method for generating embeddings (currently a placeholder, intended to call external embedding services like OpenAI).
    • Processing and chunking text content from documents before embedding and storage.
    • Querying the database for documents semantically similar to a given query text, using a Supabase RPC (match_documents) for the similarity search.

API Clients ​

  • APIClients.ts: This file provides the LLMApiClient class, which is intended for client-side (or internal application) use to interact with the application's own backend API endpoints for LLM operations (e.g., /app/api/llm/chat). This contrasts with LLMService.ts, which handles direct communication with LLM providers. This separation is a key pattern, ensuring that direct LLM provider access and API keys are managed server-side. It supports:
    • Standard chat completions.
    • Structured chat completions, passing a schema configuration to the backend.
    • A specialised getRequirementsCompletion method for a specific use-case.
    • It also uses a lazy initialisation pattern for its instance via getLLMApiClient.

LLM Providers ​

Located under the providers/ subdirectory, these modules implement the actual communication with specific LLM vendors.

  • providers/openaiProvider.ts: This is the concrete implementation for OpenAI models.
    • It uses the official OpenAI SDK.
    • It integrates with @instructor-ai/instructor to facilitate reliable structured data extraction from OpenAI's responses using Zod schemas.
    • Includes error handling with a fallback to standard completion if the structured output generation fails.
    • Proxies requests through Helicone for monitoring and potentially other features.
    • Allows the specific OpenAI model (e.g., gpt-4o) to be set dynamically.

Module Exports ​

  • index.ts: This file serves as the primary CJS-compatible export point for the AI module, consolidating and re-exporting the services, clients, and providers for easy consumption by other parts of the application.

Key Architectural Patterns ​

Several important patterns are evident in this module, guiding its current and future development:

  1. Provider Abstraction for LLMs: The design clearly separates the core LLM interaction logic (LLMService) from the specific implementations for each LLM provider (like OpenAIProvider). This allows for easier addition of new providers (e.g., Anthropic, Gemini) by creating new classes under the providers/ directory.
  2. Lazy Initialisation: Critical services and providers that depend on runtime environment variables (especially API keys) are initialised lazily. This is a robust pattern that accommodates various deployment and execution environments.
  3. Structured Output with Zod: There's a strong emphasis on obtaining structured and validated data from LLMs using Zod schemas, primarily facilitated by the @instructor-ai/instructor library. LLMService further supports dynamic Zod schema creation from configuration objects.
  4. Client-Server Separation for LLM API Calls: The distinction between LLMApiClient (for use within the application, calling its own backend) and LLMService (for backend communication with LLM providers) is a deliberate architectural choice. It enhances security by keeping LLM API keys on the server and allows for centralised control, caching, or other logic.
  5. Foundation for Retrieval Augmented Generation (RAG): VectorService provides the essential components for RAG systems, including text chunking, (mocked) embedding generation, vector storage, and similarity search capabilities.