# DSPy.rb

> Build typed agents and model-backed programs in Ruby with signatures, tools, bounded execution, and explicit evaluation.

DSPy.rb turns typed task contracts into provider requests and validates returned structure. Ruby code still owns composition, side effects, failure policy, and evaluation.

## Overview

DSPy.rb is a Ruby framework for building agents and model-backed programs with programmatic prompts. It provides:

- **Type-safe signatures** - Define inputs/outputs with Sorbet types
- **Modular components** - Compose and reuse LLM logic
- **Explicit optimization** - Run an optimizer against examples, a metric, and an execution budget
- **Operational controls** - Add observability, tests, and application-owned error handling

## Packages and capability boundaries

The canonical [package and capability matrix](https://oss.vicente.services/dspy.rb/getting-started/packages/) separates install availability from support status and model-specific behavior. It is generated from `docs/src/_data/package_capabilities.yml`.

- `dspy` — core; supported; require `dspy`
- `dspy-openai` — provider-adapter; supported; require `dspy/openai`
- `dspy-anthropic` — provider-adapter; supported; require `dspy/anthropic`
- `dspy-gemini` — provider-adapter; supported; require `dspy/gemini`
- `dspy-ruby_llm` — provider-adapter; preview; require `dspy/ruby_llm`
- `dspy-code_act` — optional-feature; supported; require `dspy/code_act`
- `dspy-datasets` — optional-feature; supported; require `dspy/datasets`
- `dspy-evals` — optional-feature; supported; require `dspy/evals`
- `dspy-miprov2` — optional-feature; supported; require `dspy/miprov2`
- `dspy-gepa` — optional-feature; supported; require `dspy/gepa`
- `gepa` — supporting-library; supporting; require `gepa`
- `dspy-o11y` — optional-feature; supported; require `dspy/o11y`
- `dspy-o11y-langfuse` — optional-feature; supported; require `dspy/o11y/langfuse`
- `dspy-deep_search` — optional-feature; supported; require `dspy/deep_search`
- `dspy-deep_research` — optional-feature; supported; require `dspy/deep_research`
- `dspy-schema` — supporting-library; supporting; require `dspy/schema`
- `sorbet-toon` — supporting-library; preview; require `sorbet/toon`

`DSPY_WITH_*` flags select local gemspecs only when developing this monorepo. Application users install the named gem. Provider capabilities still vary by adapter, model, endpoint, and SDK version. `dspy-code_act` executes generated Ruby and does not provide a sandbox.

## Core Concepts

### 1. Signatures
Define interfaces between your app and LLMs using Ruby types:

```ruby
class EmailClassifier < DSPy::Signature
  description "Classify customer support emails by category and priority"
  
  class Priority < T::Enum
    enums do
      Low = new('low')
      Medium = new('medium')
      High = new('high')
      Urgent = new('urgent')
    end
  end
  
  input do
    const :email_content, String
    const :sender, String
  end
  
  output do
    const :category, String
    const :priority, Priority  # Type-safe enum with defined values
    const :confidence, Float
  end
end
```

### 2. Modules
Compose model calls and agents from typed building blocks:

- **Predict** - Basic LLM calls with signatures
- **ChainOfThought** - Step-by-step reasoning
- **ReAct** - Tool-using agents
- **CodeAct** - Dynamic code generation agents (install the `dspy-code_act` gem)

#### Module task guides

Lifecycle definitions and the runnable callback example belong to [Module Lifecycle Callbacks](https://oss.vicente.services/dspy.rb/advanced/module-lifecycle-callbacks/). Fiber-local model resolution and propagation remain in [Module Runtime Context](https://oss.vicente.services/dspy.rb/advanced/module-runtime-context/).

Application-owned batching, joins, failure policy, and measured limits belong to [Concurrent Predictions](https://oss.vicente.services/dspy.rb/advanced/concurrent-predictions/).

### 3. Tools & Toolsets
Create type-safe tools for agents with comprehensive Sorbet support:

```ruby
# Enum-based tool with automatic type conversion
class CalculatorTool < DSPy::Tools::Base
  tool_name 'calculator'
  tool_description 'Performs arithmetic operations with type-safe enum inputs'
  
  class Operation < T::Enum
    enums do
      Add = new('add')
      Subtract = new('subtract')
      Multiply = new('multiply')
      Divide = new('divide')
    end
  end
  
  sig { params(operation: Operation, num1: Float, num2: Float).returns(T.any(Float, String)) }
  def call(operation:, num1:, num2:)
    case operation
    when Operation::Add then num1 + num2
    when Operation::Subtract then num1 - num2
    when Operation::Multiply then num1 * num2
    when Operation::Divide
      return "Error: Division by zero" if num2 == 0
      num1 / num2
    end
  end
end

# Multi-tool toolset with rich types
class DataToolset < DSPy::Tools::Toolset
  toolset_name "data_processing"
  
  class Format < T::Enum
    enums do
      JSON = new('json')
      CSV = new('csv')
      XML = new('xml')
    end
  end
  
  class ProcessingConfig < T::Struct
    const :max_rows, Integer, default: 1000
    const :include_headers, T::Boolean, default: true
    const :encoding, String, default: 'utf-8'
  end
  
  tool :convert, description: "Convert data between formats"
  tool :validate, description: "Validate data structure"
  
  sig { params(data: String, from: Format, to: Format, config: T.nilable(ProcessingConfig)).returns(String) }
  def convert(data:, from:, to:, config: nil)
    config ||= ProcessingConfig.new
    "Converted from #{from.serialize} to #{to.serialize} with config: #{config.inspect}"
  end
  
  sig { params(data: String, format: Format).returns(T::Hash[String, T.any(String, Integer, T::Boolean)]) }
  def validate(data:, format:)
    {
      valid: true,
      format: format.serialize,
      row_count: 42,
      message: "Data validation passed"
    }
  end
end
```

`Toolset.to_tools` calls `new` with no arguments and returns one proxy per declared method, so exported Toolsets need a zero-argument constructor. A schema describes and converts call arguments; it does not authorize an operation, sanitize input, sandbox side effects, impose a timeout, or bound an agent loop.

The core gem's text-processing feature is loaded explicitly and exports stable names:

```ruby
require "dspy/tools/text_processing_toolset"

tools = DSPy::Tools::TextProcessingToolset.to_tools
tools.map(&:name)
# => ["text_grep", "text_wc", "text_rg", "text_extract_lines",
#     "text_filter_lines", "text_unique_lines", "text_sort_lines",
#     "text_summarize_text"]

word_count = tools.to_h { |tool| [tool.name, tool] }.fetch("text_wc")
word_count.call(text: "one two\nthree")
# => "Lines: 2, Words: 3, Characters: 13"
```

`text_grep` and `text_rg` use shell commands and temporary files. Their current implementation has no elapsed-time or output cap, suppresses command stderr, and is intended only for trusted patterns in a controlled runtime. `max_iterations` limits model-directed steps; it does not cancel a running tool.

### 4. Type System & Discriminators
DSPy.rb uses sophisticated type discrimination for complex data structures:

- **Automatic `_type` field injection** - DSPy adds discriminator fields to structs for type safety
- **Union type support** - Struct unions can use `_type` when their members are distinguishable; scalar and ambiguous unions need an explicit application boundary
- **Reserved field name** - Avoid defining your own `_type` fields in structs
- **Recursive filtering** - `_type` fields filtered during deserialization at all nesting levels

### 5. Recursive Types with `$defs`

DSPy.rb supports recursive types in structured outputs using JSON Schema `$defs`:

```ruby
class TreeNode < T::Struct
  const :value, String
  const :children, T::Array[TreeNode], default: []  # Self-reference
end
```

The schema generator creates `#/$defs/TreeNode` references, compatible with OpenAI and Gemini.

Use `default: []` when omission means an empty array. In the base DSPy schema and constructor, `T.nilable` permits `nil` while `default:` permits omission. Adapters may tighten requiredness—OpenAI strict structured outputs require every property—so inspect the adapter schema you deploy.

### 6. Field Descriptions for T::Struct

Add field-level `description:` kwargs that flow to JSON Schema:

```ruby
class ASTNode < T::Struct
  const :node_type, String, description: 'The type of node (heading, paragraph, etc.)'
  const :text, String, default: "", description: 'Text content of the node'
  const :level, Integer, default: 0  # No description needed - self-explanatory
end

# Access programmatically
ASTNode.field_descriptions[:node_type]  # => "The type of node..."
```

### 7. Optimization
Improve accuracy with real data:

- **MIPROv2** - Advanced multi-prompt optimization with bootstrap sampling and Bayesian optimization
- **GEPA (Genetic-Pareto Reflective Prompt Evolution)** - Reflection-driven instruction rewrite loop with feedback maps, experiment tracking, and telemetry
- **Evaluation** - Comprehensive framework with built-in and custom metrics, error handling, and batch processing

> Install the optional `dspy-gepa` gem (and set `DSPY_WITH_GEPA=1` when working from this monorepo) before using the GEPA teleprompter.

```ruby
# Evolve instructions with GEPA
feedback_map = {
  'self' => ->(predictor_output:, module_inputs:, **) do
    DSPy::Prediction.new(score: 1.0, feedback: "Call out mistakes for #{module_inputs.input_values[:question]}")
  end
}

gepa = DSPy::Teleprompt::GEPA.new(metric: metric, feedback_map: feedback_map)
optimized = gepa.compile(program, trainset: train_examples, valset: val_examples)
```

## Quick Start

The canonical executable guide is [Quick Start](https://oss.vicente.services/dspy.rb/getting-started/quick-start/). Add the core `dspy` gem and the `dspy-openai` adapter to a Gemfile, run `bundle install`, export `OPENAI_API_KEY`, save the following as `classify.rb`, and run `bundle exec ruby classify.rb`.

```ruby
source 'https://rubygems.org'

gem 'dspy'
gem 'dspy-openai'
```

```bash
export OPENAI_API_KEY=sk-your-key-here
```

```ruby
require 'dspy'

class Classify < DSPy::Signature
  description "Classify the sentiment of a sentence."

  class Sentiment < T::Enum
    enums do
      Positive = new('positive')
      Negative = new('negative')
      Neutral = new('neutral')
    end
  end

  input do
    const :sentence, String
  end

  output do
    const :sentiment, Sentiment
    const :confidence, Float
  end
end

DSPy.configure do |config|
  config.lm = DSPy::LM.new(
    'openai/gpt-4o-mini',
    api_key: ENV.fetch('OPENAI_API_KEY')
  )
end

classifier = DSPy::Predict.new(Classify)
result = classifier.call(sentence: "This book was fun to read!")

puts result.sentiment.serialize
puts result.confidence
```

Missing `OPENAI_API_KEY` raises Ruby's `KeyError` before LM initialization. Output values vary, but a successful result contains a `Classify::Sentiment` and a `Float`. Typed output validation constrains shape; it does not establish factual or task correctness.

## Provider packages

Choose from the canonical [package and capability matrix](https://oss.vicente.services/dspy.rb/getting-started/packages/). The provider rows below are derived from that matrix:

- `dspy-openai` — prefixes: openai, openrouter, ollama. Boundary: Structured output, vision, streaming behavior, parameters, and fallback behavior depend on the selected model and compatible endpoint.
- `dspy-anthropic` — prefixes: anthropic. Boundary: Model and API-version support determine structured-output, tool, image, document, and streaming behavior.
- `dspy-gemini` — prefixes: gemini. Boundary: Schema, media, safety, and streaming behavior depend on the selected Gemini model and SDK/API version.
- `dspy-ruby_llm` — prefixes: ruby_llm. Boundary: Provider coverage is not a uniform capability promise. Registry data, explicit provider overrides, authentication, attachments, schemas, and streaming vary by underlying provider, model, RubyLLM version, and provider SDK; document input is currently restricted to Anthropic.

## Evaluation & Metrics

Comprehensive testing and measurement framework. Typed DSPy::Scores, built-in score evaluators, evaluation export, and the bounded Langfuse delivery lifecycle belong to [Score Reporting](https://oss.vicente.services/dspy.rb/production/score-reporting/):

```ruby
# Basic evaluation with built-in metrics
metric = DSPy::Metrics.exact_match(field: :answer, case_sensitive: false)
evaluator = DSPy::Evals.new(predictor, metric: metric)

# Type-safe examples using DSPy::Example
test_examples = [
  DSPy::Example.new(
    signature_class: YourSignature,
    input: { question: "What is 2+2?" }, 
    expected: { answer: "4" }
  )
]

result = evaluator.evaluate(test_examples, display_progress: true)
puts "Pass rate: #{result.pass_rate}"        # => 0.95
puts "Total: #{result.total_examples}"       # => 100
puts "Passed: #{result.passed_examples}"     # => 95

# Advanced metrics with detailed results
numeric_metric = DSPy::Metrics.numeric_difference(field: :score, tolerance: 0.1)

# Custom multi-factor metrics
quality_metric = ->(example, prediction) do
  return 0.0 unless prediction
  score = 0.0
  score += 0.5 if prediction.answer == example.expected[:answer]  # Accuracy
  score += 0.3 if prediction.explanation&.length&.> 50           # Completeness  
  score += 0.2 if prediction.confidence&.> 0.8                   # Confidence
  score
end

# Error-resilient batch evaluation
evaluator = DSPy::Evals.new(
  predictor,
  metric: quality_metric,
  max_errors: 3,              # Stop after 3 errors
  provide_traceback: true     # Include stack traces
)

batch_result = evaluator.evaluate(large_test_set)
error_count = batch_result.results.count { |r| r.metrics[:error] }

# Built-in metrics: exact_match, contains, numeric_difference, composite_and
```

## MIPROv2 Optimization

Advanced multi-prompt optimization with bootstrap sampling and Bayesian optimization:

```ruby
# Auto-configuration modes for different needs
light_optimizer = DSPy::Teleprompt::MIPROv2::AutoMode.light(metric: your_metric)      # 6 trials, greedy
medium_optimizer = DSPy::Teleprompt::MIPROv2::AutoMode.medium(metric: your_metric)    # 12 trials, adaptive  
heavy_optimizer = DSPy::Teleprompt::MIPROv2::AutoMode.heavy(metric: your_metric)      # 18 trials, Bayesian

# Custom configuration with Bayesian optimization using dry-configurable
optimizer = DSPy::Teleprompt::MIPROv2.new(metric: custom_metric)
optimizer.configure do |config|
  config.optimization_strategy = :bayesian  # or :greedy, :adaptive
  config.num_trials = 15
  config.num_instruction_candidates = 6
end

# Run optimization
program = DSPy::ChainOfThought.new(YourSignature)  
result = optimizer.compile(program, trainset: training_examples, valset: validation_examples)

puts "Best score: #{result.best_score_value}"
optimized_program = result.optimized_program
```

## Main Features

### Type Safety
- Sorbet integration for compile-time checks
- Automatic JSON schema generation
- Type discrimination with `_type` field handling for union types and structs
- Enum types for controlled outputs
- Struct types for complex data

### Composability
- Chain modules together
- Share signatures across modules
- Swap predictors without changing logic
- Build reusable components

### Event System
DSPy provides two ways to subscribe to runtime events:

**Module-scoped subscriptions** (preferred for agents):
```ruby
class MyAgent < DSPy::Module
  subscribe 'lm.tokens', :track_tokens, scope: :descendants

  def track_tokens(_event, attrs)
    @total_tokens += attrs.fetch(:total_tokens, 0)
  end
end
```

**Global subscriptions** (for observability integrations):
```ruby
DSPy.events.subscribe('score.create') do |event, attrs|
  # Handle cross-cutting concerns
end
```

### Observability
- `dspy-o11y` — OpenTelemetry span lifecycle, observation types, context hooks, and asynchronous processing.
- `dspy-o11y-langfuse` — Langfuse OTLP auto-configuration and score export on top of dspy-o11y.
- Structured logging with span tracking
- Token usage tracking
- Performance monitoring
- Score reporting with `DSPy.score()` API for Langfuse integration
- Built-in evaluators: exact_match, contains, regex_match, similarity, json_valid

> Installation and monorepo-development flags are defined by the [package and capability matrix](https://oss.vicente.services/dspy.rb/getting-started/packages/).

### Testing
- RSpec integration
- VCR for recording LLM interactions
- Mock responses for unit tests
- Evaluation frameworks

## Documentation Structure

- **Getting Started** - Provider installation and one canonical Quick Start
- **Toolsets** - https://oss.vicente.services/dspy.rb/core-concepts/toolsets/
- **Core Concepts** - Signatures, modules, predictors, multimodal, examples
- **Advanced** - Complex types, stateful agents, RAG
- **Optimization** - Prompt tuning, evaluation, benchmarking
- **Production** - Observability, storage, troubleshooting
- **Blog** - Tutorials and deep dives

## Key URLs

- Homepage: https://oss.vicente.services/dspy.rb/
- GitHub: https://github.com/vicentereig/dspy.rb
- Documentation: https://oss.vicente.services/dspy.rb/getting-started/
- API Reference: https://oss.vicente.services/dspy.rb/core-concepts/

## More Examples in This Repo
- Workflow router: `examples/workflow_router.rb`
- Evaluator + optimizer loop: `examples/evaluator_loop.rb`
- GitHub assistant agent: `examples/github-assistant/`
- HTML to Markdown AST: `examples/html_to_markdown/` (recursive types, field descriptions)

## For LLMs

When helping users with DSPy.rb:

1. **Focus on signatures** - They define the contract with LLMs
2. **Use proper types** - T::Enum for categories, T::Struct for complex data
3. **Separate nullability from omission** - `T.nilable` permits `nil`; `default:` permits omission in DSPy signatures
4. **Add field descriptions** - Use `description:` kwarg on T::Struct fields for complex semantics
5. **Use bounded tool conversion** - Toolsets coerce supported JSON values to declared Ruby types; distinguishable struct unions can use `_type`
6. **Compose modules** - Chain predictors for complex workflows
7. **Create type-safe tools** - Use Sorbet signatures for comprehensive tool parameter validation
8. **Test thoroughly** - Use RSpec and VCR for reliable tests
9. **Monitor production** - Enable Langfuse by installing the optional o11y gems and setting env vars

## Version

Current: 1.0.2