Polarity
SDK Reference

Ruby SDK

Every Polarity Ruby class, method, and option — what each does, when to use it, what it returns.

polarity-keystone — the Ruby SDK. Stdlib-only at runtime (Net::HTTP, JSON, SecureRandom, OpenSSL); no transitive gem dependencies. Compatible with Ruby 3.0+.

Install

gem install polarity-keystone
# or, in a Gemfile
gem "polarity-keystone"
require "polarity_keystone"

Polarity::Client — the client

ks = Polarity::Client.new(
  api_key:    "plr_live_...",                  # optional — falls back to POLARITY_API_KEY
  base_url:   "https://api.plr.sh", # default
  timeout_ms: 30_000                          # 30 seconds, in milliseconds
)

After construction, the ten services hang off the client: ks.sandboxes, ks.specs, ks.experiments, ks.alerts, ks.agents, ks.datasets, ks.scoring, ks.exports, ks.env_secrets, ks.prompts.

plr.wrap(client, sandbox_id: nil, tracing: true) — wrap an LLM client

require "anthropic"
ks = Polarity::Client.new
 
anthropic = plr.wrap(Anthropic::Client.new(api_key: ENV["ANTHROPIC_API_KEY"]))
# Every anthropic.messages.create now auto-reports.

Two actions, controllable via kwargs:

KwargDefaultEffect
tracingtrueInitialize traced { ... } block reporting on the current thread
sandbox_idnilFalls back to POLARITY_SANDBOX_ID env var; if absent, agent mode (events scoped by API key)

Detection by class name (case-insensitive substring): any class matching /openai/i is wrapped at client.chat; any class matching /anthropic/i is wrapped at client.messages.create. Wrapping is idempotent — calling wrap twice on the same client doesn't double-emit. Returns the same client instance for ergonomics.

# Wrap-only — skip global tracing
plr.wrap(OpenAI::Client.new(...), tracing: false)
 
# Pin sandbox id explicitly
plr.wrap(OpenAI::Client.new(...), sandbox_id: "sb-explicit")

plr.init_tracing(sandbox_id: nil) — set up traced { } only

ks.init_tracing                          # picks up POLARITY_SANDBOX_ID
plr.init_tracing(sandbox_id: "sb-explicit")

No-op without sandbox id and API key. Use when you don't have an LLM client to wrap.

ks.record_llm_call(opts) — gateway / proxy escape hatch

For multi-provider gateways and raw Net::HTTP flows that have no client object to wrap. Same on-the-wire event shape as wrap(), fire-and-forget, never raises.

start = Time.now.to_f
res = Net::HTTP.post(URI(upstream_url), payload.to_json, "Content-Type" => "application/json")
body = JSON.parse(res.body)
 
ks.record_llm_call(
  provider:        "openrouter",
  model:           body["model"],
  requested_model: req[:model],                   # what the caller asked for
  input_tokens:    body.dig("usage", "prompt_tokens"),
  output_tokens:   body.dig("usage", "completion_tokens"),
  duration_ms:     ((Time.now.to_f - start) * 1000).to_i,
  input_messages:  req[:messages],                # truncated to ~4KB on the wire
  output_text:     body.dig("choices", 0, "message", "content"),
  metadata: { "gen_ai.proxy.fell_back" => false }
)

Polarity::Client.from_sandbox — inside the sandbox

ks, sb = Polarity::Client.from_sandbox
db = sb.services["db"]   # { "host" => "db", "port" => 5432, "ready" => true }

Reads POLARITY_SANDBOX_ID, POLARITY_API_KEY, POLARITY_BASE_URL from the env. Raises Polarity::PolarityError if POLARITY_SANDBOX_ID isn't set.

SandboxService

plr.sandboxes.create(spec_id:, timeout: nil, metadata: nil, secrets: nil)

sb = plr.sandboxes.create(
  spec_id:  "fix-failing-test",
  timeout:  "10m",
  metadata: { "owner" => "alex" },
  secrets:  { "ANTHROPIC_API_KEY" => ENV.fetch("ANTHROPIC_API_KEY") }
)
sb.id            # → "sb_abc123"
sb.state         # → "creating" | "ready" | "running" | "stopped" | "error"
sb.services      # → { "db" => { "host" => ..., "port" => 5432, "ready" => true } }

Returns a Polarity::Sandbox handle bound to the new sandbox id. Methods on the handle (exec, read, write, destroy, etc.) all act on that id.

plr.sandboxes.get(id) / plr.sandboxes.handle(id)

sb = plr.sandboxes.get("sb_abc123")        # network call
sb = plr.sandboxes.handle("sb_abc123")     # no network — for queue payloads

plr.sandboxes.list — every active sandbox

Sandbox handle methods

sb.exec("ls /workspace")                  # → { "stdout" => ..., "exit_code" => 0, ... }
sb.exec("npm test", timeout: "5m")
sb.read("server.js")                      # → file content as a string
sb.write("server.js", "const app = ...")
sb.delete("server.js")
sb.state_snapshot                         # filesystem snapshot
sb.diff                                   # diff vs baseline
sb.ingest_trace([{ event_type: "tool_use", tool: "ls" }])
sb.get_trace                              # events + computed metrics
sb.refresh                                # re-fetch from server
sb.destroy
sb.to_h                                   # raw server payload

SpecService

plr.specs.upload_yaml(File.read("spec.yaml"))   # → { "id" => "fix-failing-test", ... }
plr.specs.get("fix-failing-test")
plr.specs.list                                  # → [{ ... }, ...]
plr.specs.destroy("fix-failing-test")

Specs are versioned automatically — uploading a spec with the same id: field creates a new version.

ExperimentService

exp = plr.experiments.create(
  name:    "baseline-v1",
  spec_id: "fix-failing-test",
  secrets: { "ANTHROPIC_API_KEY" => ENV.fetch("ANTHROPIC_API_KEY") }   # forwarded to every spawned sandbox
)
 
plr.experiments.run(exp["id"])               # 202 — async kickoff, returns nil
plr.experiments.results(exp["id"])           # latest results snapshot
plr.experiments.metrics(exp["id"])           # computed metrics summary + trends
plr.experiments.compare(baseline_id: "exp_a", candidate_id: "exp_b")
plr.experiments.history(exp["id"], limit: 20)
plr.experiments.destroy(exp["id"])

The Ruby gem doesn't ship a run_and_wait helper yet — poll explicitly:

plr.experiments.run(exp["id"])
deadline = Time.now + 300
loop do
  results = plr.experiments.results(exp["id"])
  break results if results && (results["passed"].to_i + results["failed"].to_i) > 0
  raise "experiment timed out" if Time.now > deadline
  sleep 2
end

AgentService

plr.agents.list(limit: 50)
plr.agents.get(snapshot_id)
plr.agents.upload(
  name:       "my-agent",
  entrypoint: ["ruby", "agent.rb"],
  bundle:     File.binread("agent.tar.gz"),    # binary string of a tarball
  runtime:    "ruby:3.3",
  auth:       { "required_env" => ["ANTHROPIC_API_KEY"] }
)
plr.agents.destroy(snapshot_id)

The bundle argument is a binary string — typically the output of Gem::Package::TarWriter + Zlib::GzipWriter over your project directory.

DatasetService

plr.datasets.list
plr.datasets.create(name: "qa-pairs", description: "Q→A regression set",
                   records: [{ input: "2+2", expected: "4" }])
plr.datasets.append_records(dataset_id, [{ input: "...", expected: "..." }])
plr.datasets.list_records(dataset_id, cursor: nil, limit: 100)
plr.datasets.from_traces(name: "captures-may", since: "2026-05-01")
plr.datasets.destroy(dataset_id)

AlertService

plr.alerts.create(
  name:      "low pass rate",
  condition: "pass_rate < 0.9 over 1h",
  notify:    "slack",
  slack_channel: "#keystone-alerts"
)
plr.alerts.list
plr.alerts.update(alert_id, { condition: "pass_rate < 0.85 over 1h" })
plr.alerts.destroy(alert_id)
plr.alerts.firings(rule_id: alert_id, limit: 50)

PromptService

plr.prompts.create(slug: "system_prompt", content: "You are a helpful assistant. {greeting}")
plr.prompts.get_active("system_prompt")
plr.prompts.activate(prompt_id, version: 3)

ExportService

job = ks.exports.create(kind: "traces", format: "jsonl",
                        filters: { sandbox_id: "sb_abc", since: "2026-05-01" })
ks.exports.get_job(job["id"])           # poll for status
ks.exports.download(job["id"])          # full body string (use the signed URL for huge exports)

EnvSecretService

ks.env_secrets.list                              # names + metadata only (values are write-only)
ks.env_secrets.upsert(name: "DATABASE_URL", value: "postgres://...")
ks.env_secrets.destroy("DATABASE_URL")

Tracing — Polarity.traced(name, &block)

ks.init_tracing            # picks up POLARITY_SANDBOX_ID; no-op without it
 
result = Polarity.traced("write_file") do
  File.write(path, content)
  :ok
end
# → emits start + end events. Parent-span id propagates through nested calls
# via Thread.current[Polarity::Tracing::THREAD_KEY].

Auto-captures input, output, duration, errors. Nesting works automatically — every traced block becomes a child span of the closest enclosing one. Trace posts run in a background Thread, so they never block the wrapped work.

For manual lifecycle, use Polarity::Tracing::Span:

span = Polarity::Tracing::Span.new("custom_step")
begin
  result = do_work
  span.set_output(result)
rescue => e
  span.fail(e)
  raise
ensure
  span.finish
end

Pricing — Polarity::Pricing.estimate_cost

Polarity::Pricing.estimate_cost("claude-sonnet-4-6", 1000, 500, 100)
# → 0.01023  (USD, 6-decimal rounded)
 
Polarity::Pricing.table.size    # → 118 models

The pricing table is auto-generated from apps/plr/sdks/pricing.json (the canonical SSOT shared across all four SDKs). The server always recomputes cost at trace ingest — this client-side helper is for synchronous budget gating only.

Scorers

require "polarity_keystone"
 
# Heuristics — five modes
H = Polarity::Scorers::Heuristics
H.new(mode: :exact,           expected: "ok").score(nil, "ok")
H.new(mode: :iexact,          expected: "OK").score(nil, "ok")
H.new(mode: :contains,        expected: "world").score(nil, "hello world")
H.new(mode: :regex,           expected: /\Ahi/).score(nil, "hi there")
H.new(mode: :numeric_within,  expected: 100, tolerance: 5).score(nil, "98")
 
# Braintrust-parity
Polarity::Scorers::ExactMatch.new.score(nil, "  hi  ", "hi")  # default trims + case-sensitive
Polarity::Scorers::Contains.new(text: "world").score(nil, "hello world")
 
# LLM-as-judge — calls the configured Polarity judge endpoint
Polarity::Scorers::LLMJudge.new(
  client:    ks,
  prompt:    "Is the response polite and on-topic?",
  threshold: 0.8,
  gate:      true
).score("say hi", "hello!")

Custom scorers subclass Polarity::Scorers::Base:

class NoTodos < Polarity::Scorers::Base
  def initialize; super(name: "no_todos"); end
  def score(_input, output, _expected = nil)
    ok = !output.to_s.include?("TODO")
    Polarity::Score.new(name: @name, score: ok ? 1.0 : 0.0, passed: ok)
  end
end

Polarity::Eval — Braintrust-parity ergonomic primitive

result = Polarity.eval(
  "math",
  data: [
    { input: "2+2", expected: "4" },
    { input: "10/2", expected: "5" }
  ],
  task:   ->(q) { my_agent.solve(q) },
  scores: [Polarity::Scorers::ExactMatch.new],
  max_concurrency: 4
)
result[:summary]["exact_match"]
# → { mean: 1.0, p50: 1.0, p95: 1.0, count: 2 }

Mirrors the TS / Python / Go Eval() byte-for-byte: same percentile math (linear interpolation between ranks), same aggregate keys, same scorer contract. A row that raises is captured as error: "Class: message" and excluded from the per-scorer aggregate without breaking the run.

Prompt rendering — Polarity.render_template

Polarity.render_template(
  "Hello {name}, your hobbies are:\n{#hobbies}- {_it}\n{/hobbies}",
  { name: "alex", hobbies: %w[climb type] }
)
# → "Hello alex, your hobbies are:\n- climb\n- type\n"

Mustache-lite: {name} / {{name}} interpolation, {a.b} dotted paths, {#xs}body{/xs} sections (skipped if falsy, iterated if array with _it and merged hash keys exposed inside body). Cross-SDK parity-checked against TS / Python / Go.

RSpec matcher — pass_keystone_eval

require "polarity_keystone/rspec"
 
RSpec.describe MyAgent do
  H = Polarity::Scorers::Heuristics
 
  it "answers in markdown" do
    output = MyAgent.new.call("explain ruby blocks")
    expect(output).to pass_keystone_eval(scorers: [
      H.new(mode: :contains, expected: "```ruby", gate: true),
      H.new(mode: :regex,    expected: /\A#/,     gate: true)
    ])
  end
end

Each scorer runs against (input, output, expected). The matcher passes when every gating scorer passes; non-gating scorers contribute to the failure message but don't fail the test.

Errors — Polarity::PolarityError

begin
  plr.experiments.results("not-a-real-id")
rescue Polarity::PolarityError => e
  warn "status #{e.status_code}: #{e.message}"
end

status_code is the HTTP status (or 0 for client-side timeouts / network errors). Message comes from the server's {"error": "..."} body when present, otherwise the HTTP status text.

Environment variables

VariablePurpose
POLARITY_API_KEYBearer token. Falls back when Client.new(api_key:) is omitted.
POLARITY_BASE_URLAPI base URL. Defaults to https://api.plr.sh.
POLARITY_SANDBOX_IDAuto-injected inside sandboxes; routes traces to that sandbox.

What's not in the gem yet (vs Python / TS / Go)

The Ruby SDK is at v0.1.0 — first release. These pieces from the other SDKs land in v0.2:

  • Spec-file secret auto-forwarding. Other SDKs read secrets: blocks from a spec YAML and resolve source: env / file: / command: declarations. In Ruby today, build the secrets hash explicitly with ENV.fetch and pass it to experiments.create / sandboxes.create.
  • run_and_wait helper. Use the explicit poll loop above.
  • Auto-instrumenting framework hooks (LangChain.rb, ruby-llm). For now, wrap the LLM client directly via ks.wrap.
  • RAG / sandbox / embedding scorers. Heuristic + LLM-judge scorers ship in v0.1; the full 28-scorer set lands in v0.2.

Everything in the v0.1 surface is parity-checked against TS / Python / Go via apps/plr/sdks/scripts/parity-check.mjs — pricing math, prompt rendering, and Eval() aggregate summary all match byte-for-byte across all four SDKs on identical inputs.

GoREST API