Get started
← All comparisons

dif vs Optimizely

Checked Oct 7, 2026

This page compares dif with Optimizely Feature Experimentation, the Optimizely product for feature flags and A/B tests in application code. In Optimizely you define flags and rules in its app, the SDK polls a JSON datafile from Optimizely and decides in memory, and a saved change is typically in the datafile within two seconds. Results come from Stats Engine, which uses sequential testing so you can read them while a test runs. dif is a file format and a compiler: each experiment is a Markdown file in your repo, dif build compiles it into a TypeScript client, and a change ships with the next deploy. Optimizely is the better tool when you need statistics built in, SDKs beyond JavaScript, typed flag variables or a change without a deploy. dif fits a TypeScript team that wants each experiment reviewed as a pull request and its price on a public page.

Side by side

RowdifOptimizely
Where flags liveOne Markdown file per flag or experiment under dif/experiments/, in your git repo.In the Optimizely service. Feature Experimentation projects have a Flags tab in the Optimizely UI; flags, rules and audiences are also managed through the REST API and the Experimentation MCP server. [2][12]
How a change shipsEdit the file, merge the pull request, run dif build, deploy the app. There is no runtime toggle.Save the change in Optimizely. The docs say it is typically reflected in the datafile within two seconds. SDKs fetch the new datafile at the polling interval you set, or when a webhook fires. No deploy. [3]
Runtime dependencyNone. dif build compiles the active files into dif/generated/client.ts and @dif.sh/sdk hashes the user ID with SHA-256 in-process. No request to evaluate a flag.A JSON datafile per environment, fetched from Optimizely. The SDK polls for it and makes decisions in memory against its local copy, with no blocking request per decision. [3]
SDK languagesJavaScript and TypeScript only. Adapters for React, Svelte, the Vercel Flags SDK and an OpenFeature server provider.Android, C#, Flutter, Go, Java, JavaScript, PHP, Python, React, React Native, Ruby and Swift. Optimizely Agent is a microservice that exposes SDK functionality over REST endpoints. [4][9]
Experiment analysisNot in the open-source core. Exposures and dif.track() calls go to two handler functions you own, or to Dif Cloud, which reports lift with a 95% confidence interval.Built in. The Experiment Results page shows improvement, a confidence interval and statistical significance, computed by Stats Engine with sequential testing and false discovery rate control. Fixed-horizon frequentist and Bayesian methods are in beta. [7][8]
Pricing modelThe CLI and SDK are free. Dif Cloud is optional: $50 per month per workspace with 1,000,000 events and unlimited seats, then $0.02 per 1,000 events. Cloud has no free tier.Not published. The plans page lists no prices and no free plan. It says every plan is individually packaged. The Feature Experimentation docs describe Optimizely Rollouts as a free plan for startups and teams getting started. [1][2]
LicenceMIT for the CLI, the SDK and every adapter.Proprietary service. The SDKs and Optimizely Agent are open source under Apache-2.0. [11][10]
Self-hostingNothing to host. There is no flag server; the flags run from your repo and your build. Dif Cloud is a hosted product.Hosted by Optimizely. You can run Optimizely Agent yourself with Docker, Helm or from source, but each request carries an Optimizely SDK key and Agent reads its datafiles from Optimizely by default. [9][10]
Agent and context supportdif init writes guidance into CLAUDE.md, AGENTS.md, .cursor/rules/dif.mdc and .github/copilot-instructions.md and installs four skills. dif build writes dif/context.json, a summary of active experiments. Dif Cloud serves experiment verdicts to agents over a read-only MCP server at cloud.dif.sh/mcp.A hosted Experimentation MCP server lets an agent query projects, flags and results, create, update and archive flags, experiments and audiences, and search the SDK docs. It needs an Opti ID and an account with Opal enabled. [12]
Flag valuesString variant ids only. No typed values, no JSON payloads, no remote config.Flag variables of type Boolean, String, Integer, Double and JSON, with values set per variation. [5]
EnvironmentsNone. The schema has no environment field.The datafile holds the flag rules for one environment, and each SDK key maps to one project and environment. [3][10]
Statistics depthNone in the open-source core. Dif Cloud reports lift with a 95% confidence interval.Stats Engine uses sequential testing, so results can be monitored continuously while a test runs. Fixed-horizon frequentist and Bayesian methods are in beta. [8]
Overlapping experimentsChecked at build time. Two active experiments on one surface without a shared exclusion_group fail dif validate with E007. Within a group the earliest created experiment wins; it is a priority order.Exclusion groups keep a user in one experiment of a group, with no SDK change. The docs say not to add or remove running experiments from a group. [6]

The dif column is checked against the dif repository at v0.7.0. The Optimizely column comes from the pages listed under Sources. If a row is out of date, open an issue.

Which one to choose

Choose Optimizely if

  • You want the statistics from the vendor that assigns the variants. Stats Engine reports significance and confidence intervals from sequential tests. The dif open-source core computes no statistics.
  • A flag or a traffic split has to change without a deploy. Optimizely publishes a new datafile and the SDK polls for it. In dif that change is a merge, a dif build and a deploy.
  • Your services are written in Go, Java, C#, PHP, Python or Ruby, or you ship Android, iOS, Flutter or React Native apps. dif has a JavaScript and TypeScript SDK and nothing else.
  • A variation has to carry typed values. Optimizely flag variables can be Boolean, String, Integer, Double or JSON. dif variants are string ids.
  • People who do not open pull requests create and run tests. Optimizely has an app for that. dif has files.
  • You need separate rules per environment from one flag. dif has no environment field.

Choose dif if

  • Your app is TypeScript and a change to an experiment can wait for the next deploy.
  • You want to know the price before a sales call. The dif CLI and SDK are free under MIT, and Dif Cloud is $50 per month for 1,000,000 events with unlimited seats, with no free tier.
  • You want each experiment reviewed in the same pull request as the code it gates, with the hypothesis, owner, primary metric and guardrails in the frontmatter.
  • You want variant assignment with no datafile to fetch and no SDK key. dif build writes dif/generated/client.ts and @dif.sh/sdk hashes the user ID in-process.
  • You want CI to fail when two active experiments target one surface without a shared exclusion_group. dif validate exits 1 with error E007.
  • You want the record to stay in the repo after a test ends. dif conclude moves the file to dif/experiments/concluded/, writes the Decision block and appends one line to the surface, and the next dif new on that surface reads it.
  • You already send events to your own analytics. dif init --events custom scaffolds dif/events/exposure.ts and dif/events/track.ts, two handlers you own.

Moving from Optimizely to dif

dif has no Optimizely importer. Migration is by hand, one flag at a time, and both can run side by side while you move. A test that is still collecting data should finish in Optimizely: dif buckets with its own SHA-256 hash, so users would be reassigned.

  1. List the flags your code still decides on. Flags that need a no-deploy change, typed variables or Stats Engine results should stay in Optimizely.
  2. For each flag you move, run dif new <id> --surface <name>. The file lands in dif/experiments/active/. Copy each variation into variants and its traffic percentage into weight. Weights must sum to 100. Fill in hypothesis and metrics.primary, then set status: active.
  3. Rewrite audiences as audience.include and audience.exclude on attributes declared in dif/config.yaml, with a resolver at dif/audiences/<name>.ts for each. dif matches by equality or list membership only; anything else has to be computed in the resolver and declared as a boolean or enum attribute.
  4. Move flag variable values into code. dif has no variables; each variant id maps to one branch in dif("<id>", { ... }).
  5. Put experiments from one Optimizely exclusion group on the same surface with the same exclusion_group. Within a group dif picks by created date, it does not split traffic between the experiments.
  6. Replace decide() calls with dif(). Replace Optimizely event tracking with dif.track(), and send exposures and events to your own handlers or to Dif Cloud. The open-source core computes no results.
  7. Run dif validate, then dif build. Add "prebuild": "dif build" to package.json, because dif/generated/ is gitignored. Remove the SDK key and the datafile polling once the last flag has moved.
$ npm install -g @dif.sh/cli

Sources

Every Optimizely claim on this page comes from one of these pages. Pricing and features change; the date is when we read each one.

  1. Optimizely plans page read Oct 7, 2026
  2. Optimizely docs: Feature Experimentation introduction read Oct 7, 2026
  3. Optimizely docs: Get the datafile read Oct 7, 2026
  4. Optimizely docs: SDK reference guides read Oct 7, 2026
  5. Optimizely docs: Create flag variables read Oct 7, 2026
  6. Optimizely docs: Use mutual exclusion read Oct 7, 2026
  7. Optimizely docs: Experiment Results page read Oct 7, 2026
  8. Optimizely docs: Statistical analysis methods overview read Oct 7, 2026
  9. Optimizely docs: Optimizely Agent read Oct 7, 2026
  10. optimizely/agent README read Oct 7, 2026
  11. optimizely/javascript-sdk README read Oct 7, 2026
  12. Optimizely docs: Experimentation MCP server overview read Oct 7, 2026