Frequently asked
Questions answered. No fluff.
Everything you need to know about getting started, how testing works under the hood, and how we price. Can't find what you're looking for? Reach out — we read every message.
Getting started
2 questionsDoes Split Test Pro work outside of Shopify?
Yes. Split Test Pro works on any website — install on Shopify in two clicks via our Theme & Web Pixel extensions, or drop a single script tag on any other site (works with React, Vue, plain HTML, GTM, and most site builders). Same dashboard, same features, same stats engine.How do I install Split Test Pro?
Sign up free at app.splittestpro.com/register, then choose your platform: install our Shopify app from the Shopify App Store, or copy the script tag for your site from the setup wizard. Most setups take under five minutes.
How testing works
6 questionsWhat kinds of changes can I test?
Four test types: CSS and JavaScript for styling tweaks, content rewrites and DOM changes, redirect variants for A/B-ing two completely different URLs, and on Shopify theme tests (an unpublished theme against your live one) and template tests (an alternate product, collection, page or blog post template). A variant can combine them.Can I test server-side, without the pixel?
Yes, on every paid plan. Create feature flags and a server API key in Settings, set an experiment to run Server-side, and give each variant its flag values. Your backend evaluates the flags over a REST API or with the Node SDK (@splittestpro/node) and reports exposures and conversions, which feed the same Bayesian results. Pixel tests can share their variant with your server too — a hybrid test.What does the AI actually do?
Three things. (1) Before launch: Create with AI opens Splitty, which builds the test on your own page, and Improve with AI refines each step of the wizard. (2) During the test: continuously watches for sample-ratio mismatches, traffic drops, novelty effects, and other anomalies. (3) After the test: Generate analysis writes a plain-English summary with a clear recommendation and an estimated revenue impact.How long should I run a test before declaring a winner?
Until your winning variant reaches at least 95% probability-to-be-best with a stable credible interval. A winner is called only once every variant clears the sample floor, so a lucky first day can't decide it.Can I track revenue, not just conversions?
Yes — Split Test Pro supports continuous metrics like revenue per session, average order value, and engagement value. We model them with full distributions (Normal for continuous, Beta-Binomial for binary) so you get accurate probability and credible-interval estimates, not just averages.What metrics does Split Test Pro track?
Sessions, conversions, conversion rate, probability-to-be-best, credible intervals, and any custom revenue or engagement metric you define. Results stream in real time and can be sliced by device type and other segments.
Pricing & billing
3 questionsHow is pricing calculated?
Pricing is based on monthly sessions in active experiments — a unique visitor entering a running test, counted once per 24-hour window. Visitors who never hit an experiment don't count. Test Plans are included in Pro and Agency, and Core includes 10 AI generations a month.Can I cancel at any time?
Yes. Monthly billing, no long-term contracts. Direct customers can cancel from their account settings; Shopify customers can press Stop subscription under Settings → Billing, or manage billing from the Shopify admin. Switch tiers any time as your traffic changes.Is there a free trial?
Yes — every direct plan starts with a 14-day free trial before billing begins.