Methodology: How Statculator Computes Results
The numerical methods, conventions and validation behind Statculator’s calculators, including how results are checked against SciPy and statsmodels.
Updated
In short
Statculator computes every result in your browser with its own JavaScript statistics engine. Probabilities come from standard special-function algorithms, including the Lanczos approximation to the log-gamma function and continued fractions for the incomplete beta and gamma functions. Quantiles use a refined inverse-normal approximation or bisection, logistic regression uses IRLS, and counting uses exact BigInt arithmetic. An automated test suite checks the engine against more than 200 SciPy and statsmodels reference values.
Numerical methods
All calculations run locally in JavaScript using 64-bit (double-precision) floating-point arithmetic, which carries about 15 to 16 significant digits. Every probability distribution on the site is built from a small set of special functions:
- Log-gamma, ln Γ(x): the Lanczos approximation, with the reflection formula for arguments below 0.5. It supplies the normalizing constants of the t, χ², F, beta and discrete distributions and the log-factorials used for binomial coefficients.
- Regularized incomplete beta function, Iₓ(a, b): a continued fraction evaluated with the modified Lentz method, switching to the symmetric form 1 − I₁₋ₓ(b, a) where that converges faster. It gives the cumulative distribution functions of the t, F, binomial and negative binomial distributions.
- Regularized incomplete gamma functions, P(a, x) and Q(a, x): a power series when x < a + 1 and a continued fraction otherwise. They give the χ² and Poisson distribution functions and, through the complementary error function, the normal distribution.
- Upper tails computed directly: right-tail probabilities are evaluated from the survival function rather than as 1 − CDF, so very small p-values keep their precision instead of being lost to rounding.
Quantiles and critical values
The inverse normal function Φ⁻¹(p), which drives z critical values, percentiles and the inverse normal calculators, starts from Acklam’s rational approximation and is then refined with Halley’s method against the high-precision normal CDF.
Quantiles of the other distributions, including t, χ² and F critical values and the exact Clopper–Pearson limits for a proportion, are found by root finding: the engine brackets the answer and then bisects the interval until it is as narrow as double precision allows. For upper-tail probabilities the search runs on the survival function, so critical values for very small α stay accurate.
Tests, regression and exact counting
Hypothesis tests and confidence intervals use their standard closed-form statistics (z, t, χ² and F), with p-values and critical values from the functions above. Fisher’s exact test sums hypergeometric probabilities over every 2×2 table that has the observed row and column totals.
Linear, multiple and quadratic regression use ordinary least squares, solving the normal equations by Gauss–Jordan elimination with partial pivoting after scaling each predictor column to keep the arithmetic well conditioned. Exponential regression is fitted by least squares on ln y. Logistic regression is fitted by maximum likelihood using iteratively reweighted least squares (IRLS, a Newton–Raphson method) with step-halving, and its standard errors come from the inverse of the information matrix at the solution.
Factorials, permutations and combinations are computed with JavaScript BigInt integers, so results such as 100! are exact to the last digit. Very large values are also shown in scientific notation for readability.
Validation against SciPy and statsmodels
An automated test suite compares the engine with more than 200 reference values generated with SciPy and statsmodels. The references cover the special functions; the probability, cumulative, survival and quantile functions of the normal, t, χ², F and discrete distributions; descriptive statistics such as skewness, kurtosis and percentiles; the t, χ², ANOVA, Fisher and correlation tests; least-squares and logistic regression; and the Wald, Wilson and exact proportion intervals.
Each reference value must be matched within a stated tolerance, from 10⁻¹² (for example, the log-gamma function) to 10⁻⁶ (logistic regression estimates), measured relative to the value when it exceeds 1 and absolutely otherwise. The suite also checks textbook values, such as the two-tailed z critical value 1.959964 at α = 0.05 and C(52, 5) = 2,598,960.
Rounding and display
Calculations carry full double precision throughout; rounding happens only when a number is displayed. You can choose 2, 3, 4, 5, 6, 8 or 10 decimal places (4 by default), and trailing zeros are trimmed, so 2.5000 is shown as 2.5.
A p-value smaller than the chosen precision can show, which at four decimals means below 0.0001, is displayed in scientific notation, for example 3.21 × 10⁻⁸, instead of being rounded to 0. Other very small or very large values are also shown in scientific notation. Intermediate values in the step-by-step working are rounded for display, so recomputing by hand from those rounded figures can differ from the final result in the last digit.
Conventions
Where textbooks and software packages disagree, Statculator uses these defaults and says so on the relevant pages:
- Standard deviation and variance: sample formulas divide by n − 1. Population formulas, which divide by n, are reported alongside where they apply.
- Quartiles: the default is the median of the lower and upper halves, excluding the median when n is odd (the TI-84 and most introductory textbooks). The Quartile Calculator also offers Tukey’s hinges (halves including the median) and linear interpolation at position (n − 1)p + 1 (Excel QUARTILE.INC, the R default) or (n + 1)p (Excel QUARTILE.EXC, Minitab).
- Test decisions: H₀ is rejected when p ≤ α.
- Fisher’s exact test: the two-sided p-value is the sum of the probabilities of all tables with the observed margins whose probability is less than or equal to that of the observed table.
- One proportion: the Wilson score interval is the default; Wald and exact Clopper–Pearson intervals are also available.
Limitations
Floating-point arithmetic has finite precision. Results are normally accurate to more digits than are displayed, but accuracy can degrade for extreme inputs: very large degrees of freedom or counts, probabilities extremely close to 0 or 1, and statistics far out in a distribution’s tail. P-values below about 10⁻³⁰⁰ fall outside the range of double precision and are reported as an upper bound rather than an exact value.
Iterative methods can struggle in edge cases. In logistic regression, for example, if a predictor separates the 0 and 1 outcomes perfectly, maximum likelihood estimates do not exist, and the reported coefficients and standard errors become very large and should not be interpreted.
A correct calculation does not guarantee a correct conclusion. Every method assumes something about the data, such as independence, approximate normality or large enough expected counts, and checking those assumptions remains the user’s responsibility; see the Disclaimer.
Reporting an error
If you believe a result is wrong, please report it through the contact page or by email to hello@statculator.com. Include the calculator’s name or URL, the exact inputs, the result Statculator gave, the result you expected, and where the expected value came from (software and version, textbook or hand calculation). Those details are usually enough to reproduce and check the problem.