Absolute error: The absolute value of the difference between a predicted or modeled value and the actual value.
AIC: Akaike Information Criterion. A measure used for model comparison which is calculated from the log-likelihood of the data given the model of interest and from the number of degrees of freedom used by the model.
Alpha: 1) The Type I error rate. 2) A generic symbol for a parameter, commonly used for the intercept parameter in regression.
Alternative hypothesis: Typically the complement of the null hypothesis. Also called the research hypothesis.
ANCOVA: Analysis of Covariance. A modification to ANOVA to include multiplicative interactions between explanatory variables.
ANOVA: Analysis of Variance. A method of decomposing the sum of squared errors into those attributable to a group or regression coefficient and those attributable to the residuals. Used to analyze the 'sources' of variation, and to test if there exist any differences in means between three or more groups. Is expanded into MANOVA (Multiple ANOVA), and ANCOVA (Analysis of Covariance). Often accompanied by a Post-Hoc analysis like Tukey's test. See F-distribution.
Arithmetic mean: See "mean". There are other measures for the mean, such as the "geometric mean" which are used in specific cases.
Autocorrelation: Also called Auto-correlation. Literally 'correlation with self'. Typically used in the context of a single dimension, usually time, but technically refers to all types of autocorrelation. Special cases include "spatial autocorrelation". See also "correlation".
Average: Colloquial term for "mean". Can also refer to any measure of centrality, such as the "median", or "trimmed mean", but much less often.
Bayesian: Bayesian analyses use information in addition to the observations from a given study, such as expert opinions or observations from a previous study. Antonym to "frequentist" methods. "Metastudies" are Bayesian in nature.
Bell curve: Describes the shape of the probability density of a normal / Gaussian distribution about the mean. See "normal distribution".
Beta: 1) The type II error rate. 2) A generic symbol for a parameter, commonly used with slope parameters in regression. 3) A mathematical function that acts as a continuous analogue to the combinatorical 'choose' function. 4) A probability distribution that uses the beta function.
Correlation: Refers to the method of relating two numeric variables together (e.g. 'We ran a correlation on family income and reading speed'), the relation itself (e.g. 'There is a positive correlation between family income and reading speed'), and the most common measure, the "Pearson correlation coefficient", of strength of such a relation (e.g. 'In our sample, the correlation between family income and reading speed was 0.29') . A correlation relationship is a specific type of "association".
Degrees of freedom: Called "df" for short. Degrees of freedom is a "parameter" which means different things for different "probability distribution"s. For example, in the "Student's T distribution" and the "AnoVa F distribution", it refers to the amount of information known about the "standard deviation"(s).
Dependant variable: A depreciated term for "response variable". Sometimes used to emphasize that a relationship between variables is only associative and not causative.
Distribution: Short for "frequency distribution" in the context of real observations, and short for "probability distribution" in the context of theory. Statements without this context, such as in "right skew", typically apply to both definitions.
Euler's Gamma: A mathematical function that acts as a continuous analogue to the factorial function. Used in the beta function (definition 3).
Expected value: See "mean".
Explanatory variable: Also called an "independent variable" or "x variable". This is (one of) the variable(s) that is being used to predict the value of a "response variable" in a model.
Frequency distribution: Also called "distribution". Refers specifically to the values that are observed instead of a theoretical "probability distribution".
Frequentist: Frequentist analyses only consider observations from the study in question. Most methods based on a single study are frequentist in nature.
Gaussian distribution: See "normal distribution".
Gosset distribution: See "Student's T distribution". The 'Student' in the T distribution refers to William Gosset's pseudonym of 'Student', which he published his discovery of the T distribution under when he worked at the Guiness brewery and it was being used as a trade secret for quality control. It does not refer to the T distribution for Students to use.
Imputation: A generic term for methods to replace missing data by using appropriate observed data.
Independent variable: A depreciated term for "explanatory variable". Discouraged because of the potential for confusion surrounding the term 'independent'.
Kriging: Also called "spatial autocorrelation". Correlation between responses that is dependent upon the spatial relationship between observations. Connotation implies mining or geological context.
Least squares regression: See "linear model".
Left skew: Antonym to "right skew" and "positive skew".
Likert scale: An "ordinal" scale of something qualitative, coded into numeric values, typically 1-5 or 1-7. (e.g. A scale from poor to excellent. A scale from strongly disagree to strongly agree.).
Linear model: Also called a "regression" or a "least squares regression". Provides a predictive trend of a "response" given set values from one or more "explanatory" variables. Has many expansions and generalizations, including "multiple regression" and "logistic regression".
Markov Chain: A finite state system in which the probability of reaching the next stage only depends upon the current state. This is a special case of a stochastic system.
MCMC: Markov Chain Monte Carlo. A method used to simulate complex systems. See Monte Carlo, and Markov Chain.
Mean: Also called the "average", "arithmetic mean", and "expected value". Note that there are other measures of the centre of a distribution that are not equivalent to the mean, such as the "median", and "trimmed mean".
Median: Also called the second "quartile" and 50th "percentile". Not affected by extreme values at all, unlike the "mean".
Monte Carlo: A method of estimating a probability distribution, or any measure (e.g. expected value) based on such a distribution by generating many values from that distribution. Typically done by simulation. The name comes from someone estimating the probabilities of different dice outcomes by literally rolling combinations of dice many times in the Monte Carlo Casino in Monaco.
MSE: Root Mean Square Error. The square root of the mean square error. Used to evaluate models, and a possible optimization criterion for linear regression. See RMSE, residuals.
Negative skew: Antonym/opposite to "right skew" and "positive skew".
Non-Parametric: Methods that require only loose assumptions about the "probability distribution" that observations follow. Compared to "parametric" methods, "non-parametric" ones can handle data that is farther from ideal, but produce results that may be harder to interpret.
Normal distribution: Also called the "bell curve" or the "normal distribution". A "probability distribution" that describes many phenomena, especially those that can be described as the sum of many distinct factors, such as reading skill, height, and exam grade. The normal distribution is an idealization, and the "Student's T distribution" is frequently used in its place.
Parameter: A value that describes some feature of a "probability distribution", such as the "mean" and "standard deviation" of a "normal distribution", or the "degrees of freedom" on a "Student's T distribution".
Parametric: Methods that center around assuming observations follow a type of "probability distribution", such as the "normal distribution", "chi-squared distribution", or "Student's T distribution". Most default analyses are parametric. See also "Non-Parametric".
Pearson correlation coefficient: See also "correlation". The most commonly used measure of measuring the strength of a correlation relationship. Has strict requirements for proper use relative to the "Spearman correlation coefficient", which is a "Non-Parametric" measure.
Positive skew: Also called "right skew". This term is more often used when describing a distibution by using summary statistics such as the mean and median (i.e. "The mean is greater than the median because the higher, more positive values are the extreme ones"). Antonym to "left skew" and "negative skew".
Probability distribution: Also called "distribution". Refers specifically to a theoretical model to explain the values of observations.
Regression: See "linear model"
Research hypothesis: The hypothesis that one is trying to find evidence for.
Residuals: Errors of estimation. The difference between a modeled value and the actual value.
Response variable: Also called the "dependent variable" or "y variable". This is the variable that is being predicted in a model, such as a "linear model". Also see "explanatory variable".
Right skew: Also called "positive skew". This term is more often used when describing a distribution by using a "histogram" graph. (e.g. "The extreme values appear more on the right of the graph"). Antonym to "left skew" and "negative skew".
RMSE: Root Mean Square Error. The square root of the mean square error. Used to evaluate models, and a possible optimization criterion for linear regression. Often used in the place of MSE because the measure is of the same scale as the errors themselves (e.g. if the errors were twice as large, the RMSE would be twice as large too). See MSE, residuals.
Skew: Also called "skewness". Measure of the amount and direction of the asymmetry in a distribution. See also "right skew", "left skew", "positive skew", and "negative skew".
Spatial autocorrelation: Also called "kriging". Connotation is general. Correlation between responses that is dependent upon the spatial relationship between observations. Typically used for auto-correlation over a space of two-dimensions, but could be used to higher dimensions.
Spearman correlation coefficient: See also "correlation". A more flexible but less popular measure of correlation relationship strength than "Pearson correlation coefficient".
Stochastic: Any system in which the probability of the next value depends on the current, and possibly additional previous states. Examples include time-series systems and Markov chains.
Student's T distribution: Also called the "Gosset distribution" by fans of history and Guiness beer. A "normal distribution" for the more realistic scenario that the true theoretic "standard deviation" is unknown and is being estimated from the data just like everything else. Also has "degrees of freedom".
Trimmed mean: A compromise measure between the "mean" and the "median". For the x% trimmed mean, the top and bottom x% of the data are removed first, and the mean is taken. The 0% trimmed mean simplifies to the mean, and the 50% trimmed mean simplifies to the median.
Tukey's Test: A post-hoc, or after-the-fact, analysis done on data using results from an ANOVA. This test is used to test the (multiple) hypotheses of each pair of group means in the data.
Type I error: A decision error where a null hypothesis is rejected when in fact it is true. The acceptable chance of a Type I error is called alpha, or the Type I error rate.
X variable: Term for "explanatory variable" that emphasizes that the values of this variable are shown with the x-axis in graphs.
Y variable: Term for "response variable" that emphasizes that the values of this variable are shown with the y-axis in graphs.