Product
Model
Validation
Building confidence in your modeling results requires a clear understanding of their validity and the importance of further calibration. Forvio using key metrics to ensures model credibility for optimal decision-making.
General KPI
NRMSE (core metric)
(Normalized Root Mean Square Error): Measures model accuracy by comparing predicted values with actual values. It is normalized to the range or mean of the data, allowing for better comparison between different models or datasets.
MAPE.LIFT (only with calibration)
(Mean Absolute Percentage Error Lift): Measures the percentage error between predicted and actual values and expresses the improvement (lift) in model accuracy after applying a certain intervention or adjustment.
Decomp RSSD (core metric)
(Decomposition Root Sum of Squared Distance): Evaluates errors in the decomposition of time series or other decomposition models by summing the squared errors of the individual components of the model.
R² (additional metric)
(R-Squared): Indicates how well the model explains the variability of the data. It represents the percentage of the variation in the dependent variable that is explained by the independent variables in the model. A value close to 1 indicates high model accuracy.
Out of the sample validation
The general approach in machine learning involves testing a model on unseen data by repeating the process multiple times using a dataset split. Optionally, you can enable cross-validation in Forvio with an 80/20 split, where 80% is allocated for model training, 10% for model selection, and 10% for validation.
Interactive validation
Forvio offers an interactive calculation of all KPIs for selected periods, e.g. the last 7, 30 or 90 days, or any specific range, e.g. promotional season. This is a much better evaluation option as customers can select the period that is more critical to the business, or the period in which a specific mix of channels has been used.
Confidence intervals
Confidence intervals are statistical ranges that estimate the uncertainty or variability of each marketing channel’s impact on overall outcomes. They provide a range within which we expect the true effect of a marketing channel to lie, typically with 95% confidence. Channels with narrow confidence intervals are usually considered more reliable for investment, while those with wider intervals may require further examination or additional testing. Forvio can display these confidence intervals to represent the uncertainty for each channel.
