Model Risk, Testing and Validation
Between 2000 and 2001, National Australia Bank took write downs totaling USD 1.2 billion on its US mortgage subsidiary HomeSide Lending. The losses were attributed to a series of errors in how the firm modeled its portfolio of mortgage servicing rights. This is not an isolated incident. In finance, models are widely used. Usually, they are formulas or mathematical algorithms. Examples include portfolio optimization models, option pricing formulas and VaR measures. Flaws in such models cause firms to misprice assets, mishedge risks or enter into disadvantage trades. If traders or other personnel become aware that a model is flawed, they may exploit the situation fraudulently. Flawed risk management models can cause firms to act imprudently—either too aggressively or too cautiously. For banks, they can result in regulators imposing additional capital requirements.
Berkowitz and O’Brien (2002) used daily P&L and VaR data from six major US banks between 1997 and 2000 to assess (backtest) the performance of the banks’ VaR measures. They concluded that
Banks’ 99th percentile VaR forecasts tend to be conservative, and, for some banks, are highly inaccurate.