Mooted VAR substitute cannot be back-tested, says top quant
The Basel Committee on Banking Supervision has proposed replacing value-at-risk with a metric, expected shortfall, that fails to meet a key criterion of its mooted new trading book regime – it cannot be properly back-tested – according to Paul Embrechts, professor of mathematics at Eidgenössische Technische Hochschule Zürich (ETH Zürich).
In the fundamental review of its Basel 2.5 trading book capital rules, published in May last year, the Basel Committee proposed getting rid of VAR – a metric banks have been using since 1996 to work out their own capital requirements – because it does not accurately capture tail risk. Expected shortfall is a better measure in that respect, the review argued. Unlike VAR, it also satisfies a mathematical condition, known as coherence – meaning diversification is captured more effectively.
But the review also proposes to make back-testing a far more important element of the capital regime, with regular tests being used to decide whether risk is being properly measured at the trading desk level – and failure of those tests resulting in modelling approval being removed for that desk. Expected shortfall, a so-called spectral risk measure, cannot be back-tested because it fails to satisfy another mathematical condition, known as elicitability, Embrechts told a seminar on mathematical finance at London’s Imperial College yesterday.
“The question we have been considering is: ‘Is there a coherent risk measure that is also back-testable?’ And the answer seems to be no. That now seems to be the dichotomy – if you want robust back-testing, and say we want to build our systems on that basis, that excludes – almost mathematically – coherence. We have a proof of that for spectral measures,” he said.
The issue is also raised in a study of academic literature on trading risk measurement that was conducted by the Basel Committee prior to the publication of its review – it said the literature shows some criticisms of expected shortfall have either been resolved, or found to be less serious than initially thought, and that there have been “improvements in back-testing methodologies”. It goes on to suggest a work-around if expected shortfall back-tests prove not to be satisfactory – generating a parallel VAR measure and testing that instead: “If the VAR is rejected, the corresponding expected shortfall calculation can hardly be correct,” it says.
And Embrecht’s claims were disputed by one audience member – a senior modelling expert at a European regulator – who claimed the definition of elicitability was too narrow to account for all back-testing methodologies.
“If you look at the definition, elicitability is about testing in one step. But if you can break up expected shortfall into two elicitable pieces you will be able to backtest in two steps,” he said. Embrechts said he was “not so sure” about this and that further work was needed.
When asked which of the two mutually exclusive qualities he would recommend the committee insist on – coherence or elicitability – Embrechts said back-testing was more important than diversification, though he was quick to add caveats and also said the committee should maintain its sceptical stance on modelling.
“If you held a gun to my head and said: ‘We have to decide by the end of the day if Basel three-and-a-half should move to expected shortfall, with everything we know now, or do we stick with VAR?’, I would say: ‘Stick with VAR’.
“Stick with VAR, but think more about the multipliers and model uncertainty – we know the enemy in VAR. Especially put much, much, much less confidence and belief in trying to regulate complex products with these numbers,” he said.
A feature on the fundamental review – and the ongoing trading book capital debate – will appear in the April issue of Risk
This article was first published on Risk