May 13, 2010
Dealing with the limits of science
Ever since the crisis began - some will say way earlier than that - there've been those who said that risk can never be measured and as such, all efforts in that direction are a waste of time and a distraction from managing the business. The crisis added a new dimension to the debate, by accusing the rocket scientists of being a cause of the crisis.
So this works like this: I'm going to a tailor and order a pair of flary trousers to impress my girlfriend (don't comment on the tastes), she ridicules me for the bell-bottoms, then I complain that the trousers are too large at the ankles because the tailor's scissors weren't sharp enough or that he wasn't careful when he operated the scissors.
The RiskMetrics Technical Document, arguably the blueprint of the widest used - and most maligned - measure of market risk, VaR, makes clear from the beginning (Chapter 2, "Historical perspective of VaR") the limitations of the statistical approach proposed within. The caveats are not, as may seem at first read, simply limitations of liability; in fact, at the top of page 22, the document recommends that "Risk managers should use both approaches — the statistical approach to monitor risks continuously in all risk-taking units and the scenario approach on a case-by-case
basis to estimate risks in unique circumstances."
Now, any rocket scientist starting to read the document may be forgiven for skipping the above recommendation - their brief was not, until very recently, to address scenario analysis but to produce more and more sophisticated versions of the statistical approach. Of course, any scientist worth her salt needs to state clearly that when data are sparse, statistics will be shaky and potential for financial loss greater than estimated - but we assume that scientific integrity survives the transition to the financial world.
So what should a rocket scientist do when asked to model something she shouldn't, like firm-wide operational unexpected loss at 99.9% confidence? a. Do it at 75% and state that this is the best that current science can achieve? b. Do it at 99.9% and ignore quantile estimation errors, in the safe knowledge that few other scientists and nobody else can spot the crucial omission? c. Reject the instruction, stating that the work cannot be done as requested, and change jobs?
The limits to our science and technology are painfully obvious all around us - space, oil fields, medicine, etc - so, when snake oil is offered for sale, the buyer better beware - whether that is a trader 'buying' the model from the quant or, more often than not, the trader's client buying overpriced or 'faulty' financial contracts. The failure of quantitative finance damns us all, buyers and vendors alike, but rocket scientists bear less responsibility than most for it all.
Posted by jaguar36 at 05:00 PM
| Comments (0)
