by alex on October 20, 2011
Ben Sapiro showed off his Binary Risk Assessment (BRA) at SecTor recently. While I didn’t see the presentation, I’ve taken some time and reviewed the slides and read through the documentation. I thought I’d quickly give my thoughts on this:
It’s awesome and it sucks.
That’s not damning with faint praise, rather, it’s acknowledging that it’s not really “risk” but is a useful tool if your goal is to be quick and dirty about vulnerability severity.
In other words, this is much better than CVSS, and should probably replace it immediately.
TILTING AFTER THE WRONG WINDMILLS
In fact, it’s a shame that Ben chose to compare this to OCTAVE, FAIR, SOMAP and others. Because if he positioned this as “stop screwing around with CVSS” and “not really risk but a vuln rating” I would be telling everyone how much I liked it in that role.
In addition, if he positioned it with the Accounting/Audit Industrial Complex as good tool in the toolbox to compete with^H^H^H^H^H^H augment their RCSA nonsense, I could probably welcome it there, as well (though not as an optimal solution).
The power of BRA is the fact that Ben chose to make things “binary”. I can see this simple approach working well because it doesn’t allow you granularity – none of this arguing over “Moderate” or “Moderate-High” – just yes/no.
Also, Ben’s done a really good job thinking through what creates risk. For the FAIR familiar there’s the concepts of TCap and Control/Resistance Strength. I like that.
Speaking of subjectivity, I believe that Ben uses the power of binary choice to suggest that BRA “highlights” subjectivity. Not to be a rude pedagogue, but it really doesn’t “highlight” subjectivity as much as it just doesn’t give you many choices as to where to “put” that subjective measurement. Everything about it is still subjective (but that’s OK), and to reduce (or as I would rather “address properly”) that subjectivity would take more complexity than I believe Ben wanted to build (again, that’s OK).
As such at the end of the day, Ben’s right, it’s never going to be a replacement for what he calls “complex” analysis methodologies. And because it doesn’t properly address subjectivity, BRA is not for formal risk or threat modeling. I could never use it in my current capacity, as BRA just leaves a few too many questions unanswered. I don’t have time for the arguing, I just want your SME estimate, throw that puppy into OpenPERT and be done with it.
Furthermore, it’s odd because even though BRA suggests that it is designed to to “not ask anyone to guess on event frequency in the absence of statistical data (whatever that is)” it seems Ben’s intellectual honesty still could not let him escape the need to highlight it. If you look at BRA the model, that occurrence thing there, yeah, that’s frequency. It’s just a “binary” frequency determination which means….
BRA only talks about what’s possible.
As a risk model, this is the point at which we reference the Tacoma Narrows suspension bridge that oscillated wildly in the wind. Constructed nicely and all, but a small fundamental flaw in design renders it crazy bad for its purpose.
Also, impact is difficult for me to buy into because it uses asset value. I hate to break it to you, but asset value mainly matters to threat motivation modeling. The accounting value of the asset is RARELY the same as the losses we actually realize.
CHOOSE IT OR CHUCK IT?
So it wouldn’t be a review without such criticism. This is one reason I hate reviewing things, because it is a critical process. So please note that the above isn’t said with malice, it’s just an examination of the model itself.
In fact, as a tool, I wouldn’t dismiss it just yet. If your security group isn’t formally into risk, is stuck doing too much with CVSS for too little return, I’d jump all over this. If you have bigger fish to fry than an enterprise risk assessment but have the regulatory duty to create a risk register, BRA might just be the thing. If you find yourself faced with an absurd RCSA from audit or something – I might whip out the sweet BRA iPad app and run a scenario or two through. If I actually wanted a risk analysis, however, I would go elsewhere.