This seems to be a foreign idea in our industry. We typically use empirical methods to collect data sets but we very rarely use any theoretical methods. To understand the difference think about rolling a pair of dice. Empirical probability would analyze previous rolls to help determine the probability of what you will roll in the future. Theoretical probability would analyze all of the possible outcomes of rolling to help determine the probability of what you will roll in the future. Both are equally valid methods.
I think the reason that everyone has had such a hard time with ROI is that the approach is wrong. We are spending all of our time trying to collect information that is in no way a real indicator of BIM ROI. We are convinced that the secret to BIM's success is locked somewhere in the number of RFI's, the length of the schedule, or in the number of change orders. But no matter what empirical data you bring back it still does not answer the one question the industry really wants to know.
How do you prove that doing the project with BIM brings more value to the project than doing it without BIM?
Everyone knows that there is no empirical data (one-offs don't count) to help answer this question so it is just assumed that there is no possible way to answer the question. But instead we try to force the issue. We analyze one project that has less RFI's than average and proclaim that "BIM reduces RFI's". We do the same thing with Change Orders, Schedule, and anything else we can think of. But the logic is horribly flawed.
You would think that bad ROI's would result in people just disregarding them. But in my last post, I talked about how BIM is often made to sound confusing. So to help simplify things, I found a good example of what almost all examples of BIM ROI sound like to me. Please read the following exchange between Lisa and Homer Simpson:
Lisa: ...I could claim that this rock keeps tigers away.
Homer: Oh, how does it work?
Lisa: It doesn't work.
Lisa: It's just a stupid rock.
Lisa: But I don't see any tigers around, do you?
Homer: Lisa, I want to buy your rock.
In this example BIM is the Rock. The ROI is the fact that you don't see any tigers. Replace the word tigers with RFI's or some other totally inconsequential item and that is how ROI is currently reported for BIM.
Unfortunately this practice regularly leads to one of two results. Either you see the flaw in the logic and don't buy the rock, or you buy the rock and jump into a tiger pit to see if it works. Both of those results are bad for BIM and the industry as a whole.
As usual at Epic BIM, since I brought up the problem I will try to offer a solution. As mentioned above, I don't have one yet, but I feel like the answer lies somewhere in the theoretical approach instead of the empirical approach. Or at the very least, a mix of the two. When we report ROI what we are really trying to do, is give anyone concerned a better feeling out the probable outcome of future work. So just like my dice example, either method can provide a valid answer to the problem. But in the absence of empirical data, we need a more theoretical model. A way of analyzing projects in advance and giving you an idea of the expected outcome using BIM processes and the expected outcome not using BIM processes. This is a very viable option given the correct criteria. The problem is that everyone is so busy collecting useless information that no one (that I know of) is actually looking into what that criteria is.
Just like the dice example, this analysis is more difficult than using empirical data. But like the dice example, the results are just as valid. I know there are many that will say, well who cares we are all going to do BIM anyway because the market place demands it. I am not entirely sure that's true any more. I think lately there have been a lot of people that have become jaded with the BIM. I feel a decided loss of momentum for BIM in the industry. I think that if we don't want the BIM train to come to a halt the experts need to offer a better answer for those that are wondering why we do BIM, other than the rocks they have been selling.