Tag Archives: Statistics

How you feel about CAD in the Cloud

In February 2010, I ran a poll to gauge your feelings about CAD in the Cloud and left it going until October 2010. Just over a month ago, I asked the same question again and ran an identical poll. I have closed that second poll, and here are the comparative results (new poll at the top, old poll at the bottom):

Cad in the Cloud 2010 and 2011 Polls

I’m not pretending this is a scientific survey. There is bound to be some self-selection and other sources of bias, as there must be with anybody’s survey processes (very much including Autodesk’s CIP). Looking at the logs, I haven’t detected any obvious attempts to interfere with the poll, although there’s nothing I can do to stop people voting multiple times if they have access to multiple IPs (e.g. a work and home account). There are fewer votes in the newer poll only because I didn’t let it run nearly as long; the rate of voting actually doubled this year.

Caveats aside, what can we say about the way in which poll respondents’ views have changed in the last 18 months or so? The most obvious change is a marked reduction in the proportion of people who now feel Hopeful. There is a small increase in the number of voters who are now Excited or Frightened, but a big leap in those who are Concerned.

Combining the two positive choices (Excited and Hopeful) to come up with a simple “Cloud approval rating”, in 2010 39% of voters were positive about CAD in the Cloud. In 2011 that number is down to 29%. In other words, people voting Cloud-positive are about a quarter down on last year.

Most poll respondents here are Autodesk customers, so I think it’s fair to say that this result reflects poorly on the effectiveness of Autodesk’s Cloud marketing. After a year of providing an increasing selection of Cloud products free of charge, and a big Autodesk Cloud launch and publicity immediately before the poll, one would expect a lot more people to be sold on the Cloud, rather than the reverse. I don’t see that Autodesk has done anything atypically bad with its Cloud marketing (certainly nothing like Dassault’s SolidWorks train wreck), so what’s going on? Is CAD on the Cloud really an unsellable prospect?

Cloud discussions generating interest

This is one of those self-indulgent posts you probably hate, so feel free to skip it and just read the more interesting stuff.

Last month, my site statistics went through the roof. Here’s a graph that shows the number of unique visitors and the number of visits per month since I started the blog in February 2008. Page views, hits (a pretty useless statistic) and bandwidth all spiked in a similar fashion.

I remember being very surprised when over 1,500 people visited my blog in the first month, as I would have been very happy with a few hundred readers. I was astonished when more than 5,000 people visited here on the second month. Last month, there were 30,921 unique visitors who visited 58,342 times, viewing 129,206 pages. I’m sure there are other CAD blogs with many times the traffic, but for this blog, October’s numbers were crazy. The mentions on upFront.eZine didn’t hurt, but the daily statistics were already high and didn’t show a huge leap afterwards.

So what’s going on? Well, just posting anything rather than little or nothing (as has happened here from time to time) obviously helps a lot, but I think it’s more than that. I think it’s the Cloud generating interest. While it might be tempting for Cloud proponents to associate interest with excitement, that would be a mistake. Judging from the comments and poll responses here and elsewhere, I’m convinced that many more people are interested in CAD in the Cloud because they are concerned about it, they fear it, they even hate it. Given that atmosphere, I think CAD in the Cloud is going to be a very hard sell.

What proportion of Autodesk customers really are on Subscription?

In my recent interview of Autodesk Subscription VP Callan Carpenter, he made these statements:

…there is a very small fraction of our revenue that comes from upgrades at this point in time.

We’re down to very low single digits of customers who upgrade, and of those only half of those upgrade 1 or 2 years back. So we’re talking about approximately 1.5% of our revenue that comes from customers upgrading 1 and 2 versions back.

…[customers who upgrade] 1 or 2 [releases] back, a very small percentage of our customer base, less than 2% of our customer base that was buying those upgrades.

Others are calling those numbers into doubt. Deelip Menezes (SYCODE, Print 3D) estimated the numbers of AutoCAD users not on Subscription at 66% (or 43%, depending on which bit of the post you read), by counting the AutoCAD releases used by his customers and making assumptions about their Subscription status from that. That’s an extremely suspect methodology, as I pointed out:

Your numbers don’t really tell us anything about Subscription v. upgrade proportions. All they tell us is that large numbers of people wait a while before installing a new release. We all knew that, surely.

However, Deelip’s post did prompt me to point out this:

…there is a fair point to be made about people on earlier releases who have hopped off the upgrade train altogether, or at least for a significant number of years. How would they be counted in Callan’s figures? They wouldn’t exist at all, as far as his income percentages are concerned.

Owen Wengerd (ManuSoft, CADLock) asked a random sample of his customers and came up with 82% of them as non-Subscription customers. He also noted that he could come up with a 3% non-Subscription figure if he cooked the books by selectively choosing a convenient time slice. Owen doesn’t state the numbers in his sample, or indicate (or know) how many of the non-Subscribers are also non-upgraders.

I’ve added my own poll (see right) just to add to the mix.

Nothing we can hang a conclusion on yet, then. But Ralph Grabowski (WorldCAD Access, upFront.eZine) uses Autodesk’s own figures to point out that upgrade revenue has increased 18% and Subscription revenue only 7% in the last year. I’m not qualified to perform an analysis of the 2011 Q1 fiscal results, but I can find the figures listed as Maintenance revenue ($195 M) and Upgrade revenue ($51 M). That looks to me like about 21% of the Subscription/upgrade income is coming from upgrades.

Also, according to the published figures, Autodesk has 2,383,000 customers on Subscription. If that represents about 97% of customers, does that really mean Autodesk has only about 2.5 M customers? If I’m looking at these figures in the wrong way, feel free to put me right.

So, what’s the truth? What proportion of Autodesk customers really are on Subscription? 3%? 21%? 43%? 66%? 82%? I’m going to ask Callan a follow-up question about this and will report back on what he has to say. In the spirit of this post, I’ll be asking him for a lot more detail. Watch this space.

The Machine that Won the War

I just wanted to get on the record that I don’t trust claims based on statistical data without being able to review in detail the methods used to obtain and interpret the data. Even with the best intentions, full integrity and honesty, it is not difficult to come to completely the wrong conclusions based on apparently compelling statistical evidence.

This isn’t just theory, I’ve seen it happen. Detailed percentages presented at upper governmental levels, based on huge sample sets, giving a totally false impression because of errors and assumptions that occur at various places in the process. The exact same question asked twice in the same survey, giving very different results depending on the section in which the question appeared, providing an unstated context to the question. The devil is in the details, and the details can be extremely subtle.

I have a “put up or shut up” rule that applies to anybody who makes claims based on unrevealed statistical evidence. It applies to corporations, news outlets, bloggers, government ministers, everybody. Without allowing scrutiny of the full details, all statistical claims are null and void, as far as I’m concerned. “Trust me” doesn’t cut it. Sorry, no exceptions.

What does this have to do with the title? Those familiar with Isaac Asimov’s short story of that name will understand. I’m sure Robin Capper worked it out immediately.