Heidi Hewett just reported the following on her blog, about a productivity study:
According to a recent independent study, AutoCAD® 2011 can help you work up to 44% faster with the latest productivity enhancements.
I have a couple of problems with that sentence. First, it’s not an independent study. It’s a study conducted by long-time respected CAD figure David Cohn, but it was specified and paid for by Autodesk:
This productivity study was performed at the request of Autodesk Inc., which funded this work.
That’s not exactly independent then, is it? Second, the study does not state that AutoCAD 2011 is responsible for a 44% improvement. That’s a figure that combines both the effects of AutoCAD 2011 (over AutoCAD 2008), plus the effects of using a newer, faster PC. Just stating that figure wthout such a disclaimer is misleading.
Now to the study itself. Let me make it clear that I have no problem with David Cohn, who is respected, experienced and honest. I do not doubt that his study accurately describes his observations of the time taken to perform the chosen operations on the chosen drawings. The problem is that the study is designed to concentrate purely on a set of AutoCAD operations that benefit from the changes of the last three releases. In other words, the dice are very heavily loaded. To David’s credit, he states that very clearly in the study report:
Each drawing was chosen based on a number of criteria designed to showcase one or more features of the software that did not exist in AutoCAD 2008 but were added in subsequent releases. While each drawing could certainly be produced using the features and functions available in AutoCAD 2008, the advanced capabilities added in subsequent releases would likely enable a typical user to produce the drawing faster using AutoCAD 2011.
Since the premise of the test was to determine how much time could be saved by using a new feature, the test itself was already predisposed to show that using AutoCAD 2011 is more productive than using AutoCAD 2008.
A quick skim-read shows that there are several other problems with the study. For example, it doesn’t attempt to measure the productivity of those operations that are common to both releases, which are much more likely to be used in bulk by typical users. The report states that the Ribbon interface is likely to be more productive, but makes no attempt to justify that by comparing the exact same operations performed using the two interfaces.
In addition, both AutoCAD 2008 and 2011 are measured on a typical middle-age PC using XP, but only 2011 is measured on a modern PC running Windows 7. The report states that the latter tests were performed after the former tests, so the times will also be biased by familiarity with AutoCAD 2011, the drawings and the operations required. That’s where the 44% figure comes from, and it doesn’t mean anything.
What’s the point of studies like this, that are self-evidently designed to produce a good-looking outcome? Who are they supposed to fool? Come on Autodesk, either do these things properly or don’t do them at all. Please.
I have to say that based on my own experience, I would say that there is about a 15% increase over 2010. But I would say this jumps to about 30% on drawings where .SHX file errors were present. This is using the same systems as we were on 2010.
Similar paid-by-Autodesk studies from earlier years also show massive increases in productivity. If one were to multiple them all together (percentages are multiplied, not added), then AutoCAD users would now be working the mythical 1-hour day.
But they’re not.
Instead of these direct-to-marketing studies, I would be more interested in a study for A2011 between ribbon vs. classic interface and layer palette vs. layer dialogue. All companies that I know adopted classic interface and a most appreciated tip is LAYERDLGMODE variable.
Ditto Ralph’s and Joao’s comments.
Another issue I would like to point out is the effect of using some of these new tools in real world (i.e., large) drawings. It doesn’t take long browsing Autodesk’s own Civil 3D forums and/or various blogs to figure out that in order to maximize productivity, you have to keep the Properties palette turned off, use the classic layer manager, disable the InfoCenter, and go back to using classic hatching. It’s unfortunate that some of the new features that could save time in theory, actually slow you down, regardless of the quality of the hardware in use.
Ditto R.K.’s, Joao’s and Ralph’s comments.
To be fair, the improvements to the polyline editing tools are fantastic. It’s basic stuff I wished for back in 2007 (http://worldcadaccess.typepad.com/blog/2007/09/what-needs-to-b.html), but better late than never, I always say.
Transparent hatch – um, yeah, okay. It’s handy in the rare cases I would be doing illustrative work in AutoCAD. Which, given the power of Photoshop, Illustrator, MS Paint, etc. is a rare thing indeed in my case, and I think for most people as well.
The improvements in the 3D realm are great, but still trails almost every other 3D app out there. It’s laggy where it cannot afford to be be and still lacks lots of functionality for heavy duty 3D work to be effective compared to SketchUp, Modo, MoI, 3ds Max and others.
Performance-wise, any functional productivity gains in the AutoCAD 2011 toolset are killed with power-sapping blather which doesn’t improve anyone’s experience. For example, under 2011, I have to disable the Properties Palette (or at least keep it “minimized”) in case I commit the sin of selecting a linear dimension. Doing so with the PP open sends AutoCAD into a performance tailspin as it has to gather and display all of the dimension properties.
Oh, and by the way, this is on a brand new drawing, with nothing in it except a dimension, using the OOTB Standard style, on a shiny new 2.8Ghz Core i7-860 based machine with 8GB of RAM running Windows 7 64-bit.
The first thing I do with many of my students moving “up” from 2008 to 2011 is to teach them how to disable all of the performance-sapping things and get the program back to normal.
Using AutoCAD Mechanical to create parts lists showed “Productivity up 7800%”. Remember this and other outrageous claims published in Autodesk’s Productivity Study done comparing AutoCAD and Mechanical. It was misleading and so are many others.
Steve, having downloaded and read thru’ David Cohn’s paper and putting the ‘independent ‘ argument aside, the problem David’s effort has created is one of relevance. It is not possible for readers to determine if David’s trials are applicable to their situations. For a report of this nature to be taken seriously we need to know much more about how and why and in fact be able to see his trials in action. Just one simple point here: when a selected drawing was first done was it done in 2008 and then 2011? If, as it would appear, the same person is doing both drawings then the order it is done is both relevant and important in determining the outcome.
In an earlier posting of yours in relation to the Ribbon; I showed how wrong it was to take, at face value, comments on productivity improvement. I used in my trials, on that occasion, a completely new users to both old and new interfaces and clearly showed the Ribbon was no more productive – indeed quite the reverse. But once the user had been ‘blooded’ with both, variations in performance could immediately be seen when doing the same task again. David’s experience, if he is the one doing the testing, almost immediately warns me to look more critically.
Hatching – in AutoCAD – is another of the so called productivity improvements pulling the wool over people eyes! Associative hatching was a real improvement and easily proved but the changes made in 2011 not so!
I would like to ask if David (that is Autodesk) would be willing to publish (video) of the testing done for this report. That would be of considerably more value, to all and more in line with where we are with technology in this day and age. Is it already available and I have not been able to find it?
We have been using AutoCAD 2011 in production work for a full week now.
The in-place polyline editing is really nice… but on the whole, the program just isn’t very ‘zippy’. I’ve been trying to think how to best explain it, and right now, this is pretty close to how I’m feeling:
In this illustrative example, Mr Tudball portrays me.
I’ll leave it as an exercise to the reader to determine which character portrays AutoCAD 2011…
Another thing that is rather bothersome (not limited to the current version) is that when AutoCAD crashes up comes a dialog box asking to send an error report to the Mother Ship. OK, fine, fer sure, fer sure, go ahead, send it, and be done with it…
But no, not quite yet…, NOW I’ve got to wait for IE (not my browser of choice), to crank up, then go to some “Thank you for your support!” page. Is this really necessary?
It seems to me that the sooner I can restart the program, the happier I’d be, and the sooner I’d forget about the annoyance – but apparently, I need to see a little sideshow first. Autodesk, if you want my data, which I’m okay to provide you in order to assist you, I could do without the little song and dance routine…
Little off topic, sorry, but still related to overall productivity, methinks.
Yeah, that IE “thanks” is irritating as heck.