How to produce PLM market share data at an amazing level of detail

by Pete Maskell
@PeterM_Cambashi
LinkedIn

I recall an anecdote from when a consulting firm paid a Chinese market research supplier to provide a report on the PLM market in China. In that report, there happened to be data on CD-adapco’s revenue in China. Since I was with CD-adapco as vice president until Jan. 1, 2011, I was amazed that the Chinese PLM report had CD-adapco China revenue numbers at a level of detail that even we at CD-adapco didn’t have!.

    – Dennis Nagy, BeyondCAE

In the latest edition of upFront.eZine I found this particular paragraph from Dennis Nagy worthy of comment. Dennis was describing some of his experiences as part of a helpful response to http://www.upfrontezine.com/2017/05/measuring-the-unmeasurable-cambashis-methods.html , and he raises an interesting question: can research and analysis deliver revenue numbers with accuracy which matches or exceeds a vendor’s own figures?

I’ll begin by very openly admitting that Cambashi’s models contain market numbers to levels of detail that the vendors themselves don’t even have. I’ll even go further by admitting that those detailed numbers have increased error bars. In fact, as you increase the detail, the error bars increase.

That’s quite a frank thing for a company that produces market numbers to say, so I should probably elaborate quickly before someone quotes me out of context.

Highly granular data, to levels of detail often finer than vendors themselves produce, improves the aggregate quality of our data and improves our ability to customise data for our clients.

Very broadly Cambashi sees the world in terms of three main axes – products, countries and industries – and for each of these axes we model data to as fine a detail as we reasonably can. I’m going to focus on industries now because that’s the area on which I spend most of my time.

Industry approach to market sizing

Cambashi maintains a standard list of 112 industries. Those 112 industries were chosen to provide granular detail in industries important for engineering and design software. This means very granular detail in areas such as manufacturing, construction and utilities.

Before we can understand the market for engineering and design software itself, we need to better understand the markets in which it is used. Cambashi captures general market data such as employment data, value added data, investment data, growth forecasts and application spend data. All of this data is updated annually (or better) and all of this data is mapped to our standard list of 112 industries.

From this we know which industries are large, we know which industries employ lots of mechanical engineers (or architects), we know which industries are investing in machinery (or buildings), which industries spend heavily on applications and which industries are forecast to grow quickly in the next 5 years.

It’s fair to say we know these 112 industries very well.

Product / vendor approach to market sizing

Going back to the software vendors: PTC publishes data for 7 industries, ANSYS for 11 industries, and other companies publish more, less or no industry data at all. In every case, no matter the published level of detail, we attempt to model the vendor by Cambashi’s standard 112 industries. Here the “triangulation” methodology described by Peter Thorne in the Upfront.eZine article mentioned above comes in to play: PTC might not publish how much of their “Retail & Consumer” figure is “Mfg of domestic appliances” or “Mfg of wearing apparel” but because we know the overall size of those industries, and can estimate their propensity to spend on manufacturing software, we can make an estimate to that level of detail.

I’m desperately trying not to turn this article into a hundred page methodology report but I would like to add that this is a somewhat simplified explanation. The country axis is another key contributor to the estimates, PTC is strong in the US and Germany but not so strong in China, but because we have all the industry data described above for these countries we can weight the contribution accordingly. Another very important constraining factor to this process, and perhaps one that isn’t highlighted enough, is Cambashi’s combined experience and expertise. We have a number of engineering software experts on our team, many of which have worked for the major AEC, MFG and GIS software vendors. Their qualitative input to this process is invaluable.

So we’ve reached the point where we have an estimate for PTC’s revenue in the “Mfg of domestic appliances” industry and this is where I circle back to my point for this article: this estimate has a large margin of error (although the process above means that it can’t be very wrong). The only people who can verify that is PTC and in fact, that is part of our process, providing vendors with cuts of our data for verification. But this particular data point for this particular vendor has not (yet) been verified.

Why the ultra-fine level of detail?

So now we get to the key question of why go to the effort of producing data to this level of detail? There are two main answers to that.

1. Comparability. The process I’ve described above is repeated for every vendor in the same way; using the same general market data for the same 112 industries. Whether the company publishes industry data or not, the method is consistent and repeatable. Which leads to the assertion that, although we may be uncertain of the absolute size of any given data point, the figure is consistent industry to industry and provider to provider. We can’t be certain of the absolute size of “PTC – Mfg of domestic appliances” but we can compare that figure confidently against another industry for PTC or indeed the same industry for another provider. These comparisons are then more reasonable than comparing one of PTC’s 7 industries with one of ANSYS’s 11. Each vendor has its own set of definitions for their industry segmentation. Even something simple like “Automotive” may include electrical suppliers for one vendor, but exclude it for another.
2. Aggregation. A common theme we see at Cambashi is clients asking for market data in their own terms. Vendors want to see the world in their standard set of 12 industries, or their standard set of 7 software categories. To do this it’s essential we have a base of very granular data from which we can map the data to the client’s needs. Is “Mfg of Soaps & Detergents” a process industry or a consumer goods industry? The answer may be different for Autodesk and Aveva, but the key point is that the fine level of granularity of our data allows us to make this distinction on the client’s terms.

Aggregation brings about another important benefit and that is improving accuracy. The more our data is aggregated the smaller the error bars become, because we have much better check data at these higher levels – total company, or company-in-country revenues, total country spend – and sometimes, survey results for total industry or industry-in-country. And our Sudoku-like balancing of numbers means a good number in one cell helps guide all the other cells in that row, column and every other dimension in the model.  Although we can’t be certain about the size of “PTC – Mfg of domestic appliances – India”, when that number is rolled up into, “all providers – Mfg of domestic appliances – APAC” we can have much higher confidence in the result, because of the influence of the good data we have at the higher levels.

Other benefits

Furthermore, modelling at a more granular level improves accuracy in other ways. For example, vendors may only consider the “Electricity utilities” as a single segment. But sub-industry granularity allows us to better model, for example, GIS software which is used significantly more in electricity transmission whereas Plant software is used significantly more in electricity production.

I can’t comment on the quality of the data the consulting firm purchased from the Chinese market research supplier, but I can say that producing data to levels of detail finer than the vendors themselves is an important part of what Cambashi does.

If you want to find out more about our datasets and request samples, check out our Market Data pages.