Using Analyst Rankings to Help Select the Best CPM Software Apps

This article is part 8 of an 8-part series on evaluating the best CPM tools for your business. Part 8 focuses on why and when to use third-party rankings from analysts when evaluating the best CPM software applications.

While some companies don’t need to go through a detailed selection process to come up with a list of the top Corporate Performance Management (CPM) software solutions for their organization, others have their work cut out for them. If you belong to the latter category, here is a vendor evaluation tool that may be of help.

One of the key items on many CPM product evaluation checklists is to look at how third-party firms, usually referred to as analysts or analyst websites, review and rank vendors.

Below, we will discuss the use of analysts to help evaluate and score the best financial reporting and planning solutions for your business requirements. This type of third-party research can complement the findings and opinions from your internal team.

When analyst firms are of less importance

Before we go into detail about analyst firms, let’s briefly cover some situations where analyst reports with CPM vendor rankings are not as useful and, in some cases, cost extra time and money in the selection process. One such example is when there is already a leading CPM solution partnered with your ERP vendor and offering pre-built integrations and other benefits that outweigh other potential vendor differences.

Another example is when multiple people on your staff have deep knowledge of a leading CPM solution that they have used before, ideally while they worked at a company from the same industry to ensure that there is still a fit.

Which analyst firms should you use?

While there are a few firms with analysts that are CPM industry experts who do months of research every year to analyze trends and rank vendors, there are many more analysts that are a waste of time or are even directly misleading in their rankings.

Examples of professional CPM analyst firms
- Analyst and survey-driven rankings: Gartner and Dresner Advisory Services
- User-driven rankings: G2
Unqualified or misleading “analyst” firms
- Clickbait websites that will come up with their own vendor lists with no proper research, purely to get ranked on a search engine in order to sell advertising or get “sponsorship” money vendors pay to be on (e.g., a “Top 10 CPM Vendors” list)
- Websites owned by CPM vendors ranking themselves
- “Research reports” from “analysts” paid for by a specific CPM vendor

So, if you consult analyst reports to help find the best CPM tool for your company, be conscious of who or what is proving the advice.

How do you know if analyst reports are biased or fair representations of vendors?

It is clear that almost all research performed by a human being is biased one way or another, either consciously (e.g., based on vendor sponsorships or who they speak with the most) or unconsciously (e.g., based on the knowledge of the analyst). Even a firm as well-known as Gartner arguably has some bias in their reports because they include analyst comments, and they include vendor revenue as one of the drivers in their “Magic Quadrants” – something which may or may not indicate who a “leader” should be in a specific industry. Especially with the pace of technology changes and acquisitions in the marketplace, the best CPM solution for your business 6 months ago may no longer be the top choice today.

Other analyst reports or vendor rankings are websites that are driven by user feedback as compared to analyst research. You could argue that these websites provide the most neutral feedback although not as detailed and analytical as the major reports produced by full-fledged analyst firms.

Then there are the countless firms that provide “awards” and rankings based on payments from vendors. These are highly biased and should be avoided. It is fairly easy to detect them by reviewing the firm’s website and observing the lack of depth in CPM research and content.

What professional analysts get right (and sometimes don’t)

The top CPM analyst firms typically do one major CPM report per year. All or parts of the data in the report is driven by customer surveys completed by each vendor’s customers. Because of all the work that goes into these reports and because customers don’t want to be constantly bombarded with surveys, the reports will represent data that is up to 15-16 months old. This lag is because customer surveys typically start 3-4 months before the report is released and then the report will be out on the market for a year until next year’s report is out.

Because many cloud software vendors have monthly releases, and new features arrive all the time, these reports could be missing important vendor features. Websites like G2 and others are starting to follow the same model; they are constantly updated whenever a customer decides to leave their feedback. However, they are less detailed and structured.

Examples of analyst firms that review CPM software

There are a number of companies that provide CPM vendor reviews and market research. Here are examples of three different categories of such firms:

Gartner: Analyst + customer survey-driven vendor rankings. Also provides research reports across almost all categories of IT firms. A major CPM report is produced once per year.
Dresner Advisory Services: Customer survey-driven vendor rankings. Also provides research reports. Almost exclusively focused on CPM (they refer to it as EPM, or Enterprise Performance Management). A major CPM report is produced once per year.
G2: Customer feedback driven. Ranking reports are produced automatically on their website (g2.com). Rankings are continually updated as customers enter their feedback on the G2 website.

How much does it cost to use an analyst firm?

Some CPM vendor research reports are free while many are licensed by the CPM vendors themselves and shared with customers. However, be aware that vendors naturally will only license and share research reports that shed a good light on their product, so there is a bias here to be cognizant of.

Some analyst firms also provide selection services, either as paid calls with their CPM advisors or as full-blown gigs where they will lead or participate in the entire selection process as a “consultant.” In past years, they used to then provide clients with RFP templates with hundreds of pre-defined questions. These have become less popular in recent times as both vendors and internal evaluation teams dread lengthy narratives always shaped to sound good, or they can miss entire functionality areas that are up and coming.

In any case, it is almost always beneficial to do your own homework first by listing your current pain points in as much detail as possible and quantifying ($$) the cost and effort of running the current reporting and planning processes. This “homework” should also include the management team’s vision of the business benefits they want to achieve with a new CPM solution.

Conclusion

The leading cloud CPM software solutions have a lot of features and functionality, and changes and improvements are being released all the time. So, when you and your internal team are coming up with a shortlist of the best CPM solutions for the company’s needs, it is advisable to do your own research, review product demonstrations, and make sure the top candidates integrate easily with your ERP and other important systems. If you have special complexities or a lack of time and evaluation skills on your team, then an analyst firm can assist you in various ways.

Links to useful software research and evaluation assets