[PPM] POC vs Pilot

POC or PILOT approach to you PPM implementation?

This article briefly explains the key ingredients and differences between a POC (Proof of Concept) and a Pilot approach when selecting an implementation approach for a project and portfolio management solution.

In general, either a POC or a Pilot is recommended as a way forward to ensure things are as they should be, before a potential larger project could impact all users within an organization. In other words, reducing the risks for project failure later. Often organizations don’t understand the core difference between the two approaches, even though the requirements towards the vendor is very much depending on the chosen approach.

To fully understand the differences between a POC or a Pilot, one should start by looking at the defined success criteria for each approach:

Success for a Proof of Concept

The POC must answer all the questions set out by the success criteria in the form of pass or fail. Even if an organization decides to not move ahead with the project, the POC might still be very successful as it has analyzed key risks before a vital decision is taken. A POC is not successful if the defined questions or risks have not been answered in full. A POC is like mitigating the risks with the highest impact for your potential PPM implementation.

Success for a pilot approach

The pilot is successful, when all required functionality has been deployed, used and validated in full by a low number of users (typically 5% of total expected users). In other words, the pilot is successful when the entire system can be moved into production, and the pilot is obviously unsuccessful if it can’t be moved to production.

Comparing the two approaches could be shown as below. In this case, when implementing a PPM system such as Microsoft Project Online.

Running a POC

When going for a POC approach it is likely due to concerns the organization have regarding specific areas of importance or doubt e.g. can our work process really be implemented? is performance good enough? will it integrate to our ERP system? etc.

These questions are key ingredients for the defined “success criteria” that should be known for all participants of the POC project. When questions are agreed upon, they must be written in a way that allows for other to later validate the result. The most used approach to this, would be to use a “test case” approach with two categories of validation “technical” and “business/process”.

In the POC the PPM system should (only) be configured so it can validate all required use cases (questions and concerns). This should be done in a MVP (minimum viable product) manner, as we only do a POC to validate our questions and concerns, not the things we expect or know is already working.

Running a Pilot

When choosing to go for a Pilot approach, it’s because the organization has already selected the preferred system such as Project Online. In this case, all functional requirements, and the organizational processes (workflows), must be configured, and used in real life scenario but only for a few users (pilots). Managing this kind of Pilot project can be done in multiple ways depending on the project model chosen such as Prince2, PMI or even an agile (Scrum) approach. Keep in mind, the scope should come from a requirements specification, meaning change request management must take place, making it hard for a 100% agile approach.

Best practices and steps used in a PPM system Pilot:

1. Understand the reporting requirements from a portfolio manager perspective
   1. Which technology, mobile devices, data refresh rate, notifications etc
2. Which project types are in scope for the pilot
   1. Large or small projects, Departmental types, etc
3. What governance should be applied
   1. Project Lifecycle Model, Phases, Stages and Gates
4. Stage activities
   1. Which activities must be performed and when e.g. risk analysis, cost/benefit etc
5. Specific fields needed
   1. Looking at the portfolio reporting requirements gives an understanding of which data and custom fields must at least be available for data input
6. Configuration of the requirements identified in step 1-5
   1. Validated by the pilot users
7. User training
   1. And definition of who does what and when
8. Timeframe for pilot users using the system
   1. Typically, 2-4 months with weekly feedback loops
9. Pilot approval and potentially migrated into production and made available for all users
   1. Typically, this also involves more training
10. Plan for next phase and iteration
    1. Often seen is a stronger focus on project planning and tracking activities but it could be something else such as integration to LOBs etc

POC and Pilot hybrid

In some cases, it would make perfect sense to first run a POC followed by a Pilot. An example could be that an organization need to sure that a special integration will work in real-life before considering running a pilot that requires user involvement. The POC would then be a simple matter of answering one question/requirement, and if positive, then move ahead with the planning of a Pilot to further test the general requirements and organizational processes.