Risk Management for Research and Development Projects

Risk Management for Research and Development Projects

Harsha Rodage CSU East Bay, Hayward, CA

Howard Lei CSU East Bay, Hayward, CA

Farnaz Ganjeizadeh CSU East Bay, Hayward, CA

AbstractThere are multiple uncertainties during Research and Development (R&D) phase of the projects such as technological and market uncertainties which are transform into project risks. Hence, the development of risk models is essential to assist the managers to make appropriate decisions during the initial stage of the project life cycle. Although, many approaches are developed for performing the risk analysis during the growth and maturity periods of the project, the methodology for risk evaluation during R&D phase needs further research. In this paper we describe a new approach for dealing with the impact of the risks of the technological projects during R &D phase. This approach is based on the analysis of Knowledge gaps i.e. the gap between what we should know in order to succeed in the project and what we really know in the following two phases: Phase 1 – Risk identification and assessment; and Phase 2 Risk mitigation.

1. INTRODUCTION

   Risk can be sensitivity to stochastic variables. Risk does not necessarily mean uncertainty. It should be noted that, uncertainty is far worse than risk. Uncertainty comes with no information. For risk identification as well as analysis, there will generally be lot of information available. This information can be used for risk assessment and an appropriate measure can be taken to mitigate it.

   Risk can arise from miscommunication or because of the basic nature of the R&D projects. Much scientific research is based on investigating known unknowns, things that we know we don't know. In other words, scientists develop a hypothesis to be tested, and then in an ideal situation experiments are best designed to test the null hypothesis. To begin with, the researcher does not know whether or not the results will support the null hypothesis. It is common for the researcher to believe that the result that will be obtained will be within a range of known possibilities. Occasionally, however, the result is completely unexpectedit was an unknown unknown, things we do not know we don't know. These are completely unexpected situations and are generally

   rare. There are also unknown knowns which can also contribute to risks. These are the thing that you feel you know but actually they are unknown which can be result of communication problems. [1]

   The unknown unknowns are difficult to identify, management of unexpected risk will be discussed in a later section. As an initial stage, we will consider only the known unknowns and the Unknown knowns. Both these type can be mitigated following a normal path of risk identification and mitigation.

   Many New Product Development processes start as R&D projects. Risk of failure is risk of failure for any reason. In technological, innovative industries, new products arise from a combination of innovative technological as well as marketing concepts. In these cases there are scientific or engineering problems to solve resulting in a technological risk as well as the commercial risk of failure in the development projects which create them. Risk is defined here as the risk of failure to achieve success as described by the technological specifications which enable the product and the profit generating objectives which are established at the outset. [2 The application example discussed is this report is a New Product Introduction (NPI) which has following stages. Any NPI can be generalized to have these stages. The risks arise at each of these stages. Thus risk identification can be performed by focusing different areas of these stages. As seen in Figure 1, each project starts by portfolio management where only those projects are selected which align to the business strategy. The technical concept and market strategy play a major role in conceptualizing the product. Each of these areas has risks associated with new technology and/or venturing into the new market segment. All new product development projects have an R&D phase. These projects should not be executed as the normal routine projects, instead they should be applied the risk management techniques discussed in this report.

   Fig 1. Different stages of new product development

2. RELATED WORK

   An extensive literature search was conducted to order to build the necessary framework for this research. In

   Common methodologies for Risk assessment and management, the authors use the Risk register for risk identification [Turnbull, 2001]. The risk assessment is done using two dimensions: likelihood and impact. In their application example for R&D projects they reduce the assessment dimension to likelihood to simplify the process. This simplification does not give an accurate assessment of risks. [9]

   The paper A performance-oriented risk management framework for innovative R&D projects, talks about risk management using Quality Function Deployment (QFD) [Wanga, 2010]. The risk measure is calculated in terms of probability of performance outcome and utility value of performance outcome. [8]

   Project management under Risk: Using the real options approach to evaluate flexibility in R&D discusses risk in terms of variability in cost, technology, market requirement and schedule [Huchzermeier, 1998]. The authors suggest flexibility as an option to reduce risks; there are many other unaddressed topics such as delayed decision can be expensive to the project, market uncertainties reduce over time. [10]

   In paper Managing Technology Risk in R&D Project Planning: Optimal Timing and Parallelization of R&D Activities, author talk about parallelization of activities to reduce the project risk [Crama, 2005]. But sometimes due to technical dependency and increased cost, it is not possible to parallelize activities. [11]

   Management of Scientific Research and Development Projects in Commonwealth Agencies, discusses the generalized approach for managing R&D projects [Barrett, 2003]. The risk management is done using risk register for risk identification and assessment. [12]

   Analysis of risk and time to market during the conceptual design of new systems paper treats every knowledge gap as a risk and the way to mitigate that risk is closing the knowledge gap [Hari, 2003]. In this paper, all the knowledge gaps are identified and quantified just like risk matrix.[13] Whereas the paper Managing Project Risks as Knowledge Gaps quantifies Knowledge gaps by statistically analysing the past project information [Regev, 2006] [14].

3. RISK IDENTIFICATION

   The first step in managing risk is to identify all possible risks. The objective of risk identification is the early and continuous identification of events that, if they occur, will have negative impacts on the project's ability to achieve performance or capability outcome goals. They may come from within the project or from external sources. Risk identification should start with identifying the risk areas.

   Risk identification is an iterative process. As the program progresses, more information will be gained about the program (e.g., specific design), and the risk statement will be adjusted to reflect the current understanding. New risks will be identified as the project progresses through the life cycle. [5] [6] Table 1 show a generalised risk checklist considering different areas such as Technical, Design, Testing, Project

   TABLE 1: RISK IDENTIFICATION CHECKLIST

   No.	Risk Area	Risk Potential
   		No	Yes
   			Low	Medium	High
   1	Technology
   	New technology?
   	Improved version of existing technology?
   	Low understanding of the research area?
   	Involves use of advanced tools?
   	Skillset of the staff is insufficient?
   2	Design
   	Unclear functional requirements?
   	Complex design?
   	Reliability problems?
   	Lot of design changes?
   	Unclear design strategy?
   	Problems with interfacing with other system?
   	Testability issues?
   	Is design maintainable?
   	Design is not user friendly?
   3	Testing
   	Difficult to perform test?
   	Acceptance test required?
   	Integrated testing required?
   	Special equipment and/or software tools required for testing?
   4	Market
   	Unaware of the new market?
   	Customer trust issues because of the new technology?
   	Low product awareness?
   	Is Competitor's product better?
   	High product cost?
   	Long time-to-market and/or accelerating time- to-market is not possible?
   5	Project functions
   	Schedule slippages?
   	Unclear project scope?
   	Roles and responsibilities not well defined?
   	Changes in Task schedules?
   	Changes in Task priorities?
   	Customer requirement changes?
   	Lack of communication?
   	Virtual teams?
   	Cross functional teams?
   	No definite milestones?
   	Monitoring the project is difficult?
   6	Resource
   	Attrition of team members?
   	Need advanced equipements/tools?
   	Requirement of infrastructure?
   	Number of skilled staff is low?
   	Finding skilled labor is difficult?
   7	Financial
   	High training costs?
   	Budget problems?
   	Wrong esimates?
   	High development costs?

   No.	Risk Area	Risk Potential
   		No	Yes
   			Low	Medium	High
   8	Supply chain
   	Establishing new supplier?
   	Lead time issues?
   	High component costs?
   	Supplier end quality problems?
   	Limited availability of suppliers?
   	No alternate source avaialble?
   	Unclear procurement strategy?
   9	Compliance/ Regulatory
   	EPA compliance required?
   	Design for Environment (DFE) compliance?
   	Are the solid/liquid/gaseous residue amounts high?
   	Any regulatory standards to be followed?
   	Any additional approvals needed?
   	Hazardous material involved?
   	Significant exposure/contamination potential?
   	High amounts of non-biodegradable waste?
   10	Other/Miscellaneous
   	Hazardous work conditions?
   	Direct hire/subcontractor complxities?
   	Any special work authorization procedures to be followed?
   	Does adverse weather conditions cause delay?
   	Quality issues?

   functions, Organizational, Environmental, Regulatory, and Financial.

   Table 1 lists the risks from many different areas of a project. The technological risks include uncertainties that are caused by new technology and/or low understanding of the technology. Design risks are the reliability, testability issues. There are issues which arise due to complex designs. Market risks arise when the product is new to the market and there is low awareness. Also the time-to-market is an important parameter to watch out for. Project risks include schedule, cost and scope risks. The checklist presented in Table 1 touches all the above areas and many more.

   After the risks are identified, they need to be assessed for prioritizing. The knowledge gap measurement is used while assessing the R&D project risks. Knowledge gap index is explained in the next section.

4. KNOWLEDGE GAP

   When developing a new product, the bridging over the lack in knowledge and technologies forms an integral part of the project. This Knowledge Gap makes the R&D project difficult to plan. Existing risk management does not take knowledge gap into account. Proper identification and closure of these knowledge gaps can reduce risks. A simple yet effective definition of the knowledge gap is the gap between what we should know to guarantee project success and what we really know at a given point of time [13] [14]. Since, most of the R&D projects are new and do not have any predecessors, the nominal scale for knowledge gap measurement is the most suitable approach. For the measurement of knowledge gap we will use a generalized

   questionnaire. The project manager should perform this survey with his/her team by including the project specific questions. We need to measure the gap between the existing knowledge and the knowledge required for the project to be successful. For each task, the project manager should fill out the required knowledge level. The team members should fill out the existing knowledge level section. Thus, quantifying the knowledge gap can lead us to better estimations and can also minimize the risks.

   TABLE 2: KNOWLEDGE GAP MEASUREMENT

   No	Topics	Prior Knowledge level	Required knowledge level	Knowledge Gap
   1	New technol oggi es us ed
   2	New components
   3	New s uppl y cha i n

   Table 2 shows the example of the topics. The list of topics can be from any area of the project such as procurement, development, management. For each item the knowledge gap can be measured. The application example will discuss this in more detail.

5. RISK ASSESSMENT

   Risk assessment means how much impact a particular event will have on the cost, schedule and other objectives. The probability of the event occurring will also be taken into consideration in assessment. Detectability of the risk events also forms an important measure. If the event is difficult to detect, then even if the impact is low, the event can be treated as high risk. As mentioned in the PMBOK [7], the risk index forms an important part of risk assessment. Normally the risk index is defined as follows:

   Risk index = Probability X Impact X Detection. (1)

   Each of the three dimensions is rated according to a five- point scale. For example, detection is defined as the ability of the project team to discern that the risk event is imminent as shown in Eq. (1). A score of 1 would be given if even a chimpanzee could spot the risk coming. The highest detection score of 5 would be given to events that could only be

   discovered after it is too late. Similar anchored scales would be applied for severity of impact and the probability of the event occurring. The weighting of the risks is then based on their overall score.

   But this paper discusses R&D projects where knowledge gap is also an important factor. Hence, each risk weighed by multiplying the knowledge gap factor. For example: if the knowledge gap is high for one of the risk items, then the factor will be high towards 5, the risk index will also be high. This particular task will be categorized as the high risk item. Hence,

   Risk Index for R&D projects = Probability X Impact X Detection X Knowledge gap (2)

   Each risk event is evaluated using Eq. (2), and then prioritized according to the risk index. Here the knowledge gap is also given the equal importance. After the risks events are assessed the next action is to develop the mitigation plan.

6. AN APPLICATION EXAMPLE

   For the application example we have selected a PCI based system. The Block diagram of the Dual Redundant Ruggedized Computer is shown in the Figure 3. The system can be divided into 3 major assemblies:

   1. Dual CPU Assembly.

   2. Dual Display Assembly.

   3. Keyboard Assembly.

   The Dual CPU Assembly consists of 2 independent CPU units which can be slid in 5U Rack mount housing. The CPU unit has a backplane of 4-slots with Rear I/O connectors on all slots.

   All I/O signals of the boards are routed to the Rear I/O connector J2 defined in the cPCI standard. The connection of these I/O signals from the board to the unit circular connector external world is by means of a piggy-back PCB assembly mounted on the cPCI backplane. Figure 2 depicts the mechanical assembly and wiring structure of the cPCI system.

   All external world connections are by means of MIL-C- 38999 series III connectors. The connectors are mounted on the Rear Plate of the unit. Special consideration has been given in the mechanical design to ensure that the unit withstands severe shock and vibration environments.

   Fig 2. Block Diagram of the cPCI based system as an application example.

   For risk identification the checklist mentioned in Table 1 is used. Also the RBS (Risk Breakdown Structure) is developed from the project plan and WBS (Work Breakdown Structure). This paper uses RiskyProject plug-in for risk evaluation.

   Fig 3. Scatter plots for evaluating the correlation between project and task duration

   Figure 3 shows the scatter plots for the duration of the project and different tasks in the project. The first scatter plot shows the correlation between the overall project duration and the duration of the task Conceptualization of the scheme. If all the scatter plots are evaluated then one can

   The risk register and risk matrix is shown as follows:

   make out that the project duration is more closely related to the duration of tasks such as Design, Procurement and

   System wiring. This means that the project manager needs to concentrate more on the completion of these tasks. The risks which delay these tasks can cause a threat to the duration of overall project. Hence, the project manager should take necessary steps to mitigate or reduce the risks pertaining to these tasks.

7. KNOWLEDGE GAP MEASUREMENT

   The risk register does not have the knowledge gap as an index. Hence for all the risks the knowledge gap is multiplied with the one of indexes. And this product is provided as an input to the tool. For knowledge gap measurement the template from Table 2 is used. In this various risks are noted down and the prior knowledge level and the required knowledge levels are noted after discussing it with team. The difference between these 2 levels is the knowledge gap for that particular risk. [14]

   TABLE III: KNOWLEDGE GAP MEASUREMENT FOR APPLICATION EXAMPLE

   No	Tpics	Prior Knowledge level	Required knowledge level	Knowledge Gap
   1	New technologgies used	0	5	5
   2	Functional requirements	1	4	3
   3	Defining Project scope	2	5	3
   4	New design strategy	1	5	4
   5	New components to be used	1	5	4
   6	Complex design	1	5	4
   7	Porduct awareness	2	5	3
   8	Skills required	3	5	2
   9	Testing strategy	2	5	3
   10	Standards to be used	2	4	2
   11	DFE experience	3	5	2
   12	Cross functional teams	2	4	2
   13	Supplier information	2	5	3

   Fig 4. Risk Register

   Fig 5. Risk Matrix

   As shown in figure 4 and figure 5, the project has multiple risks. The project has very little chance of finishing as per current schedule. The risk register has listed many risks that increase the cost and delay the project. Hence, if we

   mitigate some of the risks then the project has a better chance of finishing on time. RiskyProject tool gives out a project report where it lists the project risks. Figure 6 shows the risk report for the application example.

   Fig 6. Project risk report

   Some major risks are listed in this report. While developing the mitigation plan, project manager can concentrate on these risks. After the risk mitigation plan, the project summary can be re-evaluated for pending risks. As can be seen here, the new technology, unclear functional requirements, unclear project scope and unclear procurement strategy are the major risks for this application example. Filling the knowledge gap and establishing the right communication channels can be the possible mitigation plans. The next section discusses the mitigation plans and their impact on the project.

8. MITIGATION PLANS

   Risks are identified and assessed for the application. Looking at the risk register following mitigation/response

   plans are developed as shown in Figure 7. The first mitigation plan: Provisions in the design covers almost 12 risks. It includes many action plans such as future provisions for changes, researching the new technology before starting the design, including regulatory compliance, DFX. This is the risk avoidance technique by improved execution. The response plan Provide trainings falls in the category of Risk acceptance and management category. This can be applied to various risks such as Skillset of staff is not sufficient. There are certain risk categories such as

   Attrition of resource for which there are no mitigation plan, these risks also fall under acceptance category. The other mitigation plans cover the rest of the risks. The risk matrix after mitigation is as shown in figure 8.

   Fig 7. Mitigation plans for application example

   Fig 8. Risk matrix after mitigation plans

   As shown in Figure 8, Monte Carlo simulation depicts reduced risks after the mitigation plans are executed.

   All the risks are shifted from the red zone to green zone; this means the mitigation plans have reduced impact, probability or the knowledge gap. The knowledge gap reduction is possible by training also over the time as team learns more about technology the risk of knowledge gap reduces further.

9. SUMMARY AND FUTURE WORK

In an effort to improve risk management for R&D projects the new methodology is developed. There is an inherent need for the development and implementation of robust risk management frameworks in order for Research and Development projects to be successful. This methodology applied to projects can help quantify and manage risks effectively. This is a simple methodology where the traditional risk assessment method is enhanced to suit R&D projects. Proposed risk method expands the concept of the two-dimensional Risk Score, which is calculated from the probability of risk occurrence and the perceived impact of risk occurrence, by adding a detection

score to the risk analysis. In this methodology, a new index

knowledge gap is incorporated to the existing risk score. The knowledge gap is measured and quantified for each risk and the risk can be prioritized on this newly formed risk index. This achieves improved risk prioritization. The method was tested on an R&D project a cPCI based system. The identification and quantification of a risk, as well as its impact, can have significant effects on the success of a development project. Project managers can implement the proposed method to improve project success by focusing on key risks, which in turn can influence the success of a company in an increasingly competitive marketplace.

The report discusses quantifying the knowledge gap. In future, both qualitative and quantitative approach can be combined to evaluate the knowledge gaps. The knowledge gap index is calculated and it is manually incorporated in the risk index calculation of the RiskyProject plug-in. In future this plug-in can be modified to include the knowledge gap measurement index column. This is a third party tool; hence it was not possible to do this modification in the present scope of the report.

REFERENCES

1. David c. Logan. (2009). Known knowns, known unknowns, unknown unknowns and the propagation of scientific enquiry. Oxford Journals, v 60, pp 712-714. Retrieved from http://jxb.oxfordjournals.org

2. Wadlow D. (1998-2007). The role of risk in the design, evaluation and management of corporate R&D project portfolios for new products.

   Retrieved from http://sensors-research.com/articles/risk_ppr.htm

3. Mehata S. (2011, Mar 30). Enhanced R&D Risk Management [Web log post]. Retrieved from http://rdmanagement.blogspot.com/2010/11/enhanced-r-risk- management.html

4. Kaplan R., & Mikes A. (2012, June). Managing Risks: A New Framework. Harvard Business review. Retrieved from http://hbr.org/2012/06/managing-risks-a-new-framework/ar/1

5. The MITRE Institute, September 1, 2007, MITRE Systems Engineering (SE) Competency Model, Version 1, pp. 10, 40-41. Retrieved from

   http://www.mitre.org/sites/default/files/publications/10_0678_presentat ion.pdf

6. Ashley D., Diekmann, J., and Molenaar, K. (2006). Guide to risk assessment and allocation for highway construction management. FHA, International Technology Scanning Program. Office of international program, FHA, Washington, DC, 4-17. Retrieved from http://international.fhwa.dot.gov/riskassess/index.cfm

7. Project Management Institute, A Guide to the Project Management Body of Knowledge, (PMBOK Guide), Fourth Edition, ANSI/PMI 99- 001-2008, pp. 273-312

8. Wang,J., (2010). A performance-oriented risk management framework for innovative R&D projects. Technovation. doi:10.1016/j.technovation.2010.07.003

9. Turnbull J., Andrews D., Bennett L., Bennett P., & Buchanan D. (2001). Common methodologies for Risk assessment and management. Royal academy of engineering working group. Retrieved from http://www.raeng.org.uk/pubications

10. Huchzermeier A., Loch C. (2001, January) Project Management Under Risk: Using the Real Options Approach to Evaluate Flexibility in R&D. Management Science, v.47. Retrieved from http://www.insead.edu/facultyresearch/research/doc.cfm?did=1031

11. Crama P., DeReyck B., Degraeve Z., & Leus R. (2005, January). Managing Technology Risk in R&D Project Planning: Optimal Timing and Parallelization of R&D Activities (January). Retrieved from http://dx.doi.org/10.2139/ssrn.873584

12. Barret P. (2003). Management of Scientific Research and Development Projects in Commonwealth Agencies. Commonwealth of Australia 2003. Retrieved from www.anao.gov.au/

13. Hari, M and Weiss, A. (2003). Analysis of Risk and Time to Market during the Conceptual Design of New Systems, Proceeding of the international conference on engineering design, ICED 03 Stockholm August 19-21, 2003

14. Regevy S., Shtubz A., & Ben-Haim Y. (2006). Managing Project Risks As Knowledge Gaps. Project Management Journal, vol. 37, issue #5, pp.17-25.

15. RiskyProject Professional 6 User Guide (2013). Intaver Institute. Retrieved from

    http://www.intaver.com/downloads/RiskyProjectUserGuide.pdf

16. Mastroianni S. (2011). Risk Management among Research and Development projects. Theses and Dissertations. Retrieved from http://preserve.lehigh.edu/cgi/viewcontent.cgi?article=2299&context=e td

17. RossGK Consulting, (2013, March 12). Planning an R&D Project Eight Risks to Success [Web log post]. Retrieved from http://rossgk.ca/blog/planning-a-project