Evaluation of Quality Management in IT Projects: an Investigation in Public Adminstracao of the State of Tocantins

Evaluation of Quality Management in IT Projects: an Investigation in Public Adminstracao of the State of Tocantins

Napoleão Póvoa Ribeiro Filho

MSc Program in Computational Modeling Systems Federal University of Tocantins

Tocantins, Brazil

Selma Regina Martins Oliveira

MSc Program in Computational Modeling Systems Federal University of Tocantins

Tocantins, Brazil

Abstract – This study aims to make an evaluation of the level of presence of indicators that allow recognizing the achievement of processes for quality management in Information Technology (IT) projects in the state of Tocantins. For this, a quantitative research was conducted with experts in IT project management in public administration. The results achieved are satisfactory for recognizing the degree of concern with quality management. With this study it is not intended to depreciate the work of the managers involved, but to value them and offer another perspective on such matters.

Keywords: Quality management of projects; IT projects management; quality planning; quality assurance; quality control.

1. INTRODUCTION

   Information technology (IT) is becoming more present in the various services and products. In the provision of public services is not different. However, what happens is that not always the government can meet all of the demands from the community. In this scenario, the time, cost and quality is directly related to the successful execution of a project. Many times the immediacy to meet a necessity makes the time to be very short time and not always the investment required is available. Thus, the concern for quality management of projects is not observed with the necessary attention. Later this could impact with higher costs or even redoing the work, because it is not always thought of the integration, or even utilization of already existing products. These are the issues will be discussed in public administration in the state of Tocantins. In order to make such investigation, that is, to check the quality management of projects, the good practices presented in the literature are taken as a parameter. In this study, more specifically, we use the PMBOK (Project Management Body of Knowlege) which is a guide that gathers a set of practices for project management generally accepted with standard [1]. It is organized by the institute PMI (Project Management Institute) and is currently on the 5th edition.

   Inside this "universe" of project management, there is a set of activities that aim to confer qualities both on the project execution and the goal of the same [2]. This area is treated in the literature as the project´s quality management. Such area, like the vast majority of the areas discussed in project management, is internally composed of processes, which are the actions carried out in order to achieve a desired result. According to PMBOK, the processes of the project´s quality

   management include the activities of the performing organization that define the quality´s responsibilities, objectives and policies, so that the project meets the needs that motivated its realization. This article is organized in the following way: coming next, the theoretical background of this work is presented. After that, shows the design of the research. Then, is shown the results and subjacent analyzes. Finally, discussions and limitations are presented.

2. THEORETICAL BACKGROUND

   1. Project Management: what does the literature says?

      According to [3], project management is the application of a collection of tools and techniques to manage the use of diverse resources for the realization of a single, complex task, within the constraints of time, cost and quality. [4] also adds that the project management involves planning, monitoring and controlling. To plan a project includes defining the job specification, determining the amount of work and estimating the necessary resources. [5] defines project management as planning, monitoring and controlling of all aspects of the project and the motivation of all those involved in achieving the project objectives on time, cost, quality and performance as specified. In [3] point of view, project management progressed in order to plan, coordinate and control the complex and diverse activities of modern industrial and commercial projects. For [6], project management is a specialized technical management to plan and control projects under a strong single point of responsibility. One interesting definition is given by [7] when he states that project management is the art and science of converting vision into reality. But the most adequate meaning is shown in [8] where project management is the application of knowledge, skills, tools and techniques to project activities that meet the requirements of the project. So considering these definitions found in the literature, we understand that managing a project seeks to achieve a predetermined goal. And to do so techniques and tools are used during all processes, where they are planned and monitored.

   2. Quality Management in Projects

      When exploring the meaning of quality, the responses may be the most varied. Quality in a broader context and has many meanings, depending on the client type, the value what money means in the project, convenience and even practicability (Jonas, 2012). For [9], the essence of the concept of quality is that the product generated by the process

      of any organization must satisfy the implicit and explicit needs of the recipient. In ISO 9000, quality is defined as the degree to which a group of inherent characteristics fulfills the requirements. A very similar definition may be found in [8]. For [10], the last word on quality is not given by who produces the product, but to whom the product serves. The quality management of the project includes processes required to ensure that the project will satisfy the needs for which it was undertaken [11]. Quality management in projects aims to ensure that the project meets the customer's needs and involves all activities of the project throughout its cycle [8]. For [12], the purpose of quality management of a project is to make sure that the project matches your requirements. For [2], this management includes the processes required to ensure that the project meets the needs for which it was undertaken.

      More broadly, it is widely accepted in literature the view that the performance of a project is based at least in measures of time, budget and quality ([13], [14], [15]). In this view it is implicit that the project delivers a quality product within the time and budget and thus the processes used should be good enough. However, this perspective is the opposite of which is the mark of quality management: continuous improvement [16]. This view also underscores the importance of customer´s perception in determining the quality of a service.

      1. adds that customers do not only evaluate the result of a service, but also consider processes involved in the service delivery. Given the above, it is evident that it is necessary to have a more integrated view to the approaches of quality management in projects, specifically regarding to its processes.

   3. Quality Management in Projects according to the PMBOK

      The PMBOK is a guide that aims to identify the subset of project management knowledge that is generally recognized as good practice. These areas are gathered based on nine knowledge areas which are: project integration management, project scope management, project time management, project cost management, project quality management, project human resource management, project communications management, project risk management and project procurement management. This guide is published by PMI and is in its 5th version. For PMBOK, good practice means that there is general agreement that the correct applying of these abilities, tools and techniques can increase the chances of success of a wide range of different projects. For managing project quality, the PMBOK has three processes that aim ensuring that the project achieves its goals in order to meet the needs for which it was done. This means that quality management seeks to ensure that the products generated have the characteristics of quality for which they were designed. The processes of PMBOK are: quality planning, perform quality assurance and perform quality control. These processes usually occur simultaneously on the project and are also related to other processes in other areas of management. For every process of PMBOK there are inputs, tools and techniques used and outputs generated. In quality management project is worth mentioning that many outputs of the process in quality planning are used as inputs to other processes. Thus, many outputs of the processes of quality assurance and quality control are updates of the outputs of quality planning.

   4. Quality Planning

      The process of quality planning involves identifying the relevant quality standards for the project and determining how to satisfy them. According to PMBOK it is one of the leading processes during execution of the group planning processes and the development of the project management plan and must be done in parallel with other processes of project planning. Thus, the quality should not be checked, but planned and incorporated into the project, and quality planning is the key to the implementation of the quality process since it aims to identify the relevant quality standards for the project and define how to meet them [18].

   5. Quality Assurance

      For PMBOK, quality assurance is the application of planned, systematic quality activities to ensure that the project will employ all necessary means to meet requirements. This process includes determining whether the project activities are consistent with the policies and procedures chosen. Functions for process auditing, correction of deficiencies and continuous improvement activities are part of quality assurance. Such functions may provide a reduction of project costs and ensure consistent performance of the products.

   6. Quality control

   The realization of quality control involves monitoring specific project results to determine if they comply with relevant quality standards and identifying ways to eliminate the causes of unsatisfactory outcomes. Quality control seeks to monitor the results of the project in order to identify if achieved the relevant quality standards and to identify ways to eliminate the causes of unsatisfactory performance. The activities of this process are carried out continuously during execution of the project. Controlling the quality avoids that product may be rejected due to the fact of not being in accordance with the quality criteria established [19]. These processes defined in PMBOK were the basis for determining the criteria relating to what would be discussed in the analysis that is proposed in this work. So, inputs, tools and techniques for each process were considered, as it is better described below.

   1. DESIGN OF THE RESEARCH

      Many times is noticed that the urgent need to meet a certain demand makes the product developed by IT departments have the time as a limiting factor. The integration of this solution with the rest of the system, checking if there are skilled labor for the service or the impact that such a solution may cause the system as a whole often in these cases is not considered in sufficient depth. This may mean that there is outsourcing service, raising the cost and often the time as well. The quality is always the last factor to be checked. The context of this research attempts to find, within specialized working in the public administration of the state of Tocantins, what level of concern they have towards the quality of IT projects. To perform this study quantitative approach was used. To collect data, a questionnaire was developed based on the three processes of PMBOK that deal with quality management in projects: quality planning, perform quality assurance and perform quality control. Considering its inputs, tools, techniques and outputs, a questionnaire was prepared and submitted to IT

      project managers in public administration in the state of Tocantins. This questionnaire contains 28 questions. The first four are designed to identify the profile of the interviewed expert (education and experience). The other 24 questions seek to identify the degree of presence of documents, techniques and procedures described in PMBOK for the quality management of projects. Except for the first four questions, for each question were given five options in order for the expert to identify the degree of presence of that item in his department or agency. The following table presents the alternatives with their values and meanings.

      Value of each item	Meaning
      0	Total absence
      1	Low presence
      2	Moderate presence
      3	High presence
      4	Full presence

      Table 1 – Alternatives questionnaire and their meanings

      As seen in the table above, what is expected of the respondent is to identify the degree of presence of requirements to measure the concern with the quality management of projects. There is a fine line that differentiates the terms "high presence of" and "total presence." The first means that a quality requirement exists and is well used in quality management. "Total presence" can be understood as a requirement of dominated project and in process of improvement. Out of 24 questions on quality management, 16 are related to the process of quality planning, 4 are related to the implementation of quality assurance and the other 4 are related to the implementation of quality control. One realizes that the number of issues related to the quality planning process is much greater than the issues related to other processes. This is due to the fact that many of the planning outputs (such as the project management plan, the organizational process assets) are updated in other processes. Also, quality planning, because of its major importance, features a larger number of inputs and outputs allowing to identify the quality conditions necessary for the project´s product. Since the quality requirements for a project may not be the same for others, it is essential that planning is done by experts and in a detailed way for each expected output of the project. According to [2], the quality plan involves identifying which quality standards are relevant to the project and determining how to satisfy them. Such process is very relevant during the project´s planning and should be done regularly and in parallel with other planning processes.

   2. RESULTS AND SUBJACENT ANALYSIS Based on the questionnaire described in the

      preceding section, it was possible to identify several characteristics in relation to those responsible for the management of the IT projects. Figure 1 illustrates the examination in relation to the degree of education of specialists.

      Figure 1 – Level of education of project managers

      It is noticed that most project managers, after graduating made at least one specialization course. This allows us to state that from the managers of IT projects in the state of Tocantins interviewed, 70% had searched for some extra knowledge besides the one acquired in his degree. However, is not questioned in the survey if the course done after graduation was related to project management. This observation does not diminish the quality of the work or the manager, though. It is expected that the project manager execute their projects on schedule, budget and quality standards established and that meets the customer expectations. Thus, more knowledge and experience, even if not specific in project management, will provide support to achieve the goals established for the project. This becomes evident in [20], where six skills and its associated behaviors have been identified as most important for a project manager in the case of people who are part of the project. The table below shows the identified skills.

      Skill	Application
      Managing emotions	Show an honest understanding for the needs of other people and show an understanding of the motives of others and act accordingly.
      Building trust	Show open concern for others. Accept people for what they are. Empower people more and ask them to take on board more responsibilities.
      Effective communication	Hold off-line communications with others to develop effective relationships such as ad hoc and informal talks. Explore the viewpoints of others before making decisions.
      Motivating others	Tell people that they are talented and skilled. Talking more to people to gain more long-term commitments rather than compliance.
      Influencing others	Selling others that benefit for doing something or doing something differently. Share with others what it feels like to work in a highly successful team so they adopt the behaviours that are associated with success.
      Cultural awareness	Develop, display and apply an awareness of the cultural differences of team members. Show an understanding and knowledge of the values and beliefs of other cultures.

      Table 2 – Most important skills for a project manager

      Another analysis that could be performed is regarding to the time when the professional interviewed plays the role of project manager. Figure 2 allows us to get a better view in this regard.

      Figure 2 – Years of experience as a project manager

      According to the previous figure, the great majority (70%) of respondents have at least 02 years experience as a project manager. Of these, approximately 43% have at least 5 years experience. It is possible to notice that most managers of IT projects in Tocantins act as such for at least two years. In other words, they already have some experience in office. This makes the decisions taken by the manager on a project to be based on their "know-how". Regarding the degree of presence of the factors related to the processes that deal with quality management in projects, according to PMBOK, with the collected data a chart was generated and is shown below.

      Figure 3 – Average rattings by process

      In this Figure is possible to analyze the average obtained from the degree of presence identified by specialists, with each process of quality management. All average values were below 2, when the maximum possible value would be 4. Analyzing such averages is feasible to affirm that the processes used in quality management in projects exist, but in a moderate way in departments and agencies studied. Among these processes, there is a slightly greater presence of quality planning. This can be justified because of the fact that many inputs of this process are reused in other processes. This is

      the reason it is evident the presence of elements that characterize the quality planning. This chart means that when managers start a project there is evidence that they already have an idea of the desired quality measures. Analyzing the average level of presence of factors related only to the processes that deal with quality planning, Figure 4 was generated.

      Figure 4 – Average rattings of the analyzed criteria to quality planning

      It can be seen in this chart that the criterion that had higher average of the experts is the criterion number 5, which deals with the organizational process assets. These assets, which are inputs of the quality planning process, deal with the results of the processes implemented in the organization for carrying out their work as communication requirements of the organization, procedures or guidelines for project closure, procedures for financial control and procedures for controlling the changes. The criterion evaluated with the lowest average was question number 11. In it was discussed about the cost of quality, in other words, all costs incurred during the lifetime of the product. Analyzing the Chart 2 and the considerations made, one realizes that in the government of Tocantins there is a plan to be executed in projects. All criteria investigated in relation to planning, were present. However, only a few showed a higher average than the indication of moderate presence. This implies that quality planning exists, however improvements can be implemented at this stage of the project. The Figure 5 deals with averages on questions related to the process of quality assurance. In this chart, it was perceived that the criterion that had higher average was the one that deals with process analysis. This analysis is a technique that seeks to identify the need for process improvements. This analysis also examines problems and limitations experienced and activities that do not add value identified during the process´ operation. And the criterion which obtained the lowest average was the one that was related to the use of techniques and tools of the quality management plan such as affinity diagrams, interrelationship digraphs, tree diagrams, prioritization matrices or diagrams of networks of activities. In short, it can be stated that trying to ensure the project´s quality is something that occurs in the IT sector in the state of Tocantins. However, like every criteria analyzed showed no more than a moderate presence, one sees here possibilities for a better development of this phase of the project.

      Figure 5 – Average rattings of the evaluations related to quality assurance

      In Figure 5 shows the averages of the questions associated with the quality control process. The criteria related to requests for changes and delivery (questions 1 and 2) were the ones with the highest average assessed by the experts. Both deal with inputs to this process. The approved change requests perform the integrated change control, in other words, it is a document that indicates which changes were approved or not. An approved change request should include modifications such as repairs of defects, revised work methods and revised schedule. The delivery attribute is related to a unique and verifiable product, result or capability that results on a validated delivery requested by the project. As also seen in previous processes, some techniques and tools are used to perform quality control of IT projects in Tocantins. However, based on these averages, it is possible to improve execution of such a process, which would impact directly in the quality control referred.

      Figure 6 – Average rattings related to quality control

      • Process analysis, which follows the steps outlined in process improvement plan to identify the necessary improvements.

        In the following analysis, the experts interviewed were divided into two groups: those who had two to four years of experience and those who had more than four years of experience. Considering the average of all their grades, the result is shown in Figure 7.

        Figure 8 – Average of responses of experts according to their experience in project management

        In analyzing this chart is becoming plausible to state that, as the project manager gets more experience in the area, the greater is his concern with ensuring quality control in IT projects. Regarding the characteristics of the quality planning process, aspects that gets greater difference in the average, considering that more experienced managers have assessed more presence, were those that dealt with meetings (with the project team to develop the plan of quality management), quality management plan (a document that describes how the organization's quality policies are implemented) and the plan of process improvement that details the steps for analyzing processes of project management to identify activities that increase their value. Yet, the biggest difference in the chart is identified when the following attributes related to the quality assurance process were analyzed:

      • Management of quality and control tools, which deals with the use of techniques and tools of the quality management plan.

      • Quality audits, which is the process used to determine whether project activities are in accordance with policies, procedures and processes from the organization and from the project.

      Figure 8 makes clear that the experience of the project manager, analyzing good practices from the literature, is a factor that directly influences its quality. Thus, the higher their knowledge, the greater is the evidence that techniques and tools are being used to ensure the quality of the project in execution. Experience makes the manager worries more about ensuring that the established quality parameters are met in the project.

   3. CONCLUSIONS

The objective of this study was to verify, in IT projects in the state of Tocantins, the presence of requirements which allow detection about the concern with quality management. The requirements considered in this work are in accordance with the procedures described in the guide for the best practices in projects PMBOK. Through this study it was possible to verify the level of applying good quality-related practices IT projects in the state of Tocantins. Thus, it is affirmative to say that the objective of this study was achieved because through it was possible to identify how the quality is worked on projects on the referred state. By analyzing the results, one realizes that a concern does exist with the quality management in IT projects. However, this concern is restrained. There are possibilities for improvements in such management. From the processes of PMBOK, there is a greater concern with quality planning of the project. It was possible to determine that the concern with quality assurance and quality control is directly related to the level of experience of the project´s manager. Evidence of quality management in projects have also been found. However many of the related management practices in question have shown to be restrained in the organs evaluated. This indicates that many of the practices carried out in the projects are the result of years of experience of managers and not always based on good practices presented in the literature. The major limitation of this study was the universe of project managers interviewed. A bigger quantity of such professionals interviewed would allow us to draw a profile of quality management with a much higher level of precision. It was unable to find articles dealing with the high demand for IT services that exist in public administration in Tocantins. Such reasoning would enrich this article. This paper showed the importance of concern for quality projects. Through the management of quality is possible to obtain positive effects on vital factors of the projects such as, for example, cost and schedule. Improving the processes that are running is also something one can do. And most importantly, with such management there is a much greater possibility that the project's product is delivered with quality that satisfies both the client and the project manager. As recommendations for future research, and from a deeper analysis is desired to establish a direct relationship between cost, time and quality in IT projects in the state of Tocantins. It is hoped that this work will bring a more thoughtful look at the issue of quality in projects, especially in the area of information technology. The demands from the public service can best be met if processes are reused and their products present high levels of quality. Positive impacts would appear in the project´s spending and its realization periods can be better

observed.

REFERENCES

1. Jung, J.Y. and Wang, Y.J. (2006), Relationship between total quality management (TQM) and continuous improvement of international project management (CIIPM), Technovation, Vol. 26, pp. 716-722.

2. Chang Y.F., Ishii. H. (2013), Fuzzy Multiple Criteria Decision Making Approach to assess the Project Quality Management in Project, 17th International Conference in Knowledge Based and Intelligent Information and Engineering Systems – KES2013.

3. Lock D. Project Management, 5th ed. Gower, Aldershot, 1994. Oisen, RP, Can project management be designed? Project Management Quarterly, 1971, 2(1), 12-14.

4. Badu, A. J. G., Nalina Suresh, Project management with time, cost and quality considerations, European Journal of Operational Research, Vol. 88, pp 320-327.

5. British Standard in Project Management 6079, ISBN 0 580 25594 8.

6. Burke R. Project Management. John Wiley and Sons, Chichester, 1993.

7. Turner, JR, Editorial: International Project Management Association global qualification, certification and accreditation. International Journal of Project Management, 1996, 14(1), 1-6.

8. PMBOK. A Guide to the Project Management Body of Knowledge (PMBOK® Guide) – Fifth Edition. Pennsylvania: PMI, 2013.

9. Anttila, J., Standardization of quality management and quality assurance: a project viewpoint, International Journal of Project Management, Vol. 10, pp 208-212.

10. Garvin, D.A., 1988, Managing Quality The Strategic and Competitive Edge, New York, The Free Press.

11. Chang, Yao-Feng et al. Fuzzy Multiple Criteria Decision Making Approach to assess the Project Quality Management in Project. 17th International Conference in Knowledge Based and Intelligent Information and Engineering Systems – KES2013, vol. 22, pp 928-036. 2013.

12. Schwalbe K. Information technology project management. 4th ed. Boston: Thomson Course Technology; 2006.

13. Barkley, B. and Saylor, J. Customer-Driven Project Management, McGraw-Hill, New York, NY; 1994.

14. Kerner, H. Project Management: A Systems Approach to Planning, Scheduling and Controlling, Van Nostrand, New York, NY. 1992.

15. Chang, A. S. and Ibbs, C. W. Development of consultant performance measures for design projects. Project Management Journal, pp 39-54. 1998.

16. Robert A. Orwig Linda L. Brennan , (2000),"An integrated view of project and quality management for project # based organizations", International Journal of Quality & Reliability Management, Vol. 17 Iss 4/5 pp. 351 363

17. Gupta, A. and Chen, I. Service quality: implications for management development. Internation Journal of Quality and Reliability Management, Vol. 12, nº 7, pp 28-35. 1995.

18. Globerson, S. and Zwikael, O. The impact of the projetct manager on project management planning processes. Project Management Journal, pp 59-66. 2002.

19. Khoja, S.A. ; Fac. of Comput. Sci., Inst. of Bus. Adm. (IBA), Karachi, Pakistan ; Chowdhary, B.S. ; Dhirani, L.L. ; Kalhoro, Q. Quality control and risk mitigation: A comparison of project management methodologies in practice. Internacional Conference on Education and Management Technology (ICEMT), pp 19-23. 2010.

20. Fisher, Eddie. What practitioners consider to be the skills and behaviours of an effective people project manager. International Journal of Project Management 29, páginas 994-1002. 2011.