Cobit 4.1: A Maturity Level Framework for Measurement of Information System Performance (Case Study: Academic Bureau at Universitas Respati Yogyakarta)

Cobit 4.1: A Maturity Level Framework for Measurement of Information System Performance (Case Study: Academic Bureau at Universitas Respati Yogyakarta)

Herison Surbakti

Universitas Respati Yogyakarta Indonesia

Abstract – Universitas Respati Yogyakarta requires methods and structured approach in its evaluation, especially in academic departments. This should to be done to assess the fit between institutional objectives with management that has been applied. COBIT is a framework used to assess, measure and control the performance of institutions in the management of IS/IT. COBIT is also accepted and harmonized by its users, because the framework is built from the goal, rules and institutional policy where all processes are analyzed by looking at the alignment between the objectives to be achieved by the procedures/policies implemented by the institution. In this study the author uses COBIT (Control Objectives for Information and Related Technology) version

1. on the domain Planning Organization (PO). The results of the study conducted performance measurements are made in the form of academic information system analysis, mapping the level of maturity and recommendation for Universitas Respati Yogyakarta, which is expected to be a management model for other institutions.

   Keywords

   COBIT, measurement of performance, audit, CSF, KGI, KPI

   1. INTRODUCTION

Some companies do not hesitate to invest their share in the field of Information Technology (IT), although the investments make enormous drain on the budget. This is done as an effort to get the convenience and benefits of the use of IT, which is expected to help the performance of the company to conduct a competitive business strategy. However it turns out, the investment made by the outcome often is not generated.

Assessment and evaluation of investments that have been issued for the implementation of IT is proper to be considered. Based on some research explained that the company has begun to realize and start doing performance measurement and evaluation. To make the use of performance measurement and management of IT, then The IT Governance Institute (ITGI) as an institution conducting the IT governance arrangements that have standardized tools or frameworks is widely used in the world including COBIT, ITIL, COSO, ISO, and so on [2],[4],[5],[9].

COBIT is a reference method/framework for measurement and control of information technology. COBIT framework is a standard that is considered the most complete and

thorough as IT audit framework as developed based on the rules/procedures of internal company/institution where COBIT is used, so the time will be measured in accordance with the conditions, rules, procedures and norms that work in the company. COBIT has also been developed on an ongoing basis by professional NGOs auditors spread throughout much of the country, where in each state builts a relationship that can manage these professionals.

Performance measurement in academic information system at the Universitas Respati Yogyakarta that utilizes IT as a means of supporting the process is expected to support the management of the education process. For example at the beginning of the selection for new students, the learning process which is done, lectures supporting components such as the method used, the curriculum and other provisions such as faculty, students, facilities, other facilities until the graduation of students who need to be evaluated in order to produce quality and good service and competitive education.

1. Problems

   1. There has been no measurement of the performance model of academic information system at the Universitas Respati Yogyakarta.

   2. COBIT as a framework used as a reference for measuring the performance of academic information system at the Universitas Respati Yogyakarta.

2. Limitations

Limitation of the problems in this study is:

1. This study refers to the standard of COBIT 4.1 (Control Objectives for informatian and related technology) made by the IT Governance Institute (ITGI) an institution that manage and organize IT governance.

2. COBIT domain used is on Planning Organization (PO).

1.4. Research Objectives

The purpose of this study is:

1. Make a model of the academic information system refers to the COBIT framework according to the characteristics of Universitas Respati Yogyakarta.

2. Gain the insight about the performance of academic information system at the Universitas Respati Yogyakarta.

3. Perform analysis and assess the fit between policies on academic standards at the Universitas Respati Yogyakarta implementation, it is used as a reference for the evaluation of the existing academic system at the Universitas Respati Yogyakarta.

   1. LITERATURE REVIEW

      1. Information Technology Architecture

         The Information Technology Architecture (ITA) is described through a set of views, each from the perspective of a different stakeholder. An individual view captures items meaningful to the stakeholders as elements and their interrelationships expressed in a standard form, the structure of the view, and the views correlation with other views. Typically, groups within the organization possess a mixture of Commercial Off-The-Shelf (COTS), frameworks and custom-developed legacy applications as well as data stored in databases and directories. In such an environment, functionality and data embedded in one application cannot be easily merged with functionality and data provided by another. Such an endeavor usually requires the creation of a yet another separate application. Further, each of these applications relies on a proprietary client for user interaction [10].

         Figure 2.1 Conceptual View of the architecture (EOHHS CTO Organization Information Technology Architecture, Version 2.0, 2007)

      2. Information System

         It conceptually describes the data objects, events, activities, and transactions, which have no meaning or effect directly to the user. Information is data that has been processed in such a way that increases user knowledge [10],[1],[3],[4]. While knowledge is a combination of instincts, ideas, rules, and procedures, that drives the actions or decisions. The relationship between data, information and knowledge can be mapped in Figure 2.2, while the data, information, and knowledge can be described in the form of a pyramid, as shown in Figure 2.3.

         Figure 2.2 Relationship of data, information, and knowledge

         Figure 2.3 Description of the data, information and knowledge

      3. Academic Information System

         To achieve the goal of the learning process that is done, it needs a system to regulate the course of the process of the learning system, from start of new admissions, teaching and learning activities, faculty, students, lecturer's method used, the curriculum, the process of filling up the study card, and the graduation evaluation process. This is what is known as the academic system [4],[9]. Academic System is made with the purpose of regulating the learning process to fit the expected goals, in accordance with the vision and mission of educational institutions that shelter, and also in accordance with the rules instituted university education.

      4. COBIT Framework

      COBIT Framework is an IT governance framework aimed at management, IT service staff, department control, and audit function more critical to the business process owner. COBIT Framework as aframework for the management needs for measurement and control of information technology provides the tools to measure the ability of information technology that will continue to be developed. COBIT framework is created to serve as a reference to exercise control over information technology and

      can ensure confidenciality, integrity and availability of the data [5],[6],[7].

      The results obtained from the IT Governance is used as a reference or management guidelines, one of which is COBIT which has a measuring tool such as maturity models, CSF, KGI and KPI. COBIT has a measurement indicator management of information technology in business processes. The relationship between the measuring tool set is described in Figure 2.4.

      Figure 2.4 Model CSF, KPI dan KGI

   2. RESEARCH METHOD

The research method used in this study:

1. Studying the source literature of COBIT Framework, audit trails, and academic information system.

2. Data collection from internal documents regarding institutions such as vision, mission, goals, IT architecture, organizational structure, master plan development, including IT management policies.

3. General identification of the learning process that is carried out at the Universitas Respati Yogyakarta.

4. Perform an analysis of the weaknesses, strengths, opportunities and challenges of the institution (SWOT Analysis).

5. Direct observation of the academic activities to gain referring to the direct point of the matter.

6. Perform analysis of the academic information system that refers to the COBIT framework using a questionnaire.

7. Conducting interviews to related parties such as bureau of academic chief, chairman of the study program, IT staff and administrative staff, and faculty.

8. Perform data analysis.

3.1 Stages of Performance Measurement of Academic Information System

The research method used is performance measurement system with reference to the academic information system and considering some points as follows:

1. Internal Institute Environment

   1. Vision, Mission, and Institution Goals

   2. Academic Institutions Activity Cycle

   3. Strategy and Scope of Business (Business Scope)

   4. Distinctive Competence

   5. Development Master Plan

   6. Planning factors

   7. Strategic Planning

   8. Architectural Institute of Information Technology

   9. Integrated Academic Information Management System of Institutions

   10. Integrated Systems Students

From the data obtained, and then the researcher performed the steps in the performance measurement system of academic information such as explained figure 3.1.

Figure 3.1 Stages Performance and Measurement of Academic Information System

4. IMPLEMENTATION

1. Performance Measurement of Academic Information System Framework Based on COBIT

   To perform the management and measurement of performance against the academic information system, the framework which is used as a reference or guideline is COBIT Framework which includes the maturity model, CSF, KPI and KGI. In general, IT arrangement in Universitas Respati Yogyakarta aims to drive, ensure, and control the institutions in order to achieve goal, namely to reduce the risk factors and make improvements in all aspects so as to support the institution's performance [4],[8].

   Figure 4.1 describes the framework for the establishment of IT management adopted from COBIT domains directly with Planning Organization (PO) as a focus of study.

   Figure 4.1 IT Institutional Management Framework Based on COBIT's PO domain (ITGI, Management Guidelines, 2008)

2. Performance Measurement Model of Academic Information Systems

   Academic information system procedures and policies in its practice have clear rules, standardized and should be informed. The determination of the performance achievement indicators (strategic objectives) using CSF, KPI and KGI are used to achieve this. Figure 4.2 is a planning model for the management of academic information system at the Universitas Respati Yogyakarta with reference to the COBIT Framework.

   Figure 4.2 Managing and Organizing the design of academic information system

3. Performance measurement process on the domain of Planning and Organizing

The measurement process is expected to manage IT resources and achieve strategic goals and objectives such as effectiveness, efficiency, confidentiality, integrity, availability, compliance and reliability of the academic management information systems, so this stage is the translation of the vision and policy institutions. To make the process of performance measurement at the PO domain, the used of indicator known as KGI (Key Goal Indicators) and KPI (Key Performance Indicators), this both indicators related to the CSF (Critical Success Factors) because CSF was raised from the need for business done by management and if not done then it will hamper the pace of organization.

To map the institutional objectives and business requirements, the process can be done by translating and identifying indicators associated with the domain Planning Organization, such as: Establishing the performance of each process PO, Assign KGI, KPI Assign, Assign CSF, and Maturity models. The measurement process is shown in Figure 4.3.

Figure 4.3 Performance Measurement Process of Academic Information System Management

1. Descriptive Maturity Model and Measurement Techniques

   Descriptive measurement techniques are made by the nominal size to sort objects from the lowest to the highest, these measurements only give the order by rank. Measurements were carried out directly from values that refers to the value of the existing sorting in maturity models as show in table 4.1.

   Level	Name	When
   0	Non-Existent	Management has not recognised the need for a process for defining service levels. Accountabilities and responsibilities for monitoring them are not assigned.
   1	Initial/Ad-Hoc	There is awareness of the need to manage service levels, but the process is informal and reactive. The responsibility and accountability for defining and managing services are not defined. If performance measurements exist, they are qualitative only with imprecisely defined goals. Reporting is informal, infrequent and inconsistent.
   2	Repeatable	There are agreed-upon service levels, but they are informal and not reviewed. Service level reporting is incomplete and may be irrelevant or misleading for customers. Service level reporting is dependent on the skills and initiative of individual managers. A service level co-ordinator is appointed with defined responsibilities, but limited authority. If a process for compliance to SLAs exists, it is voluntary and not enforced.
   3	Defined	Responsibilities are well defined, but with discretionary authority. The SLA development process is in place with checkpoints for reassessing service levels and customer satisfaction. Services and service levels are defined, documented and agreed-upon using a standard process. Service level shortfalls are identified, but procedures on how to resolve shortfalls are informal. There is a clear linkage between expected service level achievement and the funding provided. Service levels are agreed

   Table 4.1 Maturity Model (ITGI, Management Guidelines, 2008)

   		to, but they may not address business needs.
   4	Managed	Service levels are increasingly defined in the system requirements definition phase and incorporated into the design of the application and operational environments. Customer satisfaction is routinely measured and assessed. Performance measures reflect customer needs, rather than IT goals. The measures for assessing service levels are becoming standardised and reflect industry norms. The criteria for defining service levels are based on business criticality and include availability, reliability, performance, growth capacity, user support, continuity planning and security considerations. Root cause analysis is routinely performed when service levels are not met.
   5	Optimised	Service levels are continuously re-evaluated to ensure alignment of IT and business objectives, whilst taking advantage of technology, including the cost-benefit ratio. All service level management processes are subject to continuous improvement.

   This simplifies the process of calculating and mapping the maturity model. These measurements were made to map the position of IS governance maturity model into the existing, so that from the model that will be scalable of the IT management in planning and organizing domain occupies a level determined in accordance with criteria that are owned. From its placement then it can be used to evaluate candidates for the management for further improving and enhancing the performance of the management of the existing IS governance in the institution.

   1. Summary Results of Audit Implementation Design Model

      Recapitulation of this implementation shows all the results of data from questionnaires filled out by respondents from different levels of management. The questionnaires recapitulation use descriptive measurement techniques. This is a standard indicator outcome of determination recap associated with quantitative data expressed in simple calculations such as the total value of the whole, the index, average, and percentage. The answers given by the respondents will have scores that is equivalent to each of its maturity level. The total respondent is 25 and the recapitulation of PO results in table 4.3.

      The respondents calculation summary per PO (Planning Organization) domain by using descriptive calculation formula obtained results as shown in Table 4.3 and 4.4.

      Table 4.3 PO Domain Questionnaire recapitulation

2. CONCLUSION

   From the results obtained it appears that the institution has a different level for each PO domain. In general, institution occupies level 3, which means that the institution has procedures in management, has been communicated and documented for each element in the institution. However, the implementation is still highly dependent on existing human resources willing to perform the procedure or not. That existing procedures are still limited to the formalization of implementation available, and management needs to improve planning and organizing.

   The discrepancy between the technical implementation of management policies with existing will be reduced by the control and measurement refers to guidelines such as COBIT Framework KPI, KGI, CSR and maturity models in COBIT Framework for monitoring ranging from policy-setting; monitoring is being conducted at the time, until the goal is achieved.

   PO	TOTAL	INDEX	%	LEVEL
   PO-1	56	2.24	44.80%	2
   PO-2	46	2.19	61.33%	3
   PO-3	49	2.33	65.33%	3
   PO-4	56	2.67	74.67%	4
   PO-5	44	2.10	58.67%	3
   PO-6	57	2.71	76.00%	4
   PO-7	66	3.14	88.00%	5
   PO-8	59	2.81	78.67%	4
   PO-9	61	2.90	81.33%	4
   PO-10	56	2.67	74.67%	4
   TOTAL	713	33.53	920.80%	36
   MEAN	71.3	3.353	92%	3.6

3. REFERENCES

Table 4.4 Percentage scale at the level of maturity

Based on the results from Table 4.3, for each process in the PO domain, it obtained graphs as in the figure 4.4 below:

Figure 4.4 Measurement graphs in maturity model

1. Alter, Steven, 1992, Information System A Management Perspective, Fourth Edition, Prentice Hall Inc.

2. García, Victoriano Valencia., Vicente, Eugenio J. Fernández, Dr., and Aragonés, Luis Usero, Dr., 2013, Maturity Model for IT Service Outsourcing in Higher Education Institutions, (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 4, No. 10, 2013.

3. Gottschalk, Petter, 2012, Knowledge driven service innovation and management : IT strategies for business alignment and value creation, ISBN: 1-4666-2512-0, 978-1-4666-2512-9, IGI Global.

4. Henderi, 2010, Good IT Governance: Framework and Prototype for Higher Eduation, Creative Communication and Inovative Technology Journal vol 3, no.2 ISSN: 1978-8282.

5. ISACA, 2006, Integrating COBIT into the IT Audit Process (Planning, Scope Development, Practises), IT Governance Institute.

6. IT Governance Institute, 2007, COBIT 4.1, Framework Control Objectives Management Guidelines Maturity Models, IT Governance Institute, ISBN 1-933284-72-2.

7. ISACA, 2006, IT Governance Global Status Report.

8. Law, M. Averill, dan Kelton, David W., Simulation Modelling and Analysis, Second edition, McGraw-Hill., Singapore, 1991.

9. Maria, Evi, 2012, The Measurement Of Information Technology Performance In Indonesian Higher Education Institutions In The Context Of Achieving Institution Business Goals Using COBIT Framework Version 4.1 (Case Study : Satya Wacana Christian University, Salatiga), Journal of Arts, Science & Commerce, E-ISSN 2229-4686, ISSN 2231-4172.

10. Skoczylas, Robert., Sevier, Raoul., Garden, Martin., Snyder, Jason., 2007, Commonwealth of Massachusetts, 2007, EOHHS CTO Organization Information Technology Architecture, Version 2.0.