Interoperability Assessment as Indicator of Innovation Management Effectiveness

Interoperability Assessment as Indicator of Innovation Management Effectiveness

Luís Fernando Sobejeiro Rigoni, José Marcelo Almeida Prado Cestari, Eduardo de Freitas Rocha Loures, Eduardo Alves Portela Santos

Industrial and Systems Engineering Graduate Program Pontifical Catholic University of Parana, Curitiba, Brazil

Abstract – Innovation management implies transversal manifestation of a development cycle within the organization, featuring an information flow and the coordination of processes that involve both human resources and heterogeneous systems. The complexity of this cycle leads to the concept of interoperability, which can be considered as the ability to exchange information and use them. Interoperability assessment may help the identification of the main barriers towards innovation management within an organization. This paper proposes an interoperability assessment model for innovation management in a Research & Development company, characterizing a diagnosis process on its potential interoperation. The assessment method is based on the Analytic Hierarchy Process, organizing the knowledge (innovation attributes) in interoperability perspectives and inferring the maturity level of the organization.

Keywords: Innovation management; Maturity level assessment; Interoperability assessment.

1. INTRODUCTION

   Constant and sustainable innovation promotes business competitiveness and growth, maintaining and improving competitive advantages previously established [1], which represents a strong argument for innovation not being an attribute at higher organizational maturity levels only. Innovation comprises a complete process, starting with the launching of an idea and proceeding to all steps before achieving a marketable product and providing economic gains. There are five types of innovation [2]: (i) those resulting in new products/services; (ii) in new production methods; (iii) in new supply sources; (iv) in the prospection of new markets; and (v) in new ways to organize businesses. An innovative company continuously adopts new ideas through structured processes, not operating just with the sporadic development of a new process or product [2].

   According to [3], strategic innovation supports both product and process innovation, allowing the understanding of the long-term view on the innovation contribution to the company competitiveness and success. Complementary, such concept also comprises modifications in several aspects, including the pricing structures, distribution channels, and value or relationship networks [4]. In [5], the author advocates for relationship networks that maximize the company resources and can be important sources of innovative initiatives, comprehending suppliers and partners. In [6] these networks generate a positive impact on the innovation performance, being located either close or distant geographically, which can be a determinant of cooperation.

   Innovation must not be conceived as something entirely new or as a purely creative process [4], given that knowledge, appropriate organizational structure, processes and competencies previously acquired, strategy, climate, culture, and leadership are elements that foster an innovation-friendly environment [7]. In such a broad and complex corporative environment, the maturity evaluation of the organizations innovation management is made necessary to enable it to sustain, repeat, and accelerate future initiatives, since there is no clear way to lead them [4]. In an economic scenario characterized by great technological evolution and full competition, innovation successfully strengthens the approach to an idea [8] contributing to the competition promotion and maximization of economic wins in the enterprise.

   To promote an innovation-friendly environment, the development of continuous, structured, and manageable processes is necessary [8], thus characterizing an information and coordination flow that occurs transversally within the organization and involving different areas or business units. In this context, the need to assess the existing barriers to collaboration in the organizational structure arises, aiming to ensure the appropriate performance of the innovation management. The characteristics of such corporative environment lead to the concept of interoperability, thus broadening the business assessment dimensions.

   According to the European Interoperability Framework (EIF) [9], interoperability can be defined as the ability to share information and reciprocally make use of the shared information, besides enabling the employment of information technology systems. The EIF identifies three interoperability perspectives: (i) technical interoperability, including the interconnection of information systems via predetermined data exchange standards; (ii) semantic interoperability, focused on ensuring that exchanged data mean the same to all involved parties; and (iii) organizational interoperability, involving the organization of business processes and infrastructure to improve data exchange.

   In a more organizational context, interoperability consists of the ability of two or more business units or enterprises to jointly cooperate for a common objective [10], including all systems used (internal and externally) or affected. To point out elements that may cause problems to organizational interoperability, the assessment model proposed by [11] extends two dimensions of the EIF: barriers (conceptual, technological and organizational) and concerns (business, process, service, and data). From a public administration perspective, the authors in [12] present some attributes characterization for interoperability assessment. Table I

   presents some examples of barriers found within each concern.

   The authors in [13] also exemplify the cooperation barriers bringing two more determinants besides geographic proximity. The first is the higher probability of choose partners that have a similar thematic field of work, ensuring that both parts could exchange and understand the transferred knowledge. Another barrier is previously relations, where the probability of collaboration highly increases when they already had prior collaborative activities.

   With the purpose to evaluate how prepared an enterprise is to cooperate within an organizational environment based on innovation, the present paper brings an assessment and diagnosis model of the potential organizational interoperability maturity. The assessment model is founded on a specific methodological framework, allowing the identification of the attributes that qualify the innovation domain under interoperability perspectives. The obstacles in an absolute evaluation of the fulfillment of such requisites, both regarding the tacit nature of the knowledge involved and the conformity quantitative uncertainty, inspired the use of a multi-criteria assessment method based on AHP [14].

   TABLE I. BARRIERS VS. CONCERNS EXAMPLES, ADAPTED FROM

   [11]

   Conceptual Technical Organizational

   The resulting of the organizational diagnosis comprises the assessment of a business unit in a company whose core is the development of technological products and services, besides having links and interface with the public administration. The relevance of the potential interoperability assessment is characterized by the need for a deeper knowledge of the barriers imposed by the innovation management and its assessment, considering the intrinsic complexity regarding the coordination of the resources involved, information, processes, and strategic orientation. The business units can be assigned to research on quite distinct knowldge areas, inciting different execution processes, specialization, and heterogeneous knowledge bases, constraining the coordination and collaboration and thus affecting the innovation management success within the organization.

   The current paper presents the methodology employed for the selection of assessment criteria based on the structuring and application of questionnaires to experts in innovation. Afterward, the prioritization of these criteria (attributes) is developed, followed by the presentation of the assessment and diagnosis method, considering the conformance to a specific maturity level that best characterizes the fulfillment of attributes. Finally, an application scenario of the model in a technological innovation company and the analysis of results are presented.

2. METHODOLOGY

   As a referential model for identification of the innovation

   Business

   Business models, visions, strategies,

   Infrastructure, technology

   Work methods, business rules, organizational

   attributes, the Innovation Management Maturity Model (IMMM) [15] was chosen. The IMMM contains 69 attributes divided into five maturity levels and grouped according to

   policies structure

   three factors: people, process, and tools. This model was

   Process Processes models

   Service Services models

   Tools supporting processes modeling and execution

   Tools supporting services and applications

   Responsibilities, Process management and rules

   Responsibilities, service and application

   selected since its assessment factors are also adherent to the interoperability concerns (business, process, services, and data). Such aspect facilitates the establishment of capability levels of the innovation attributes under relative maturity assessment in AHP structure detailed right after.

   management and rules

   The methodology design is represented in Fig. 1 and it is

   Data

   Data models, (semantic, syntax)

   Data storage and exchange devices

   Responsibilities, data management and

   based on the IDEF0 notation, helping to model the strategy at the highest level of inputs, controls, outputs, and mechanisms.

   rules.

   The priority assessment in the AHP method is based on the mathematical analysis of a set of pairwise comparisons, in which, the priority scales between each two related terms are relying on the professional judgments of domain experts. These human judgments may not always be consistent, and the AHP method also provides a theory of consistency evaluation to assist experts in improving their judgments and obtaining better consistency. For the purpose of this research, the AHP hierarchically organizes the attributes identified to analytically determine the enterprise conformance to a specific maturity level. Also, the use of the AHP is appropriate, given its relative assessment base and thus the difficulty to collect evidence and absolute assessment of innovation attributes. The quantitative assessment, even from a qualitative perspective, allows considering the uncertainty degree of the interviewees through the calculation of answer consistency and global assessment validity.

   Boxes represent functions (i.e. processes, operations,

   activities) and horizontal arrows indicate Inputs (entering the boxes on the left) and "Outputs" (leaving the boxes on the right), in this case representing the transformation and evolution of the information (products of the research) through the process. Controls (arrows entering from the top) represent aspects that constrain or govern the function (e.g. methods and tools) and Mechanisms (arrows entering from the bottom) represent the resources that perform the function (e.g. people, software, database).

   At the prioritization stage of the interoperability criteria (A1), 19 attributes were selected from the IMMM for the design of two surveys, as listed in Table II. The distribution and application of the absolute and relative surveys were online-based processes A1 and A2 in Fig. 1, respectively. The absolute survey (A1, Fig. 1) is characterized as a setup stage of the interoperability assessment. In this stage, respondents are asked to prioritize, according to each analysis factor of the innovation management, the criteria pertinent to interoperability assessment as well. As a result of the first survey, the three most relevant criteria for each factor are obtained in an absolute way.

   Fig. 1. Methodological diagram.

   From such prioritization, the second survey (relative) was designed and applied, corresponding to the assessment of compatibility with the criteria (process A2, Fig. 1). In this stage, the survey is divided into two parts: first, the respondent is asked to evaluate the relative importance (with pairwise comparisons) of each factors criteria; second, the perception of the evidence regarding the conformance and fulfillment of each criterion is evaluated, in the context of the enterprise for which the professional works. Table III presents the maturity levels designed for the definition of the fulfillment level of each attribute.

   The design of the relative survey was founded on the structural requisites of the AHP method [14], both in the prioritization of the most relevant criteria and the assessment of the fulfillment level in each maturity level. Based on the survey, the AHP method leads the respondent to comparatively judge (pairwise comparison) all criteria and the relative importance degree among them [16]. Additionally, by adopting such process, the respondent uncertainty is verified and analyzed through the consistency degree obtained. In some cases, the value of the consistency degree can indicate a demand for a review of the comparisons performed.

   TABLE II. IMMM CRITERIA LIST

   TABLE III. DESCRIPTION OF THE MATURITY LEVELS ACCORDING TO THE FULFILLMENT OF THE ASSESSMENT CRITERIA

   People Process Tools

   Maturity Level

   Description of the fulfillment perception level of the criteria

   Executive leadership fosters innovation

   Center for Process Excellence and Innovation is well- established and reports to executive team

   Decision making is collaborative and efficient

   Everyone throughout the commercialization process understands role

   Project managers and scrum masters lead innovation and development teams leveraging best practices

   Fully automated and standardized processes that are easily adaptable

   Portfolio metrics evolved to include competitive and environmental impact scores

   Projects killed early and often during portfolio reviews

   Voice of the customer captured on an ongoing basis

   Continuous learning loop well established

   Process covers idea to launch, and through to end of life

   Highly functioning Product Portfolio Management (PPM) system integrated to other enterprise tools

   Ideation collected for collaboration

   Product roadmap tied to project execution and corporate strategy via PPM

   Entire product catalog of in-market products,

   including the Product P&L, managed via PPM

   Self-service configurable reports and metrics delivered across the organization

   Level 1 There is none or only informal evidence (conversations, motivated individuals to take action)

   Level 2 There are formal evidence (strategy, vision,

   documents, meetings)

   Level 3 There are some/partial results indicating the search

   for fulfilling the criteria

   Level 4 There are obvious results generating income for the

   company

   Fig. 2 shows the AHP structure modeled as a decision tree, in which the goal, in this case, is the potential diagnosis of interoperability in the innovation management, pointing out possible maturity levels (Lowest Level). A potential approach implies the ranking of the attributes to create ranges that are related to some maturity level. That is, according to the ranking of the group of attributes of an entity A, the entity is associated with a certain Maturity Levl, with a potential degree for future interoperability actions (ML-1, ML-2, and ML-N). In the other hand, a compatibility approach would evaluate which entity (GE B…Z) will better interoperate with a specified entity (GE A). The attributes deriving from the process A1 (Fig. 1) are referred to this structure as sub- criteria in the intermediate level (Attribute 1..n), being organized according to the criteria (i.e., people, process, and tools – EI/IMMM criteria) and positioned at the intermediate level as well.

   Fig. 2. An Example of the Decision Hierarchy

   The relative survey provides two results: (i) the pondered prioritization of the attributes that mostly influenced the interoperability assessment; (ii) the fulfillment levels of each criterion. Therefore, the AHP method application features a diagnostic stage in the identification of the potential level of existing interoperability as well as the reflection on influence factors in the innovation management interoperability.

3. APPLICATION SCENARIO

For the evaluation of the assessment model, the Information Technology department of an R&D company formed by a group of laboratories was chosen. The professionals selected to respond the surveys are involved in

the development of software products. The Super Decisions software [17] was used for the application of the AHP. The main usage stages of this software represent: (i) the modeling of the hierarchic decision tree structured in clusters and nodes;

(ii) the identification of the links between the nodes; (iii) the comparisons of importance (pairwise comparison) between nodes with mutual relation to a common node [18]. After the data computing, the software provides the prioritization and informs how much each alternative contributes to the selection of the goal in question. Fig. 3 illustrates the decision tree modeled on the mentioned software.

Fig. 3. Decision tree modeled for diagnosing the maturity level.

In the decision tree modeled, the final cluster 4Alternatives is inserted in the cluster equivalent to the Lowest Level in Fig. 2, representing the maturity levels of the presented model (IMMM). The intermediate level (criteria/sub-criteria layer on Fig. 2) is related to the following four clusters: 2Categories,

PEOPLE_ATTRIBUTES, PROCESS_ATTRIBUTES,

and TOOLS-ATTRIBUTES. In these clusters, the acronyms (PeA1, PeA2, …, PrA1, … TA1) correspond to the attributes identified in Table IV, besides being related to the items selected in the A1 process of Fig. 1 and adherent to the grouping of the IMMM (people, process, and tools).

TABLE IV. PRIORITY OF THE CRITERIA INDICATED AS THE MOST RELEVANT IN THE INTEROPERABILITY ASSESSMENT.

People Process Tools

the interoperability perspectives. Additionally, the authors suggest as a subsequent stage focused on gathering these models in a diagnostic framework to formalize continuous

improvement indicators in business management, aiming at

PeA1 – Executive leadership fosters innovation

PrA1 – Fully automated TA1 – Product roadmap and standardized tied to project execution

processes that are easily and corporate strategy via

the highest maturity level, and thus supporting the innovation management processes and, consequently, the maximization of the business.

PeA2 – Decision making is collaborative and efficient

PeA3 – Everyone throughout the

adaptable

PrA2 – Process covers idea to launch, and through to end of life

PrA3 – Portfolio metrics evolved to

PPM

TA2 – Ideation collected for collaboration

TA3 Highly functioning PPM system

V. REFERENCES

1. G. Moore. (2005). Dealing with Darwin: How Great Companies Innovate at Every Phase of Their Evolution. London: Penguin Books.

   commercialization process include competitive and

   integrated to other

2. W. J. Bigoness and W. D. Perreault. (1981). A conceptual

   understands role

   environmental impact scores

   enterprise tools

   paradigm and approach for the study of innovators. Academy of Management Journal, 24, 68-82. Retrieved from http://www.jstor.org/stable/255824

   Table IV presents the attributes selected in the setup stage (first survey) and ordered according to the importance indicated by the respondents. To obtain the values resulting from all respondents to each question (pairwise assessment) a calculation sheet proposed by [19] was employed, sustaining an aggregation based on the geometric average of the evaluations performed. After the application of the second survey and the calculation of results average, values were inserted in the model of the Super Decisions software, and the results are demonstrated in Table V.

   In the scenario under analysis, the third maturity level was the best characterized by the set of most relevant attributes that qualify the domain, according to the relative perception of the respondents. A significant part of the attributes shows weaknesses in maturity (level 1), but their smaller weigh relevance at hierarchical structure of the AHP did not affect the overall assessment of maturity level 3.

   TABLE V. RESULTS OBTAINED WITH THE SUPER DECISIONS SOFTWARE.

   Alternatives Priorities

   Level 1 30%

   Level 2 7%

   Level 3 39%

   Level 4 24%

   IV. CONCLUSIONS

   Given the fact that innovation plays a key role in business competitiveness, enhancing the organization presence in the market, it is important to identify the existing cooperation barriers in the organizational structure. The identification of such barriers is enabled by the interoperability perspectives, which preconize the performance potential in innovation management. Through the proposition of an interoperability assessment model based on the AHP model, this paper demonstrated the relation between the potential of an enterprise to interoperate and its appropriate environment to innovation management (both for private and public related entities). In the case of public administration entities, for example, it is possible to identify different (and complementary) types of barriers or concerns related to interoperability, also deriving from the lack of rationale related to attributes extraction. As a suggestion for future works, the research points out the application of the proposed methodology based on different innovation maturity models presented in literature to identify the ones that best adhere to

3. K. A. Baker. (2002). Management Benchmark Study, Chapter 14 – Innovation. Retrieved from http://www.au.af.mil/au/awc/awcgate/doe/benchmark

4. D. Esterhuizen, C. Schutte and A. du Toit. (2011). Enhancing Innovation Capability Maturity Through Knowledge Conversion. Acta Commercii, pp. 211-231. Retrieved from http://www.actacommercii.co.za/index.php/acta/article/viewFil e/162/162

5. G. Hamel. (2000). Leading the Revolution. Boston: Harvard Business School Press.

6. Z. L. He and P. K. Wong. (2012). Reaching Out and Reaching: Within A Study of the Relationship between Innovation Collaboration and Innovation Performance. Industry and Innovation, 539-561.

   http://dx.doi.org/10.1080/13662716.2012.726804

7. H. E. Essmann. (2009). Towards Innovation Capability Maturity. Stellenbosch: Ph.D. thesis. Retrieved from http://hdl.handle.net/10019.1/1308

8. F. O. Scherer and M. S. Carlomagno. (2009). Gestão da Inovação na Prática: Como aplicar Conceitos e Ferramentas para Alavancar a Inovação. [in portuguese]. São Paulo: Atlas.

9. European Commission. (2004). European Interoperability Framework. Retrieved from http://ec.europa.eu/

10. S. Koussouris, F. Lampathaki, S. Mouzakitis, Y. Charalabidis and J. Psarras. (2011). Digging into The Real-Life Enterprise Interoperability Areas Definition and Overview of the Main Research Areas. International Symposium on Collaborative Enterprises. Orlando. Retrieved from http://www.iiis.org/CDs2011/CD2011SCI/CENT_2011/Paper Pdf/ZB589UA.pdf

11. W. Guedria. (2012). A Contribution to Enterprise Interoperability Maturity Assessment. University of Bordeaux 1: Doctoral thesis.

12. J. M. A. P Cestari, E. R. Loures, E. A. P. Santos, Y. Liao, H. Panetto and M. Lezoche. (2014). An Overview of Attributes Characterization for Interoperability Assessment from the Public Administration Perspective. In: Meersman et al. (Org.). Lecture Notes in Computer Science. 1ed.: Springer Berlin Heidelberg, 2014, v. 8842, p. 329-338. DOI: 10.1007/978-3- 662-45550-0_33.

13. M. Paier and T. Scherngell. (2011). Determinants of Collaboration in European R&D Networks: Empirical Evidence from a Discrete Choice Model. Industry & Innovation, 89-104.

    http://dx.doi.org/10.1080/13662716.2010.528935

14. T. L. Saaty. (1977). A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology, 15, 234-281. http://doi.org/10.1016/0022-2496(77)90033-5

15. Planview. (2013). A New Framework for Assessing Your Innovation Program: Introducing the Innovation Management Maturity Model. Retrieved from http://go.planview.com/innovation-management-maturity- model.html

16. T. L. Saaty. (1994). How to make a decision: the Analytic Hierarchy Process. Interfaces, 24, 19-43. Retrieved from http://www.jstor.org/stable/25061950

17. B. Adams and E. Rokou. (2016). Super Decisions. Creative Decision Foundation. Retrieved from http://www.superdecisions.com/

18. A. Zutshia, A. Grilo and R. Jardim-Goncalves. (2012). The Business Interoperability Quotient Measurement Model. Computers in Industry, 5, 389404.

    http://dx.doi.org/10.1016/j.compind.2012.01.002

19. K. D. Goepel. (2013). Implementing the Analytic Hierarchy Process as a Standard Method for Multi-Criteria Decision Making in Corporate Enterprises A New AHP Excel Template with Multiple Inputs. Proceedings of the International Symposium on the Analytic Hierarchy Process. Retrieved from http://bpmsg.com/wordpress/wp- content/uploads/2013/06/ISAHP_2013-13.03.13.Goepel.pdf