Towards New Success Factors in Technology Intelligence Evidence from a European Benchmarking Study

Towards New Success Factors in Technology Intelligence Evidence from a European Benchmarking Study

Günther Schuh

Lab. for Machine Tools and Production Engineering (WZL), RWTH Aachen University

Aachen, Germany

Felix Lau, Patrick Kabasci, Hedi Bachmann Fraunhofer-Institute for Production Technology IPT, Aachen, Germany

Abstract As a reaction to the fast changing, complex business environments of today, many technology-driven firms rely on technology intelligence an integrated process of searching, assessing and disseminating relevant information and insights to decision makers within an organization. Despite this, many firms fail at assessing the relevant trends of their businesses appropriately. This paper addresses the underlying problem and examines antecedents of successful technology forecasting. Using the consortium benchmarking method and questionnaire data from more than 200 European firms, three generic success factors of technology intelligence are derived and discussed with regard to the existing literature. The three generic success factors are: systematization, understood as the establishment of transparent, goal-oriented processes, rules and activities within technology intelligence; bindingness, understood as the degree to which deliveries of technology intelligence are being incorporated in the firms strategic decisions rather than being purely interest-driven and participation, understood as the company-wide involvement and inclusion of employees outside formal intelligence units in the technology intelligence process. The authors elaborate hypotheses on how implementing the three success factors can lead to better results in technology intelligence. Subsequently, conclusions are drawn and directions for further research are outlined.

Keywords Technology management; technology intelligence; technological forecasting; scanning; innovation management; consortium benchmarking

1. INTRODUCTION

   Firms operate in a fast-changing, complex business environment today. For technology-driven firms, profound changes may arise not only from technological advancements, but also from changes in markets, regulation, economy or society. These changes may occur on a short term base, or start long-lasting developments and while some trends can be influenced by a given firm, others are determined by exogenous factors. However its nature, the omnipresence of business-affecting change appears unavoidable in competitive markets. Given this environment, decisions in technology management must be based on a comprehensive, reliable information base and proactive discussions of future trends. In order to identify potential risks and opportunities at the right time, the process of technology intelligence relies on scanning the technological environment for existing and upcoming developments. The integrated approach of technology intelligence (TI) encompasses the process of searching as well as assessing and disseminating relevant, edited information [1].

   Different studies have proven that companies currently fail to react to drastic technological changes appropriately [2]. As one of the main reasons for the limited learning ability of well- established companies, Lichtenthaler identifies a lack of awareness for technological trends. Besides generating such awareness to improve currently used technologies and to develop further business fields, companies aim to identify technological discontinuity and global changes [3].

   TI helps firms reacting, adapting and evolving in an ever changing technological environment. However, successfully designing, implementing and developing such a technology intelligence system is where many companies fail. The determination of success factors of TI and the impact of those factors has not been focused in past research. Instead, research has concentrated on specific aspects of TI such as methods, organization or tools. This paper addresses the gap and aims at explaining the drivers of success in TI.

   Following our consortium benchmarking study, we draw on a comprehensive set of data based on questionnaires specifically designed for the identification of success factors of TI. Our analysis of the data shows that certain revelations of TI use are highly likely to coincide with higher success in TI. The statistical analysis uses spearmans rho coefficients as a measure of correlation. These results are then validated with an analysis of the existing literature as well as our conducted case studies.

2. EXISTING LITERATURE

   Roots of the forecast development can be attributed to Ansoffs proclaim that environmental changes are heralded by vague precursors called weak signals in 1975 [4]. But it was the 1990s, when companies (usually large companies) began to develop significant in-house capacity for corporate forecasting [5]. One part of corporate forecasting is technology intelligence. Technology intelligence can be defined as the process of gathering, analyzing and communicating of relevant technological information. The aim of this process is to provide an information basis for decision- making to use chances and avoid risks imposed by changes in the (technological) environment [1], [6].

   According to Lichtenthaler, TI entails the systematic and continuous observation and evaluation of technological trends as a core process part of technology management [2]. The goal is a timely allocation of relevant information on technological trends in the business environment, to identify potential

   opportunities and threats [7], [2]. Three basic activities of TI are used in practice and described in academic literature: scanning, monitoring and scouting. While scanning aims at the assessment of weak signals that point at any kind of technological innovation or change in the future [8], monitoring focuses on selected identified fields and analyses them profoundly [9]. Lastly, scouting provides detailed information about specific technologies [10].

   As a result of a growing competitive pressure, growing technological dynamics, the merger of different technological fields and a high financial burden on innovation activities, technology intelligence becomes more important to companies [11]. Research on technology intelligence has existed for several years, but the available literature on the organization of TI is limited. While much of the current research focusses on forecasting methods, fewer studies consider the broader question of what makes organizations successful in technology intelligence.

   The few studies that address this topic are restrained by several limitations. For example Bürgel et al. find success drivers such as commitment from top-management or high motivation of employees. But their study is limited to multinational companies and the findings remain rather generic [3].

   Other studies such as Frießems concentrate on technology intelligence in corporate networks. The author argues that companies should gather in networks to be successful in company networks. However her research is limited to success factors concerning networking. Success factors inside companies are left out [12].

   Lichtenthaler states in his study that the success of technology intelligence mainly depends on the ability of a company to learn. If technology intelligence cannot be brought into wide parts of the company technology intelligence will not be successful. To achieve this, the author determines that technology intelligence should not be included in rigid structures, but be rather informal, structured or hybrid [13]. [13]In another paper he concludes that good handling of the technology intelligece process depends on decision making and company culture [7].

   An empirical study on search patterns of low- and high- technology companies is conducted by Grimpke and Sofka. As results they state low-technology firms usually concentrate on market knowledge and client focusing, while high technology companies focus on technological knowledge as well as research [14].

   While the studies presented aim to explain when technology intelligence is successful, none of them use extensive empirical data with a wide range of companies of different size, industry and region in combination with the benchmarking method. In doing this, the presented paper gains insight from both quantitative empirical data as well as qualitative case studies.

3. METHODOLOGY

   1. Study aim and design

      1. The consortium benchmarking method

         Consortium benchmarking is an academia-practitioner collaborative study approach in management research. The method was developed to address the often stated gap between academic research (which is primarily valid but not necessarily relevant) and industry practice (which is relevant but not necessarily valid). Successful academic-practitioner collaboration can overcome this gap when the collaboration produces rigorous knowledge useful for both groups [15].

         The consortium benchmarking method is aimed at descriptively assessing the current state of the art and best practices in a given field of management. Our study was done in accordance to the general methodology as outlined by Anderes and Friedli [16], with slight modifications as described below.

         Aim of the consortium benchmarking method is finding previously unknown success factors in a defined management function as they appear in practice in industry. The method does not attempt to explain these success factors empirically; neither does it test whether a success factor is a precursor of successful management or a result of it. Instead, it should be considered an explorative approach designed to derive new and probable hypotheses.

         The first step of a consortium benchmarking project, and a focus of this paper, is an empirical survey of the state of the industry in a specific management discipline. The procedure is the following. A consortium of practitioners of the selected management discipline states questions arising in practice on aspects of that management discipline such as processes, organization, and interfaces to other related disciplines. These questions and fields of interests are gathered in a moderated workshop using structured creativity techniques.

         A team of researchers in the selected management discipline then defines known success criteria derived from project experience, literature and results of previous consortium benchmarkings in related fields. Furthermore, open questions on success factors in the management field are collected by the researcher team. The researcher team then mirrors the collected concepts, questions and items along a framework of the management discipline to discover remaining white spots and ensure the survey broadly encompasses the field. The survey itself is then constructed from the results. Question types include both perception of general success of the responder as well as factual questions regarding how intensive certain concepts are used or whether certain design options for the considered management function are used at all.

      2. Research Questions and Survey Design

         Using the benchmarking method and a comprehensive set of empirical data on the use of TI, the overall aim of this paper is to identify success factors in TI. The following research questions were used as a guiding framework:

         1. Which structural organization of TI serves a companys information needs best?

         2. Which processes and tools lead to successful TI?

         3. How is a successful technology assessment realized?

         4. Which strategy and which cultural traits of an organization will lead to best TI-results?

            Contrarily to the method stated by Anderes, we do not selectively send the survey to a limited number of invited participants, but aim for a large spread of potential respondents. This is for two reasons. Firstly, with an increasing number of respondents, the statistical quality of the generated data will improve. Secondly, previous studies have found that TI practices vary greatly between firms. [2] This means that excluding certain industries or types of firms from the scope of research will imply the risk of missing important successful practices. As the general goal of our study was to find new successful practices, a broad scope of participating firms was preferred.

   2. Application of the Method

      In our case, the consortium comprised of 15 German companies or German subsidiaries of multinational companies sending representatives from their technology intelligence, technology management or related departments.

      After an initial meeting with the consortium the quantitative screening phase began. In this phase, the questionnaire was designed and sent out. The survey was sent out by postal mail and email to circa 10.000 companies; furthermore, it was possible to fill the form online. The questionnaire was divided into four sections according to Table I. The first section of the questionnaire deals with the respondents organization and lays the basis for the general descriptive statistics. The questions concern the size and industry of the organization and the structure of the respondents business unit. The remaining four sections deal with TI specifically. Aspects of technology intelligence considered were organization of TI; process of TI; technology assessment in the context of TI; and strategic & cultural aspects as well as controlling of TI. The latter two aspects concerned interfaces and overlaps to related management disciplines (technology assessment, technology and corporate strategy, and strategic controlling). Concepts were mirrored along an accepted framework of technology intelligence (Fig. 1) [17]. The elements of this framework along with the research questions defined the topics that are addressed in the questionnaire.

      TABLE I

      LAYOUT OF THE STUDY SURVEY

      Fig. 1. Framework of technology intelligence

      Where technology intelligence functions are decentralized, we were interested in one response per business unit conducting these activities, plus potentially a central response for central R&D. Alternatively, we were interested in one response per company.

   3. Further steps in the consortium benchmarking

   After the screening phase, we proceeded with the study as per the standard consortium benchmarking method [16].

   Candidates for successful practices were selected based on the survey evaluation (the underlying success criteria are described in the next section), follow-up telephone case studies, and evaluation of the pseudonymized telephone case studies by the consortium. After selection, the consortium and research team visited the candidates to conduct one-day on site case studies where the candidate firm presented selected aspects of their technology intelligence and was challenged by the consortium and research team. The consortium then decided on considering the candidate a successful practice. Results of the case study visits to those companies which were deemed successful practice are used in the discussion section of this paper. Of the 207 participating firms, 10 case studies were derived and 5 firms were assessed and awarded as

   »successful practices«; compare Fig. 2.

   	Survey section	#Questions
   (1)	Respondents organization & business unit in general	13
   (2)	Organization of technology intelligence	9
   (3)	Process of technology intelligence	16
   (4)	Technology assessment in the context of technology intelligence	7
   (5)	Strategic & cultural aspects as well as controlling of technology intelligence	10

   Fig. 2: Procedure for the identification of best practices

4. RESPONSE ANALYSIS

   1. Data Processing

      Data from returning surveys has been cleaned where answers were ambiguous; ambiguously answered questions or items were deemed not answered. Incomplete surveys (considered as surveys where whole sections have been left unanswered) were discarded; surveys were not discarded if only individual questions or items have not been answered, as

      this is permissible in the method. On receiving duplicate survey responses (e.g. online and via mail), the survey with the latest date of submission was used, and the others discarded. Furthermore, on questions about the intensity of use of a range of options (e.g., for methods used for a certain activity), a large share of respondents only replied regarding the use of those options they actually employed, skipping to mark those they did not use. The intensities ranged from not at all to to a great extent on a four-point scale. Thus, for these questions, if one item has been marked at intensity above not at all, the non-answered items were cleaned to not at all; this has been marked in the results where applicable. Due to these required data cleanings, our study can only provide positive information on existing influences, but cannot show absence of an influence. The formula (described in section

      »Success criteria«) for scoring the ex-ante success of each respondent has then been introduced as a factor.

      To explore new success factors, we are interested in items significantly having a correlation with ex-ante success, and on the direction of that correlation. As we cannot explain causality or composition of success using our methodology, we are not interested in the magnitude of the correlation, but only its significance. Thus we conducted a correlation analysis of the ex-ante success with each item in the survey except for those items describing the general organization and its parameters.

      For the calculation the correlations we used spearmans rho coefficients. In comparison with pearsons r, another widely used measure for the dependence of variables, this method demands less strict requirements with respect to the scales of measure and the nature of dependence (linear) between the variables. Since a linear dependence of the given variables can be assumed but not proven, spearmans rho is the preferred method. However, a second analysis using pearsons r coefficients yielded highly similar data and justifies the same general results in the discussion segment.

   2. Controlling the results: Study size and design

      We have received 207 responses to our study. Not every respondent has completed every question (which is permissible in the survey, as not all questions apply to every organization). Thus, we cannot set a fixed threshold for a correlation coefficient to sieve out significantly non-zero correlations. Instead, we calculated statistical significance for each correlation separately.

      In the results section, we will list all significant correlations at 1% level or less. Due to the nature of the study with 250 items for which a correlation analysis has been conducted, it is important to consider the probability that at a certain significance level, one or more correlations falsely deemed significant would result from the whole study. Given the number of items of this study, we find the probabilities shown in Table II. As such, for discussing findings we require at least one correlation significant to at least the 0,1% level for a conjecture, while only reporting on less significant levels as a hypothesis of influence.

      TABLE II

      GLOBAL ERROR PROBABILITIES

      Significance level Probability of at least one random

      correlation falsely considered significant 1% 91.8%

      0.5% 71.4%

      0.1% 22.1%

      0.05% 11.8%

      0.01% 2.5%

   3. Success criteria

   In order to explore new success factors, one first needs to determine how successful technology intelligence can be measured according to current knowledge. As described earlier in this paper, literature can only deliver limited answers as to what the success factors of TI are. Therefore, we used expert interviews as an addition to the literature analysis. The success criteria were derived as a result of expert interviews with both practitioners and academic experts. The interviewees had several years of experience in the field of technology intelligence. Three of the interviewed experts each had both practical and academic experience of more than ten years in the field. Additionally, five experts with 2-3 years of respective experience were involved in the discussion and study design. As an institution dedicated to applied research in close cooperation with the industry, we used our strong background of industry projects in the field of technology intelligence to assure the topicality of the used success criteria. The used categories (aspects) for the construction of ex-ante success listed in Table III. Each of the five aspects of success has been weighted equally; the individual items contributing to an aspect of success were assigned point values. The detailed values can be seen in Appendix I.

   TABLE III

   OVERVIEW OF EX-ANTE SUCCESS CRITERIA EMPLOYED IN THE

   STUDY

   General perception of success of technology intelligence Organization of technology intelligence

   Process of technology intelligence

   Technology assessment context of technology intelligence Strategic & cultural aspects as well as controlling of technology

   intelligence

   Unless noted in Appendix I, success criteria were evaluated on a four-point scale of not at all to to a great extent. Some success criteria within one aspect are only attainable if another success criteria in that aspect is present; these instances are detailed in the discussion.

5. RESULTS

   In this section the results of our empirical analysis are being presented while implications are being discussed thereafter. The results are parted into two categories. Firstly, general statistics are considered in order to shed light on the TI practices in firms.

   1. General statistics

      207 companies took part in the consortium benchmarking study. As Fig. 3 shows, the companies (or their participating business units respectively) differ in their size and industry,

      thus yielding a representative sample for technology driven firms in Europe. The number of employees range from under 250 up to several 10.000 while most companies (26%) fall within the interval of 1000 to 5000 employees. The average in number of employees is 10491, its median is 1488. Circa 29% of the surveyed firms generated revenue between 100 and 500 million euros in 2012. Thereby the arithmetic mean was 4.72 billion euros and the median 268.8 million euros. The given sample of respondents can be characterized as technology driven firms of different industries. The largest part, with 24% of the surveyed companies, belongs to the machinery and capital goods industry, the second largest (13%) is automotive engineering followed by chemical industry (8%). Conglomerates account for 14% of the surveyed firms.

      Fig. 3: Descriptive statistics of the benchmarking study

   2. Correlation analysis

   The correlations show the spearmans rho coefficients of the given items correlated with our constructed success factor as described in the section above. We ordered the results with respect to their significance level. Caveats are marked where applicable. The results yielded 28 item correlations with a significance level of p < 0.01%. Leaving those items out that were part of the constructed success factor or logically linked to parts ofit (and therefore not applicable to our discussion),

   20 remain. These 20 items will be the primary object of discussion in the subsequent section. Similarly, five applicable items with a significance level of p < 0.05% and three items with p < 0.1% apply.

6. DISCUSSION

   Whereas technology intelligence is always dependent on a firms specific characteristics, we argue that, independent from these characteristics, certain approaches or success factors enhance TI performance significantly.

   It should be stressed that our primary aim of research was the identification of generic success factors. In this context, generic particularly means industry-, company-size- and country-independent. While our data entails a wide range of different companies, we search for antecedents of successful technology intelligence practice that are valid, independent from these characteristics. Particularly, in analyzing the correlation results, we were looking for generic patterns that reinforce previously published findings based on case studies from the same consortium benchmarking project[18].

   Analyzing the results of the correlation analysis, we regarded the most significant correlations in a first step. The list of items significantly correlated with success can be parted into three categories:

   1. Systematization, i.e. the establishment of transparent, goal-oriented processes, rules and activities within TI.

   2. Bindingness, i.e. the degree to which TI deliveries are being incorporated in the firms strategic decisions rather than being purely interest-driven.

   3. Participation, i.e. the company-wide involvement and inclusion of employees outside formal TI units in the TI process.

   In the following subsection we will operationalize and discuss each of the categories as candidates for generic success factors of TI.

   1. Systematization as a Success Factorof TI

      Technology intelligence tasks can be organized in a multitude of ways. Studies show that companies organize their technology intelligence activities very differently depending on their overall company structure, culture, strategy and other aspects [19]. In a similar vein, processes can be organized variously depending on the characteristics of a given firm. Independent from the actual realization of these factors, our results suggest that the systematization of TI leads to higher success. Here, systematization of TI is understood as the establishment of transparent, goal-oriented processes, rules and activities. A systematic TI approach means topics of research, responsibilities, roles and goals of the activities are well defined rather than decided on an ad-hoc basis.

      As shown in Table IV, four questionnaire-items are considered as proxies for systematization, dealing with organizational establishment of TI, explicitly defined processes, systematic processes for megatrend analyses and defined processes for technology assessment.

      TABLE IV

      PROXY-ITEMS FOR SYSTEMIZATION AND THEIR CORRELATION WITH EX-ANTE SUCCESS

      Item # Item name Item question / answer

      of organizing TI efficiently is particularly difficult because, dealing with the future, forecasting is always subject to uncertainty. Moreover, the value of generated information is hard to measure, making effective controlling very difficult, as will be discussed later. Allocating resources on endeavors with uncertain and difficult-to-measure outputs is what makes efficiency challenging.

      TI activities can be classified according to their level of determination. Different definitions exist, but a widely excepted classification distinguishes between scanning,

      15.1 No explicit

      organizational est. of TI**

      23.1 Explicitly defined TI process**

      27.1 Syst. Process for megatrend

      analysis**

      40.1 Explicitly defined

      process for technology assessment**

      How are TI activities organized in your

      company? / -no explicit organizational establishment, but treatment by persons with responsibility for certain topics.

      Does your company / operational division have an explicitly defined technology intelligence process? If so, is this process integrated in other corporate processes? / no

      Does your company / operational division have a systematic process for conducting megatrend analysis? / no

      Does your company / operational division have an explicitly defined process to assess technologies in the context of technology intelligence? / yes

      monitoring and scouting. In this logic, scanning refers to the unfocussed search for relevant information [20]. It follows the idea that weak signals from the companys broad environment point to changes before they happen. While scanning is aimed at the entire outside surrounding, monitoring and scouting can be considered a directed search in selected technological fields (monitoring) or one specific technology (scouting) [2], [20]. In our case studies, we find that an overemphasis on undirected search (scanning) may lead to waste of resources since these activities are highly time-consuming and an upscaling of these activities comes with diminishing returns. However, a refrain from scanning may endanger the companys capability to assess trends in the broader context that they should be seen in. Also, the risk of missing relevant developments because the corresponding signals have not been picked up, increases. We assume that a

      * Significant non-zero correlations with p <0.1 %

      **Significant non-zero correlations with p <0.01 %

      Our results show that these measures of systematization correlate positively (and in one case the opposite measure is correlated negatively) with success in TI, compare Table IV. These are notably the existence of explicitly defined processes for a) TI in general, b) megatrend analyses and c) the assessment of technologies. Our results suggest that TI performs better if it follows predefined rules, guidelines and standards of practice. For example a systematic assessment of technologies entails that a firm uses standardized criteria and processes to determine the value or outlook of a given technology. These processes define who conducts the technology assessment and which criteria are used. This makes the assessment both transparent and more objective.

      Furthermore, the absence of an explicit organizational establishment of TI within firms is negatively correlated with success. This may have various causes. One possible explanation is that an explicit organizational establishment functions as a proxy for the firms commitment towards TI (under the assumption that committed companies perform these tasks better). Our assumed explanation however, is the notion that technology intelligence is most effective when tasks are clearly assigned and roles and responsibilities defined, i.e. when the TI activities are systematized. This line of argumentation is coherent with our previous findings based on case studies on the same benchmarking project. There, we found that a guiding framework, in the sense of a strategic alignment of TI, helps companies focusing on the right activities and that this is quintessential for effective and efficient forecasting [18].

      Systematization also entails allocating time and resources optimally, i.e. so that they produce most output. The challenge

      20/ 80% split is optimal. These are the average proportions that the Good Practice firms of our benchmarking study chose, compare Fig. 4. The Good Practices are the top 30 companies out of the 207 participants ranked according to their score on our ex-ante success variable.

      Fig. 4: Resource allocation on scanning, monitoring and scouting

      In our quantitative questionnaire analysis we find a slight negative correlation of success and scanning (however, only on a p < 5 % level), a significant positive correlation between success and monitoring and none when scouting is considered. These correlations cannot serve as evidence for our case study finding, but they may trigger a discussion on how resources should ideally be allocated on the different types of search approaches. We believe the correlations support our thesis that too much scanning leads to investigating (and thus excessively allocating resources to) topics that are considered interesting for the company but that may not be business-relevant.

      Lastly, systematic TI lays a basis for the effective use of software tools because these require precisely defined configurations. Software tools can improve both output quality of TI results and the cost efficiency of achieving these results. As recent literature shows, TI can significantly be improved

      with software tools including (but not limited to): patent analysis [21], semantic web crawlers [22], mobile applications for decision makers [23], data mining [24] and open innovation platforms [25].

      TABLE V

      PROXY-ITEMS FOR BINDINGNESS AND THEIR CORRELATION WITH EX-ANTE SUCCESS

      Item # Item name Item question / answer

   2. Bindingness as a Success Factor of TI

      One problem that many companies conducting technology intelligence face is that, despite accurate information being generated, this insight is not used to trigger innovation but remains largely unused. With the term bindingness we describe the degree to which TI deliveries are actually being incorporated in the firms strategic decisions and innovation efforts instead of remaining information on paper. Binding TI units are equipped with responsibility and their performance is measured and controlled. In the literature there is a wide consensus that generated information should reach decision makers and trigger innovation. However, limited research has been focused on how this may be achieved. In a case study of European firms, Lichtentaler concludes that TI should be strongly integrated in the companys planning and decision making process [13]. In a similar vein, Rohrbeck postulates in his empirical study that TI should play an initiating role within the company and finds that inertia inhibit especially large organizations from initiating adaption to change [10].

      One way to overcome given inertia is by delegating decision-making authority to TI units. We find support for this line of argumentation in our correlation results. More broadly speaking, we find support for our assumption that bindingness

      22.5 Decision

      making authority*

      37.1 TI triggers concrete development projects**

      1. No explicit evaluation of TI*

      2. Evaluation of TI based on strategic goals**

      3. Evaluation of TI based on quantitative key performance indicators*

      To what extend does the organizational

      form of your technology intelligence fulfil the following criteria? / decision-making authority

      On average, what share of newly identified technologies is subsequently followed-up in your company / operational division? / xx% of newly identified technologies on average become the subject of concrete development projects or support these projects.

      In what way is the result of your technology intelligence activities evaluated? / Not explicitly (everything is all right, as long as no trends are being missed and new technologies are continuously identified)

      In what way is the result of your technology intelligence activities evaluated? / On the basis of strategic targets (e.g. identification of substitution technologies for specific applications)

      In what way is the result of your technology intelligence activities evaluated? / On the basis of quantitative key performance indicators (e.g. number of located patents, economic benefit)

      increases TI performance. As outlined in Table V, the proxy- items for bindingness (decision-making authority, TI as a trigger for development projects, TI evaluation based strategic goals and quantitative key performance indicators) correlate with TI success. For example, decision-making authority positively and significantly correlates with ex-ante success. We argue that this is because an involvement in practical decisions improves information generating. For this, two main reasons can be stated. Firstly an involvement in actual decision-processes ensures that the information gathering is goal-oriented rather than purely interest-driven. In our consulting projects on the topic of TI, we regularly observe companies complaining that their TI units live in their own world, meaning that their research topics do not reflect the organizations information needs. Often, this is because chosen research topics overemphasize novelty and innovation and underemphasize feasibility. As a second probable reason for why TI units with decision-authority on average generate better information, is their increased accountability. If TI units can be held accountable for their performance, because they are responsible for actual decisions, an effective controlling becomes possible. This in turn lays the basis for optimizing performance. Accordingly, we find negative correlations for ex-ante success and the absence of TI evaluation (also compare Appendix II). Thus, firms that do not even try to evaluate their TI units explicitly show lower performance.

      * Significant non-zero correlations with p <0.1 %

      **Significant non-zero correlations with p <0.01 %

      Controlling TI is challenging for several reasons. Foremost, the quality of TI outputs is difficult to measure. If the generated information leads to better decisions and this is to serve as a proxy for information quality of TI the quality of those decisions typically only reveals after a long period of time. Moreover, decisions are not solely based on the information that TI delivers but have many other influencing factors. Thus, retrospective decision-quality can only limitedly serve as an indicator for information quality. Effective and efficient controlling requires performance measures that are (among other properties) valid, reliable and practical [26]. As Lönnqvist and Pirrtimäki point out, such measures are particularly challenging when the outputs of an activity are insights and knowledge. However, in their study, the authors demonstrate that Business Intelligence processes can effectively be measured and propose a combination of direct (direct assessment of intelligence results) and indirect (assessment of the utilization of intelligence) measures [27].

      Despite the outlined obstacles, our insights from practitioners suggest that TI can become subject to effective controlling. For instance, performance can be measured by the number of located patents, initiated research projects or through a qualitative assessment by a superior. Table VI lists further performance measures that the respondents of our benchmarking study indicated they used.

      TABLE VI

      PERFORMANCE MEASURES AS STATED BY RESPONDENTS

      Number of patents Share of new products Cost effectiveness

      Return on investment (ROI) Increase of EBIT

      Number of Awards New Customers

      Number of innovation projects Customer surveys

      Number of research projects

      A problem that may arise with controlling TI is that this may reduce creativity and that forecasting may be altered negatively if performance indicators are used. Authors in favor of this line of argumentation claim that measuring output will induce an incentive towards gathering a myriad of (measurable) data instead of (difficult-to-measure) future- oriented insights. Following this notion, Bürgel et al. find that Japanese firms consider controlling in TI conflicting with performance [3]. It is therefore reasonable to be cautious when applying controlling measures to TI and a mix of both quantitative and qualitative criteria might be a feasible solution. In our analysis we find a significant positive correlation between ex-ante success and the use of quantitative measures for the evaluation of TI. However we cannot find a similar correlation for the use of qualitative assessments. We therefore can neither reinforce nor mitigate th assumption that a combination of qualitative and quantitative measures for the evaluation of TI is advisable. Nonetheless, literature on the controlling of business intelligence proposes that a combination of objective and subjective indicators should be used [27]. Similarly, Mueller and Coppoolse demonstrate in an experimental study that the information quality in business intelligence can be increased remarkably using incentive systems [28]. Since studies on the controlling of TI are scarce, further research is needed to understand which measures are best suited to assess TI-performance and at the same time avoid adverse incentivizing.

   3. Participation as a Success factor of TI

   In this context, participation describes the strong, company- wide involvement and inclusion of employees outside formal TI units in the TI process. There are many ways in which a company can have its workforce participate in the TI tasks. Examples are the tasks of information gathering, technology assessment and result dissemination. Our findings show that higher participation antecedes increased success in TI.

   Participation is represented in our study through nine questionnaire-items, shown in Table VII. These include questions dealing with whether employees work on a full time or part time basis on TI, the use of own employees as information source, the use of open innovation platforms and internal events.

   Table VII lists items dealing with participation or closely related issues; all of them correlate positively with success. As we will argue, participation improves TI because it enables a company to harness the knowledge and experience of many rather than only that of a limited circle of analysts. Furthermore, knowledge sharing a crucial element for the creation and dissemination of insights is supported.

   In the following subsections, two concepts that are closely linked to participation, will be discussed: Open Innovation and Organizational Learning.

   1. Open Innovation

      In many firms, the research and development process evolves from closed innovation to open innovation. Also labelled institutional openness, external technology sources are increasingly used in the research and development processes [29]. This has to say that, besides within the R&D units, further individuals are encouraged to participate in the innovation process whether they are from the same company (bottom up innovation) or from outside. Bottom up innovation systems rely on a shared IT platform that enables sharing, discussion and assessment of ideas. Contributors can participate by either proposing their ideas or by commenting and assessing other peoples ideas. Having access to the organizations knowledge increases the innovative capacity. In our case study of Enel, an Italian utility company and one of the awarded successful practice companies, it was found that bottom up innovation is particularly helpful in the ideation process, i.e. the fuzzy front-end of innovation. In this stage, ideas are generated and assessed.

      Open innovation platforms are used to incorporate external expertise into an organization and have been discussed extensively in recent literature. As Chesbrough and Rogers show in their analysis, open innovation has become an established paradigm that increasingly becomes a standard in R&D [30]. Open innovation provides for an efficient way for companies to delegate product and/or technological development work. In this fashion, open innovation taps on the ideas of many to generate new ideas and problem solutions, thus boosting radical innovation. Hereby, the problem of the local search bias is overcome [31]. For example, Dogson et al. show in their case study of Procter & Gamble how open innovation increases the innovation effectiveness [32]. It is reasonable to assume that open innovation improves not only the innovation process in particular but the generation of information, knowledge and insights for the given company generally; thus its TI results.

      Coherently, we find significant, positive correlations for the usage of open innovation and ex-ante success in TI (item 29.1, compare Table VII). Our findings are in concordance with literature. For example Veugelers et al. conclude that the inclusion of open innovation in TI processes improves the results [33].

      TABLE VII

      PROXY-ITEMS FOR PARTICIPATION AND THEIR CORRELATION WITH EX-ANTE SUCCESS

      Item # Item name Item question / answer

      lack adequate abilities to adapt appropriately to the recognized trends. Organizational learning can be defined as an organizations capabilities to maintain and improve

      performance based on experience and knowledge acquisition,

      1. Full time employees for TI*

      2. Part time employees for TI**

      28.1 Own employees as information source**

      29.1 Use of Open Innovation platforms*

      1. Personal communicatio n by networks**

      2. Use of IT- based communicatio n platforms**

      3. Internal events **

      41.2 Board of internal technology experts for technological assessment**

      41.4 IT-based communities for tech. assessment**

      How many employees in your company are engaged in TI (full time)

      How many employees in your company are engaged in TI (part time)

      Which of the following information sources do you us successfully while searching for new technologies? / our own employees

      In your company / operational division, which IT-supported tools are used to acquire information?

      Which channels are used intensively within your company / operational unit to communicate information gathered by technology intelligence? / Personal communication by networks

      Which channels are used intensively within your company / operational unit to communicate information gathered by technology intelligence? / IT-based communication platforms (e.g. IBM- connections, »internal Facebook«)

      Which channels are used intensively within your company / operational unit to communicate information gathered by technology intelligence? / Internal events (e.g. developer days)

      In your company / operational division, who assesses technological opportunities in the context of technology intelligence? / Board of internal technology experts

      In your company / operational division, who assesses technological opportunities in the context of technology intelligence? / (IT-based) communities (company-wide)

      sharing and utilization [35]. TI can enable firms to master environmental changes by creating a continuous learning process. As research shows, this learning process is most effective if not only top management is involved but all decision makers in the firm [36]. One way to reach a great number of decision makers is to assign employees on a part time basis beside their regular function (e.g. product development, sales etc.) Asking for the number of assigned technology forecasters, our analysis of the questionnaires shows that success increases with the number of committed personnel. But the strongest correlations were not found in the number of full-time assignees but between part-time assignees and ex-ante success (Table VII). Assigning employees part time on TI tasks makes it possible to integrate a larger number of experts in the TI-process, hence enforcing knowledge exchange. The same argument can be applied to items 33.2 (recording the usage of personal communication networks),

      33.3 (recording the usage of IT-based communication platforms) and 33.4 (recording whether internal events are used to communicate intelligence results). The positive correlations of these items indicate that an inclusive approach to intelligence dissemination leads to better overall TI- performance due to increased knowledge exchange.

7. CONCLUSIONS AND FURTHER RESEARCH

   Based on empirical evidence from a European benchmarking study, this paper sheds light on how technology intelligence is practiced in technology drivenfirms and proposes underlying success factors. Firstly, descriptive statistics based on questionnaire-items are used to show how TI is organized, which processes and methods are being used, how technologies are being assessed and on which strategies TI relies. Secondly, ex-ante formulated success factors are correlated with the questionnaire-items in order to derive new success factors as directions for further research. The findings are then discussed in the light of both practical viewpoints and a literature analysis.

   * Significant non-zero correlations with p <0.1 %

   **Significant non-zero correlations with p <0.01 %

   1. Organizational Learning

   Participation is presumably important for success in TI also for another reason. Besides the classical viewpoint of TI being a process of gathering, assessing and disseminating insights, TI can be considered a learning process. This notion is outlined by several authors but to date, to the knowledge of the authors of this article, there has not been demonstrated empirical proof. Gerybadze follows this line of argumentation and argues TI should be a process of organizational intelligence in which hardly the written results (e.g. reports) but mostly the process of reaching them counts [34]. In this learning process, the firm learns how changes affect its business and how to react to this. Organizational learning capabilities are crucial since many firms do not fail because they oversee important trends, but they do so because they

   The following three generic success drivers were derived and should be validated in further research:

   1. Systematization, i.e. the establishment of transparent, goal-oriented processes, rules and activities within TI.

      It is assumed that systematization increases efficiency since it focusses research activities on those with the highest expected returns.

   2. Bindingness, i.e. the degree to which TI deliveries are being incorporated in the firms strategic decisions rather than being purely interest-driven.

      We assume that bindingness improves TI by establishing accountability which leads to better results.

   3. Participation, i.e. the company-wide involvement and inclusion of employees outside formal TI units in the TI process.

   We assume that participation improves TI through facilitating organizational learning.

   The correlations show that these factors coincide with successful practices in TI. Additionally, the analysis of given literature showed that these correlations can be reinforced from a theoretical viewpoint. Finally, case studies could, where applicable, further validate our results. We therefore consider systematization, bindingness and participation likely success drivers for technology intelligence.

   While the given paper identifies and validates these success factors, a causal relationship cannot be proven in the scope of this investigation. Instead, the causal relationship of these success factors should be validated with the use of studies using explanatory designs. Moreover, further research should focus on how the determined success factors interact. Using an explorative design, this study provides for several directions of research that the authors recommend for further research in technology intelligence. Firstly, future research should aim at explaining how systematization might enhance the success of TI. Our assumption is that, through systematization, TI- activities will be focused on those with the highest expected returns. Faced with an increasingly unmanageable information load, this approach is probably an effective strategy for companies to initially harvest the low hanging fruits. A second thesis to be further studied, is the notion that bindingness leads to more successful technology intelligence. We assume that this is true because it makes effective controlling possible. To date, the controlling of TI-activities has hardly been subject of research. What is more, for practitioners, TI-controlling imposes a great challenge. Understanding how effective controlling can be achieved is of high practical relevance. Lastly, participation presents a promising field of research. Our assumption is that organizational learning is facilitated through participation. Similarly, it can be argued that participation taps on the wisdom of crowds. Further research should thus focus on how forecasting-knowledge can most effectively be generated in large groups.

8. APPENDIX APPENDIX I

LIST OF EX-ANTE SUCCESS CRITERIA EMPLOYED IN THE STUDY

General perception of success of technology intelligence

1. Assessment: »We managed to achieve our intended technology- strategic position (leader or follower) for several generations of technologies!« (max. 1P)

2. Assessment: »We always succeeded in dealing with the radical technology leaps in our industry!« (max. 1P)

3. Assessment: »Our technology intelligence« (max. 1P)

   • is a competent support for the order-related search for specific technologies (max. 0.2 P)

   • is able to keep an eye on all relevant technological areas (max. 0.2 P)

   • is proactively delivering new ideas to our company/ operational unit (max. 0.2P)

   • delivers information to the relevant person in the right point of time (max. 0.2P)

   • leads continuously to concrete new projects (max. 0.2P)

4. Assessment: What share of the information, which is provided by your technology intelligence, is relevant for your company/ operational division? (five-point scale between none (0P) and >75% (1P) )

5. Assessment: »In the past five years, we identified all new technologies relevant to our business at the right time!« (max. 1P)

6. Assessment: »In our company/ operational division« (max. 1P) technology intelligence contributes significantly to the unique selling

points of our company (max. 1P) Organization of technology intelligence

1. To what extent does the organizational form of your technology intelligence fulfil

   the following criteria? (max. 3P)

   1. Strategic aspects (max. 1P)

      • Linkage to the strategic level (max. 1P)

   2. Customer orientation (max. 1P)

      • Cross-linkage to internal demand carriers (max. 1P)

   3. Efficiency aspects (max. 1P)

      • Long-term knowledge management (max. 0.5P)

      • Resource efficiency (max. 0.5P)

2. Does your company/ operational division provide a budget for technology intelligence activities? (max. 1P)

Yes, we have a defined budget for technology intelligence and . [Amount and trend of budget not relevant for success criteria] (1P) Process of technology intelligence

1. Does your company/ operational division define fields in which to search for new technologies (hereafter denoted as »search fields«)? If so, how? (max. 1P)

   Yes (1P)

2. Do you define guidelines for the different search fields (hereafter denoted as »search requests«)? If so, what kind of guidelines? – Guidelines for search fields (max. 1P)

   • Responsibilities (max. 1/3 P)

   • Goal of search (e.g. search for substitution technologies) (max. 1/3 P)

   • Assessment criteria (e.g. market potential, cost reduction potential) (max. 1/3 P)

3. How do you actively establish your networks for the acquisition of information? (max. 2P)

   1. Part 1 of the question (max. 1P)

      We are currently establishing a network (0.5P)

      We do have an established network and we defined the following elements for it: (1P)

   2. Part 2 of the question, only applicable if We do have an established network [] is selected (max. 1P)

   • Criteria for the choice of partners (max. 1P)

   • How did your company/ operational division successfully identify new technologies in the last five years predominantly?

Four-point-scale between Mainly by chance (0P) and Through systematic

procedures (1P)

Technology assessment in the context of technology intelligence

1. Which criteria are relevant for the assessment of technologies in the context of technology intelligence for you? (max. 1P)

   • Guidelines derived from technological strategy (max. 1P)

2. Please characterize the nature of misjudgements in relation to technology assessment. (max. 3P)

   1. Technology aspects (max. 1P four-point scale between not at all (1P) and to a great extent (0P))

      • Technologies were used which later appeared to be

        »unattractive« (max. 0.5P)

      • Technologies were not used which later appeared to be

        »attractive« (max. 0.5P)

   2. Market aspects (max. 1P four-point scale between not at all (1P) and to a great extent (0P))

      • Customer requirements were misjudged (max. 1/3 P)

      • Potential of exploitation was misjudged (max. 1/3 P)

      • Social/ political developments were misjudged (max. 1/3 P)

   3. Time horizon aspects (max. 1P four-point scale between not at all (1P) and to a great extent (0P))

   • »We were too early« (max. 0.5P)

   • »We were too late« (max. 0.5P)

Assessment: »Our technology assessment guarantees that we pick the right technologies in the technology intelligence!« (max. 1P)

Strategic & cultural aspects as well as controlling of technology intelligence

1. Does your company/ operational division have a defined technology strategy? (max 1P)

   Yes, but not in a written form (0,5P)

   Yes, in a written form (1P)

2. In your company/ operational division, how is the interface between technology strategy and technology intelligence organized? (max. 1P)

   • The guidelines for technology intelligence are deduced from the technology strategy (max 0.5P)

   • Results of the technology intelligence process serve as an information basis for the process of technology strategy (max. 0.5P)

3. In which way are the activities of technology intelligence in your company/ operational unit mainly aligned? (max. 1P)

   • Four-point-scale between Mainly demand-driven search triggered by orders of internal customers (e.g. R&D) (0P) and Mainly self-initiative search by the responsible persons (e.g. scouts) (max. 1P)

4. Assessment: »In our company/ operational division« (max. 2P)

   • sharing of knowledge is self-evident in all units (max. 1P)

the maintenance of personal networks is not only promoted, but also

requested (max. 1P)

APPENDIX II

OVERVIEW OF SIGNIFICANT NON-ZERO CORRELATIONS FOUND WITH EX-ANTE

REFERENCES

SUCCESS [1] M. Wellensiek, G. Schuh, P. A. Hacker, and J. Saxler,

Significance < 0.01%

1. (negative) 15.1 No explicit organizational est. of TI

2. (positive) 17.2 Part time employees for TI

3. (positive) 23.1 Explicitly defined technology intelligence processes and interfaces

4. (positive) 27.1 Systematical process for megatrend analysis (yes/no)

5. (positive) 28.1 Own employees as information source

6. (positive) 28.5 Research institutions / universities as information source

7. (positive) 28.8 Patents as information source

8. (positive) 32.3 Databases as methods and tools

9. (positive) 32.4 Knowledge-management systems as method and tools

10. (positive) 32.6 Technology radars as method and tools

11. (positive) 33.1 Newsletters as communication channels

12. (positive) 33.2 Personal communication by networks as communication channels

13. (positive) 33.3 IT-based technology platforms as communication channels

14. (positive) 33.4 Internal events as communication channels

15. (positive) 37.1 Trigger for concrete development projects

16. (positive) 40.1 Explicitly defined process to assess technology

17. (positive) 41.2 Board of internal technology experts for technological assessment

18. (positive) 41.4 IT-based communities for technological assessment

19. (positive) 43.1 Technology portfolio as preferred method and tool

20. (positive) 51.2 Evaluation of TI based on strategic goals

21. (positive) 22.2 extension of cross-linkage to internal knowledge experts 1)

22. (positive) 25.4 Definition of guidelines for intensity of search 2)

23. (positive) 26.7 Technology intelligence responsible for definition of search fields 2),

24. (positive) 42.7 Risk of technology development 1)

25. (positive) 48.3 Assessment of new technologies based on technology- strategic guidelines 1)

26. (positive) 49.4 Global orientation 1)

27. (positive) 54.2 Technology pioneer 1)

28. (positive) 54.6 Unique selling points 1)

Significance 0.01% but < 0.05%

1. (positive) 17.1 Full time employees for TI

2. (positive) 29.1.1 Use of Open Innovation platforms

3. (positive) 32.1 Semantic search techniques as method and tools

4. (negative) 51.1 No explicit evaluation of TI

5. (positive) 51.3 Evaluation of TI based on quantitative key performance

6. (positive) 25.5 Timeframe of search 2)

7. (positive) 42.Degree of innovation for customers 1)

8. (positive) 42.4 Increased performance 1)

9. (positive) 49.1 Avoidance of technological surprises / identification of new chances 1)

Significance 0.05% but < 0.1%

1. (positive) 22.5 Decision making authority 1)

2. (positive) 25.7 Time frame of the observation 2)

3. (positive) 26.3 (Series-) development 2)

Significance 0.1% but < 0.5%

1. (positive) 18.8 Attraction for technologies

2. (positive) 21.2 Monitoring

3. (positive) 28.9 Consultants / technology providers

4. (positive) 28.7 Publications available for purchase

5. (positive) 29.2.1 Research tools

6. (negative) 39.3 Quantitative key figures / qualitative evaluation

7. (positive) 22.8 Linkage to top-management 1)

8. (positive) 26.8 Strategic units 2)

9. (positive) 48.2 Technology strategy defines explicitly the search fields 1)

Significance 0.5% but < 1%

1. (positive) 14.7.1 Innovation management centralized

2. (positive) 14.8.1 Technology management centralized

3. (positive) 22.7 Short communication paths 1)

1. (positive) 25.8 Communication channels 2)

2. (positive) 26.2 Early/ preliminary development 2)

3. (positive) 26.5 Production 2)

   1. These items contribute directly to the constructed ex-ante success variable

   2. These items were only logically applicable if at least one success criterion was met

Technologiefrüherkennung, [Technology Intelligence] in Technologiemanagement, G. Schuh and S. Klappert, Eds, Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, pp. 89169.

1. E. Lichtenthaler, Managing technology intelligence processes in situations of radical technological change, Technological Forecasting and Social Change, vol. 74, no. 8, pp. 11091136, 2007.

2. H. D. Bürgel, G. Reger, and R. Ackel-Zakour, Technologie- Früherkennung in multinationalen Unternehmen: Ergebnisse einer empirischen Untersuchung, [Technology Intelligence in Multinational Companie: Empirical Research Results] in Technologie-Roadmapping, Berlin/Heidelberg: Springer-Verlag, 2005, pp. 2753.

3. I. Ansoff, Managing strategic surprise by response to weak signals,

   Californian Management Review, no. 18, pp. 2133, 1975.

4. K. Burmeister and B. Schulz-Montag, Corporate Foresight, in Zukunftsforschung und Zukunftsgestaltung, R. Popp and E. Schüll, Eds, Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 277 292.

5. W. Ashton and G. Stacey, Technical intelligence in business: understanding technology threats and opportunities, International Journal of Technology Management, vol. 10, no. 1, pp. 79104, 1995.

6. E. Lichtenthaler, Coordination of Technology Intelligence Processes: A Study in Technology Intensive Multinationals, Technology Analysis & Strategic Management, vol. 16, no. 2, pp. 197221, 2004.

7. C. W. Choo, The Art of Scanning the Environment, Bulletin of the American Society for Information Science, pp. 2124, 1999.

8. J. M. Utterback and J. W. Brown, Monitoring for Technological Opportunities, pp. 515, 1972.

9. R. Rohrbeck, Harnessing a network of experts for competitive advantage: technology scouting in the ICT industry, R&D Management, vol. 40, no. 2, pp. 169180, 2010.

10. G. Reger, Technologie-Früherkennung: Organisation und Prozess, [Technology Intelligence: Organization and Process] in Management von Innovation und Risiko, O. Gassmann and C. Kobe, Eds.: Springer Berlin Heidelberg, 2006, pp. 303329.

11. M. R. Frießem, Multikriterielle, kausalanalytische Betrachtung von Erfolgstreibern technologischer Frühaufklärung in industriellen Unternehmensnetzwerken, [Multi-Criteria and Casual Viewing on Success Factors of Technology Intelligence in Production Networks] Wiesbaden: Springer Fachmedien Wiesbaden, 2014.

12. E. Lichtenthaler, Third generation management of technology intelligence processes, R&D Management, vol. 33, no. 4, pp. 361 375, 2003.

13. C. Grimpe and W. Sofka, Search patterns and absorptive capacity: Low- and high-technology sectors in European countries, Research Policy, vol. 38, no. 3, pp. 495506, 2009.

14. H. Schiele and S. Krummaker, Consortium benchmarking: Collaborative academicpractitioner case study research, Journal of Business Research, vol. 64, no. 10, pp. 11371145, 2011.

15. D. Anderes and T. Friedli, Konsortialbenchmarking, [Consortium Benchmarking] in Benchmarking: Leitfaden für den Vergleich mit den Besten, K. Mertens, Ed. 2nd ed, Düsseldorf: Symposion, 2009, pp. 193206.

16. G. Schuh and S. Klappert, Eds, Technologiemanagement. [Technology Management] Berlin, Heidelberg: Springer Berlin Heidelberg, 2011.

17. G. Schuh, H. Bachmann, K. Apfel, P. Kabasci, and F. Lau, Erfolgreiche Technologiefrüherkennung: Von der Pflicht bis zur Kür, [Successful Technology Intelligence: From Basics to Excellence] ZWF Zeitschrift für wirtschaftlichen Fabrikbetrieb, no. 11, pp. 796800, 2014.

18. E. Lichtenthaler, Organisation der Technology intelligence: Eine empirische Untersuchung der Technologiefrühaufklärung in technologieintensiven Grossunternehmen , [Orgnization of Technology Intelligence: Empirical Research on Technology-Driven Companies] Zürich: Verlag Industrielle Organisation, 2002.

19. P. Savioz, Technology intelligence: Concept design and implementation in technology-based SMEs, Basingstoke: Palgrave Macmillan, 2004.

20. C. Lee, J. Jeon, and Y. Park, Monitoring trends of technological changes based on the dynamic patent lattice: A modified formal

    concept analysis approach, Technological Forecasting and Social Change, vol. 78, no. 4, pp. 690702, 2011.

21. G. Schuh, Andre Brakling, and Katharina Apfel, Identification of requirements for focused crawlers in technology intelligence, Management of Engineering & Technology (PICMET), 2014 Portland International Conference on, pp. 29182923, 2014.

22. D. H. Jeong, J. Kim, M. Hwang, S. Lee, and H. Jung, User-centered Mobile Technology Intelligence System, Proceedings on Advanced Computer Science and Technology (AST), pp. 194199, 2012.

23. S. Chaudhuri, U. Dayal, and V. Narasayya, An overview of business intelligence technology, Commun. ACM, vol. 54, no. 8, p. 88, 2011.

24. A. King and K. R. Lakhani, Using open innovation to identify the best ideas, MIT Sloan Management Review, vol. 55, no. 1, pp. 41 48, 2013.

25. D. Danner, D. Hagemann, D. V. Holt, M. Hager, A. Schankin, S. Wüstenberg, and J. Funke, Measuring Performance in Dynamic Decision Making, Journal of Individual Differences, vol. 32, no. 4, pp. 225233, 2011.

26. A. Lönnqvist and V. Pirttimäki, The Measurement of Business Intelligence, Information Systems Management, vol. 23, no. 1, pp. 3240, 2006.

27. R. M. Mueller and D. Coppoolse, Using Incentive Systems to Increase Information Quality in Business Intelligence: A Quasi- Experiment in the Financial Services Industry, in Using Incentive Systems to Increase Information Quality in Business Intelligence: A Quasi-Experiment in the Financial Services Industry, 2013, pp. 38273836.

28. H. Chesbrough, W. Vanhaverbeke, and J. West, Open Innovation: Researching a New Paradigm. Oxford: OUP Oxford, 2006.

29. H. Chesbrough and M. Bogers, Explicating Open Innovation: Clarifying an Emerging Paradigm for Understanding Innovation: Oxford University Press, 2014.

30. R. Reichwald, F. Piller, and C. Ihl, Interaktive Wertschöpfung: Open Innovation, Individualisierung und neue Formen der Arbeitsteilung, [Interactive value generation: Open innovation, individualization and new forms of division of labour] 2nd ed. Wiesbaden: Gabler Verlag / GWV Fachverlage GmbH, Wiesbaden, 2009.

31. M. Dodgson, D. Gann, and A. Salter, The role of technology in the shift towards open innovation: The case of Procter & Gamble, R&D Management, vol. 36, no. 3, pp. 333346, 2006.

32. M. Veugelers, J. Bury, and S. Viaene, Linking technology intelligence to open innovation, Technological Forecasting and Social Change, vol. 77, no. 2, pp. 335343, 2010.

33. A. Gerybadze, Technology forecasting as a process of organisational intelligence, R & D Management, vol. 24, no. 2, pp. 131140, 1994.

34. V. J. García-Morales, M. M. Jiménez-Barrionuevo, and L. Gutiérrez- Gutiérrez, Transformational leadership influence on organizational performance through organizational learning and innovation, Journal of Business Research, vol. 65, no. 7, pp. 10401050, 2012.

35. C. Billington and R. Davidson, Leveraging Open Innovation Using Intermediary Networks, Prod Oper Manag, vol. 22, no. 6, pp. 1464 1477, 2013.