Research and Technology Organizations (RTOs) are knowledge-intensive firms. They tend to rely mainly on their employees and their individual competencies (codified or non-codified), the networks and communities they are involved in as well as the structural resources of the organization itself. These resources, mainly of intangible nature, are expected to play an important role on the innovation capabilities of RTOs, but tend to be neglected in the traditional performance reporting and management systems. The main motivation for this research is to foster the introduction of an intangible measurement system in the RTO under investigation. This paper illustrates the application of an innovative approach, for the design of a performance measurement system in a RTO. The process of collection and reporting of intangible indicators is defined and validated by relying on an in-house approach and related tool, called EFFICENT. 

Keywords: Intangible asset reporting, RTO, Process definition.

---

1-    Introduction

This paper is based on a single case study and focuses on the design phase of a performance measurement system. More specifically, it illustrates the application of an innovative approach, for the definition and collection of indicators related to intangible assets in a Research and Technology Organization (RTO). RTOs are knowledge-intensive firms, which tend to rely mainly on their employees and their individual competencies (codified or non-codified), the networks and communities they are involved in as well as the structural resources of the organization itself. The latter cover the traditional intellectual property items, but also the collective practices, routines or even formalized systems and databases. All these resources are expected to play an important role on the innovation capabilities of RTOs, but tend to be neglected in the traditional performance reporting and management systems. Thus, the main motivation for this research is to foster the introduction of an intangible measurement system in the RTO under investigation.

The process of collection and reporting of intangible indicators is defined and validated by relying on an in-house approach and related tool -called EFFICENT- which allow modelling and validating electronic (B2B) transactions.

This paper is structured as follows. The following section discusses the definition and importance of intangibles measurement, reporting and management in a general context. Section 3 contextualises the issue in the RTO under investigation. Section 4 introduces the EFFICIENT approach and tool. The final section explains how EFFICIENT was used to define the process of collection and reporting of a set of intangible assets indicators in this RTO and opens the discussion to further investigation.

2-    Intangibles: Measurement, Reporting & Management

Intangibles are now widely acknowledged as the main source of competitive advantage of organizations. Several studies at macro-economic level even demonstrate that the investment in intangibles has exceeded investments in tangible goods in various countries of the world (e.g. OECD studies). Although their importance is now recognized by both practitioners and scholars, there is a lack of consensus on what the terms “intangibles” or “intellectual capital” actually cover.

There is indeed an abundant literature providing numerous definitions of these concepts (e.g. Edvinsson & Malone, 1997; Sveiby, 1997; Stewart, 1997; Roos et al., 1997; Brooking, 1997; Lev, 2001 etc.) as well as related ones such as “invisible assets” (Itami, 1980), “human capital” (Becker, 1964) or the broad literature stream on “knowledge” (e.g. Veblen, 1904; Drucker, 1959) ,  “knowledge assets” (Teece, 1987) and an even more extensive one on Knowledge Management. This abundant literature reflects the diversity of actors contributing to this research topic (e.g. standard-setters and accounting bodies, academics, practitioners and consultants), their main disciplinary field as well as their different interests in addressing the issue of managing, measuring or reporting intellectual capital or intangibles.

In this paper, we rely on the view that some resources are susceptible to generate a sustainable competitive advantage for the firm, provided that they comply with specific attributes and that they are adequately combined, integrated, activated and continuously leveraged. However, we adopt the view that the Resource-Based View (Penrose, 1959) may be too general in the sense that it does not give accurate insight on what specific resources should be developed and accumulated and how they can best be configured in view of generating this sustainable competitive advantage (Priem & Butler, 2001; Foss & Knudsen, 2003). Similarly to Reed et al. (2006), our view is therefore to strictly consider resources that embedded in individuals, stored in the organization’s systems and processes and finally, mobilized in the relationships the organization pursues with various stakeholders. Our view is therefore to restrain our analysis to intangible resources, but also to focus on intangible activities, which aim at leveraging, reinforcing or creating new intangible resources.

As already mentioned, intangibles or intellectual capital can be defined in several ways. In this paper, we retain the definition provided by the Meritum consortium, which is now becoming widely recognized.  According to this definition, intangibles refer to the “non-physical sources of future economic benefits that may or may not appear in the corporate financial reports” (MERITUM, 2002). Intangibles are also known as intellectual capital, as the two terms are more and more being used interchangeably. In turn, intellectual capital is defined as the “combination of human, structural and relational resources of an organization” (MERITUM, 2002). Intangibles therefore encompass numerous items that are highly relevant for organizations, value creators if properly managed, but which tend to remain completely out of the balance sheet. Part of these intangibles is expensed or is hidden under different labels such as “labor cost” or even “goodwill”, since they do not comply with the recognition criteria of assets (i.e. IAS38 specifies that a company can recognize an asset only if it is identifiable, controlled and from which future economic benefits are expected to flow to the enterprise).

In recent years, there has been an increasing interest for intangibles. This has been concomitant with the drastic increase of the market-to-book ratio of companies, which actually came from unity in the late 1970s to over 7 in the early 2000s for S&P 500 companies. Both the practitioner and the academic literature has lately been burgeoning on the measurement, management and reporting of intangibles. Rationale for this increased interest can be rooted back to the willingness to open the “black box” and further understand the contribution of intangibles to the value creation process of organizations.

Different streams have emerged, either in the management/strategy discipline or in the accounting field. The focus of these streams is obviously different, as the former emphasizes the need of measurement for decision-making purposes, from an internal perspective, and thereby providing management with the relevant information. The latter tends to be more oriented towards the external stakeholders of the organization, giving them an accurate picture of the organization as it is, or more accurately, as it was in the previous accounting year. Numerous initiatives in the accounting sphere tend to extend the traditional accounting framework in order to accommodate for the reporting of intangibles, or to develop an additional reporting framework. This indicates that accounting may currently shift from an external perspective to a broader one, including the internal stakeholders’ needs (so-called “intelligence” by Van der Stede, 2009).

Reporting of intangibles or intellectual capital has consequently gained strong interest in different sectors and more specifically in companies that hold few tangible assets such as the knowledge-intensive firms.

3-    The specific case of a RTO

RTOs are expected to play a crucial role in transforming research into pragmatic outputs and to offer practical tools and methods for facilitating, managing and organizing innovation activities within firms in view of producing service innovations. The main mission of a RTO is therefore to provide research, development and innovation services both to private and public beneficiaries. Traditionally, RTOs are mainly non-profit organizations. They act as interfaces between universities and firms, and usually hold a certain level of autonomy in their management, while being accountable to government and various stakeholders. RTOs usually receive some public funding but are also engaged in private contractual relationships. 

In recent years, both transparency and accountability demands have become more and more stringent and RTOs are under strong pressure to demonstrate that they deliver pragmatic outputs, according to the level of the funding they receive. For example, The RTO under investigation in this case study research signed a “performance contract” with the Ministry of Research. This contract follows performance-based governance and includes a set of KPI (Key Performance Indicators), with target values. In addition to the performance contract and related set of indicators, the organization aimed at putting in place an intermediary reporting and monitoring system. This system would consist of a set of indicators, mainly related to intangible assets. These indicators refer to intangible inputs and outputs but also reflect intangible processes or activities. The importance of monitoring stocks and flows of intellectual capital in such an organization is particularly relevant as they mainly produce intangible outputs, based on intangible resources and activities.

Given the complex structure of this organization, with multiple levels of management, the definition of a consistent set of indicators rapidly appeared a difficult task. Indeed, the management of the RTO’s core assets lies in the hands of three organizational components: research programs, research units and service lines.

The main function of a research program (RP) is to ensure the coordination between internal projects on the one hand, and between project objectives and market needs on the other hand. Program managers are primary interfaces with a specific sector or subsector they are in charge of. Their main responsibility is to define a strategic plan for this sector. This strategy should cover the whole innovation chain. More specifically, they should identify research opportunities, leading to mid and long-term results in terms of products or services that can be offered to their targeted sector. Alongside they should develop an exploitation strategy of the competencies, products and services that are available at the RTO. From an operational point of view, this strategy is implemented through research and innovation projects and program managers usually act as project leader on these projects. They are assisted by a project manager, who assumes tactical and operational activities while the project leader is responsible for the strategic issues.

Project teams are multidisciplinary and are composed of highly qualified researchers. These researchers can be qualified as “knowledge workers”. Following the characteristics highlighted in the literature (see e.g. Drucker, 2000; Morello, 2001), researchers dispose of a relatively high level of autonomy to conduct their work, they act in close relationship with the companies the RTO has cooperation with and the results of their work is difficult to qualify in terms of quality or even in terms of tangibility.

Each researcher belongs to a research unit called here Scientific and Technological Unit (STU), which is specialized in a specific scientific and technological domain. The main STU function is to achieve the development individual skills and competencies in parallel to the development of a consistent and balanced set of competencies at the unit level, which we could refer to as the collective competence of the unit.

Finally, Service Lines (SL) are architects whose main mission is to identify, combine and valorise service components developed by researchers in the framework of several projects that are included in research programs.  They act as liaison devices between the research programs and research units.
Consequently, the idea to involve all these stakeholders in the collection process emerged, as well as the willingness to rely on an in-house approach and related tool to model and to simulate the collection process of these indicators.

4-    EFFICIENT

EFFICIENT (E-business Framework For an EFFICIENT Capture and Implementation of End-to-end Transactions) guides business experts within a business network throughout the process of designing an electronic business transaction, from the identification of the business model to its implementation as a message-based B2B process chain (Schmitt & Grégoire, 2006). The primary use of EFFICIENT framework is to help organizations having complex e-messaging requirements to validate their message structures and content before beginning any IT development.

The EFFICIENT approach is organised in three phases (Figure 1):

1. The discovery phase – initial phase where business analysts’ work with stakeholders to capture all core business needs and map out roles and relationships specific to the e-business model.
2. The specification phase – Analysts define individual e-commerce transactions (or messages) using UML Activity and Class Diagrams.  Most UML tools do not give analysts the ability to accurately define all of the complex aspects of an e-transaction.
3. The validation phase – electronic messages are validated by using the EFFICIENT animator toolset.  Analysts and business partners can then test the transaction in real-time, making changes for different scenarios, and immediately see the results.

Figure 1: EFFICIENT allows business experts to validate the transaction at design time

The EFFICIENT toolset was developed as a visual tool for business analysts so that they could create highly accurate models of proposed electronic transaction structures. The first part of the EFFICIENT toolset is a CASE (Computer Aided Software Engineering) plug-in for the UML modeling tool MagicDraw™ UML. Using the plug-in, an analyst creates very accurate model of the electronic business transaction and all of its corresponding electronic messages. This model also contains business rules and the information on the relationship between all relevant parties. Once this model is created, UML data is transformed into XML files and then sent to a workflow simulation engine (the second part of the toolset) so that all aspects of the model can be animated (or “played”) as if the transaction were actually taking place. Business users can see -and interact with- each electronic message in the model via a simple web browser.

5-    Definition of the process of intangible reporting in the RTO under study

Following the EFFICIENT approach, the definition of the intangibles reporting process, in the RTO under study, was organized in three phases: (i) the discovery phase, (ii) the specification phase and (iii) the validation phase.

5.1- The Discovery Phase

In order to launch the process and following an action-oriented approach, a preliminary set of indicators had been suggested by researchers active in the field of intangible measurement. These indicators were drawn upon an extension of the taxonomy of intangibles that is under development in XBRL (Extensible Business Reporting Language) community. Indeed, the taxonomy provided by XBRL does not currently provide indicators for all types of businesses, but only aims to provide a framework on which each organization can build up its own collection of indicators. Such an approach is consistent with recommendations that can be found in the management accounting literature (see e.g. Lev, 2001). In addition, following the contingency approach (Burns & Stalker, 1961) and relying on the grounds that that there is “no universally appropriate system which equally applies to all organizations in all circumstances” (Otley, 1980), a dedicated set of indicators have been designed for the RTO under investigation. The rationales behind this customized measurement system are further explained in (Mention, 2008), which discusses the potential fit between the design of the performance measurement system, its implementation and the organizational configuration.

A drawback of this approach is the lack of comparability between RTOs, should similar performance measurement systems be implemented.  However, two points should be emphasized. First, this performance measurement system should remain an internal one and should serve managerial purposes. It does not aim to become a substitute to the official performance measurement system, consisting of negotiated KPIs that are included in the performance contract with the Ministry of Research.

Second, it has to be highlighted that few prior experiences in the field of performance measurement in RTOs can be mentioned.  In addition, those experiences focus on the reporting side and therefore primarily serve the reporting and communication purposes, although it is assumed that they also act as instruments serving decision-making purposes with respect to investment or development of strategies. Illustrations of such experiences are Austrian RTOs and universities now have to publish intellectual capital reports, following the 2002 Austrian Act (Leitner, 2005) or similar initiatives developed in Spain.

An exception may be the voluntary performance measurement system has recently been designed and implemented in a research organization in Finland (Mettänen, 2005), based on a collaborative approach with researchers from various departments who spontaneously formulated potential indicators.
Finally, given the exploratory nature of the work conducted here, our purpose is indeed not to design a performance measurement system dedicated to reporting and that would allow benchmarking between “comparable” organizations. On the contrary, our aim is to stimulate the reflections on what intangible resources and activities to measure, how to measure them and whom to involve in order to ensure a commitment in this measurement process but above all, that both corrective and preventive action is undertaken on the basis of this measurement process.

The following subsection provides information on a restricted set of indicators that have been submitted to the audience of program managers, research unit managers and service line managers during the experimentation.

        5.1.1.    Internal cooperation intensity

This indicator relates to the “relational capital” dimension of the intellectual capital framework. It more specifically intends to reflect the intensity of the internal cooperation between service lines, research & innovation programs and scientific and technological units. This is particularly important in the case study (as in most of knowledge-intensive firms) as this cooperation is supposed to positively influence the innovation capabilities of the RTO. Indeed, program managers act as interfaces with the market they target (e.g. financial or construction sector). Based on the knowledge and information they have on their market, they bring into the RTO challenges to be addressed by the research teams.  On the other hand, they also introduce to the market services that are packaged by service line managers. It is also therefore crucial that service line managers communicate with program managers in order to ensure that the services they develop and package address actual needs of the market. Cooperation with research units is also highly important, as there should be a permanent adjustment between the competencies developed in the RTO and the needs of the local socio-economic environment. In order to evaluate this intensity, a likert-scale has been used.

        5.1.2.    Number of contractual cooperation with the market

This indicator concerns the number of contracts a research & innovation program has with market during a year. The performance contract that has been introduced in the RTO includes an indicator related to so-called contractual research, which is related to the amount that private and public organizations actually invest in research, development and innovation activities. Contractual research indicator is supposed to reflect the propensity of firms to use and implement tools, methods and so on that are developed by the RTO. In this way, it gives an indication (though preliminary and limited) on the impact that the RTO has on the local market.

This indicator intends to complement the “contractual research KPI”, taking into consideration all forms of contractual relationships, disregarding if the contract includes or not financial transactions since it is considered that focusing exclusively on contracts involving financial flows may give a distorted representation of the impact of the RTO on the local economy.  Indeed, interactions with market players is crucial for researchers in their daily activities in order to develop tools and methods , and at some early stage of development, experimentation of these tools is needed in external companies, but these companies may not be willing to invest anything else than time and expertise in the project. This indicator relates to the relational capital dimension of the intellectual capital framework. It can be compared to the number of clients or customer retention indicator that could be alternatively used by private organizations.

        5.1.3.    Level of education of researchers

This indicator refers to the human capital dimension of the intellectual capital framework. It actually reflects the level of education of the research staff in the organization. In the case of knowledge-intensive firms, it is indeed highly relevant to monitor the level of education of staff, though it is a static measure. Additional indicators reflecting the ongoing education process (e.g. vocational training followed, certifications hold, enrolment in PhD or post-doctoral programs, or mobility programs, etc.) would give a more accurate picture of the human capital of the RTO.

        5.1.4.    Quality of management

This indicator is related to the quality of leadership within the organisation e.g. perceptions (internal/external) of the quality of the management team/CEO. It can be related to both the human and structural capital (i.e. image, reputation) dimensions of the intellectual capital framework. As the quality of management has been demonstrated to affect the performance of the organizations and the motivation and commitment of its members, the use of this indicator is self-explanatory. A likert-scale has been used to qualify this indicator.

5.2- The Specification Phase

In EFFICIENT, one transaction (a series of messages) is modelled inside one UML (Unified Modeling Language) package (even if some elements – like reused classes – can be defined in another package). The choreography of messages exchanged between actors is represented using an UML activity diagram, where Swim-lanes represent the different roles of the transaction, and objects represent messages exchanged between activities in these roles. Other UML constructs can be used, like forks, joins, and decisions.
Each message is defined in a class diagram with the same name, and containing a “root” class. The class diagrams contain no operations, and have a hierarchical structure (no loops, only oriented relations). They can use classes defined elsewhere.

Additional concepts that cannot be modelled in UML can be modelled in EFFICIENT's business rule definitions:

• Inter-Message Rules allow linking data between different messages. They are represented using notes on class diagrams. Default rules can be generated based on the classes reused in different diagrams and the order of messages exchanged.
• Business Rules allow modelling more advanced constraints on the model.In addition, nested transactions can be defined on activity diagrams, by using a sub activity flow linking to the activity diagram in another package containing the nested transaction.In the following, we present the results of the specification phase for each of the intangible indicators.

        5.2.1.    Internal cooperation intensity

This indicator concerns the intensity of the internal cooperation between service lines, research & innovation programs and scientific and technological units.





In the activity diagram of Figure 2, the service line manager (SLiM) evaluates his cooperation with the different service lines, research & innovation programs and scientific and technological units of the department. The structure of his message “SL_internal_cooperations” is defined in Figure 3. The director receives the SLiM report and comments it (Message “Comments_on_SL_internal_cooperations”, Figure 4).




Figure 4: Structure of the message “Comments_on_SL_internal_cooperations” from a SLiM to the Director

        5.2.2.    Number of contractual cooperation with the market

The indicator “Number of contractual cooperation with the market” is reported by a Research & innovation program manager (PM) to the organisation director.
The activity diagram of Figure 5, involves two PMs (PM1 & PM2) who report for each of the research & innovation programs they manage, the number of contractual cooperations. Collected data is aggregated by a fictive role (DataAggregator) and transmitted to the Director (Figure 6).








        5.2.3.    Level of education of researchers

This indicator concerns the level of education of the research staff in the organization.
Information on the obtained diploma(s) is gathered from the researchers, aggregated on the level of scientific and technological units, and communicated to the direction.
The activity diagram of Figure 7, involves two researchers from the same STU. Their reports are validated by their unit manager, and then aggregated (by the fictive role DataAggregator). Aggregated data is transmitted later to the department Director.




        5.2.4.    Quality of management

This indicator is related to the quality of leadership within the organisation e.g. perceptions (internal/external) of the quality of the management team/CEO. In this case (Figure 8), unit managers are evaluated by the researchers on the quality of their leadership and their availability. Aggregated data is transmitted to the director, commented by the director, and then communicated to the unit managers.




5.3- The Validation Phase

In order to validate the models, designed during the specification phase, a working group has been organised. During this working group, key members of the studied organisation were asked to play the different roles modelled in the transactions. At the end, they gave their feedbacks on the experiment as well as on the EFFICIENT tool.

        5.3.1.    Internal cooperation intensity

The “UnitName” type was changed to an enumeration of possible unit names. Similarly, the service line name and research & innovation program name were changed to enumerations of existing service line and research & innovation program names in the organisation. Some documentation was also added to the “level of cooperation” fields to explain the possible rates.

However, the most important discussion was about the process itself: the process as it is presented here was considered too simple, and should imply the research & innovation program managers and unit managers. It should take input from these three roles to compare them and produce a consolidated result. However, for time reasons, the working group was not able to produce the complete process, which should thus be created after further discussions, in a next version.

        5.3.2.    Number of contractual cooperation with the market

During the animation, the participants of the working group identified that it would be interesting to calculate the sum of contractual cooperations, in addition to the average by research & innovation program. This field was thus added to the aggregated message.

The research & innovation program manager name, which was initially provided in all messages, was also removed following the working group request, as it does not bring any valuable information and is prone to typographical errors. The process in itself was not changed.

        5.3.3.    Level of education of researchers

As for the second indicator, the animation allowed us to identify that the unit manager name was not useful in this transaction. In addition, because of potential errors in typing the unit name, it was decided to change the type of the “unitName” field, in order to use an enumeration containing the possible values, instead of a “string” field. The process in itself was not changed.

        5.3.4.    Quality of management

Following the animation, as for the previous indicators, the unit name type was replaced by an enumeration in order to limit the possible values to existing unit names. The unit manager name was nevertheless kept, as in this case it is important to specify which unit manager is concerned by the evaluation, in particular because some units have two different unit managers.

Another remark expressed by the working group participants was that the possible values for rates (“averageLeadership” and “averageDisponibility” fields) should be documented: a documentation was thus added to specify that the rates are between 1 and 5 and to specify what the meaning of these rates.

About the process, it was discussed whether the unit manager should be informed of the results before the director, and if the rates given by researchers should be anonymised or not, but no conclusion was reached during the working group and the process was left unchanged.

6- Conclusions and further developments

The presented/discussed case study research was primarily aimed at addressing the performance measurement of RTOs. This is a particularly difficult task, since the outputs produced by RTOs are mainly of intangible nature. The success of the introduced static performance measurement system consisting of KPIs has been mitigated, as it may focus too much on selected dimensions of performance and short term planning to the detriment of long-term strategy and knowledge creation. It can therefore have counterproductive effects on the way the organization is managed.

In order to address this issue, an additional performance measurement framework, focusing on intangibles, both as inputs, processes and outputs, has been submitted to empirical validation using an innovative toolset. The underlying rationale is not to add further reporting constraints to the organization but to stimulate reflection on the meaning of performance measurement and its usefulness for managerial purposes.  Involving key stakeholders has indeed facilitated the process of data collection and has leveraged the commitment of these stakeholders in the measurement and management of intangible resources and activities. 

Further steps include the extension of this measurement framework to cover other aspects of the intellectual capital of this organization, and to ensure that it is applied on a regular basis by strategic asset managers of the RTO in their daily decision-making process, which mainly consists of managing knowledge and its creation.

Acknowledgment

The authors would like to thank Ms Sophie Ramel for her support during the design and validation phases.

References