Keywords

1 Introduction

The continued increase of technological capability in engineering brings with it many challenges. Accompanied by globalisation [1], high complexity [2], and large operational scales [3], modern engineering projects require significant effort for effective management and control. Where 70 years ago engineering typically occurred through the efforts of tens to hundreds of personnel working concurrently and often co-located, engineering now routinely involves the concerted efforts of tens of thousands, developing millions of parts in hundreds of countries [4].

Driven by this change, challenges in engineering industry have manifested both in the control and management of the vast amounts of information now produced, and in the effective management of complex engineering project and operations. This has led to the development of sophisticated, through-life computational data management such as embodied in Product Lifecycle Management (PLM) and Product Data Management (PDM) systems, which have shown rapid uptake and high utilization across engineering industry.

While these systems provide game-changing capability for data management, storage, and retrieval, there remains significant opportunity for greatly extended utilization of the data held for analysis and support of effective management and enhancement of work performance. Recent work has shown great potential in development of broad-spectrum and low-level analytics of digital files for direct interpretation of higher-level work performance [5,6,7], beyond commonly applied techniques supporting process and workflow management. Of key benefit here is the potential for automatic and near-real-time analysis directly extracted from data held within PDM systems.

This paper presents an exploration of the potential for such analytics for the purposes of engineering management and support of high work performance. Through meta-analysis of 25 different analytic approaches targeted at automatic analysis of digital files (DFs - reports, communications and models) it identifies capability for analytics directly applied to PDM systems as-is, presents coverage of underlying understanding that may be developed, and identifies extensions to PDM systems for maximization of analytic capability.

2 Performance of Engineering Work

In interpretation of performance of work, two highly inter-related perspectives are frequently adopted. The first focuses on metrics of performance and direct measurement. Typically utilizing the classic “iron triangle” of time, cost, and quality, but extending to metrics such as stakeholder and user interpretation of work outputs [8] that attempt to provide quantifications of process and output that are resultant of work performance. In each, the values measured must be contextualized against the desired or expected for the case. The second perspective is concerned with critical success factors (CSFs) [9] and embodies descriptions of those characteristics of a work situation that will influence eventual performance dependent on their state. Examples include the cohesion of teams [10], definition of specification, and availability of information [11]. In application, CSFs (see Table 1) are characterized in the individual case, and then interpreted by management to support decision-making and intervention with a goal of eventual performance enhancement.

Table 1. Critical Success Factors (CSFs) in engineering management
Full size table

CSFs highlight a wealth of complex and inter-related factors which create significant complication in the effective management of engineering work. Managerial decision-making must identify the factors that are of influence upon performance, the means of their measurement, and the nature of consequent action and intervention that will support enhancement. This difficulty is exacerbated in practice by the distinct nature of each situation in that the key performance-influencing factors of one case may or may not be relevant to another case, in turn impacting their correlation with consequent work performance and appropriate methods of measurement.

Recent work has shown the viability of application of a condition-monitoring approach to work monitoring in engineering [5, 6], in which broad-spectrum and low-level data analysis is applied to work activity and used to infer higher work characteristics. This approach gives significant proven scope to detail work activity automatically and in real-time through algorithmic monitoring and analysis, thus providing managerial workers with detailed description of the situation under their responsibility and supporting their decision-making processes. This capability is in addition to managerial toolsets such as EVM, PMBOK, and Prince2.

As a digital repository of engineering data PDM systems provide high compatibility with this approach, where the digital files held within form the dataset on which analysis could be driven and from which understanding of the specific situation may be built. From effective application of analytics to PDM data, there is therefore potential to provide direct analysis and interpretation of specific engineering situations to decision-makers, thus increasing managerial capability while reducing investigative effort.

3 Data Analysis in PDM

Of vital importance in engineering workflow, PDM systems provide data storage, management, and structure that enable version control, information access, and process modelling for data management, amongst other capabilities [13]. While varying in sophistication from simple file structures to complex databases the value of PDM system capability is widely recognized, with successful implementation shown to give such benefits as increased flexibility and reliability of products, shorter time-to-market, and reduced development costs [14].

Due to variation in company structures, product, product purpose, and PDM system, utility of the data and data formats captured within PDM systems are often ad-hoc, specific to the engineering software in use, and oriented to the design itself [15]. Data captured within PDM systems typically consists of the file itself and accompanying meta-data, itself comprising a combination of manually and automatically categorized fields describing the file itself or the product it represents [16].

Dependent on the data held within the system, analytics employed within PDM systems are also of an ad-hoc nature, with focus on workflow, data and information characterization, and analysis of product, process, and programme structures [13, 16]. While these analytic methods support managerial practice through description of system/process characteristics, particularly in control and specification of process, this work contends that the data held within PDM systems provides a rich source from which interpretation of work performance characteristics may be derived directly. Further, as explored within this paper, scope exists for many analytics to be applied in an automatic and real-time manner, with all data inputs extracted directly from files without manual categorization or description of meta-data. As such, scope exists to provide analyses general to all work situations, extend capability within PDM systems, and address the significant complexities in understanding of performance within varying engineering contexts described in Sect. 2.

4 PDM Data for Analysis of Engineering Work Performance

As the direct outputs of the activities of engineers working within an engineering context, the digital files (DFs) created, modified, and captured within PDM systems are a result of the multiplicity of internal and external influences upon the engineering situation in which the engineers work. Dependent on the characteristics of the engineering situation the type and properties of the DFs and the nature of their creation/modification will vary. Accordingly, through study of the characteristics of DFs there exists scope to understand the ongoing activity within the engineering situation and as such to infer information about the characteristics of the situation itself; information that supports managerial understanding of project performance. For example, characteristics of communicative activity are dependent on team cohesion and conflict [10], which in turn are related to project success [17]. This relation between characteristics of the engineering situation, activity, and the DFs created forms the basis for the analytics studied.

4.1 Types of Analysis and Outputs of Analysis

Given this relationship between data, activity, and performance, analysis techniques may either directly describe current engineering activity, or may infer the state of CSFs (see Sect. 2). In the former, the onus rests on the manager to interpret activity patterns observed and infer performance, while in the latter the manager must compare resultant descriptions of the state of the engineering situation against expectation.

In this work, analyses are classified according to these distinctions, with all categorized as providing a description of ongoing activity or according to the CSFs to which they relate. The CSFs used for categorization are given in [6], and for brevity here separated into four primary categories, with a total of 10 sub-categories:

Person; (team/individual) - CSFs relating to the characteristics of the persons directly involved in the ongoing work.

Design - CSFs relating to characteristics of the output of the engineering work.

Process – CSFs relating to characteristics of the processes by which the ongoing engineering work proceeds.

Context; (management/information/stakeholders/resource/environment/personnel) – CSFs relating to the broader context of the company and ongoing engineering work, according to the listed categories above.

4.2 Data Types

Current PDM systems capture a variety of data-types, often bespoke to the specific company or work, with focus on those representative of the product and process (i.e. CAD files, reports). DFs in this work are classified as one of three types, the former two being commonly captured in PDM, and the third not commonly captured:

Representation: including all virtual representations and models of the object of the work, including CAD models, virtual prototypes, analysis models (e,g. FEA).

Report/Documentation: Including all textual or numerical documentation that addresses the output, process, or project, including technical and managerial reports, presentations, excel spreadsheets, databases, etc.

Communication: including all digital communications sent between workers, including email, social network, and instant messaging.

All files are analyzed through focus on their properties at one of two levels:

Physical attributes: the characteristics of a file as an entity within the digital system, typically meta-data - including, for example, size, creator, creation/modification date, filetype, etc. This data is common in PDM systems.

Content attributes: the content of the file including, for example, textual data within communication, numerical data within spreadsheets, coordinate data within CAD models. This data is common in PDM systems, but less commonly used in analysis.

4.3 Data Analytics Studied

This work compares 25 analytics developed as part of a 5-year project into analytics of engineer work. Each interprets either activity or CSFs from low-level digital files produced in engineering work. For brevity in this format these are detailed in summary only in Table 2, with further detail available from the authors on request. These analytics are not considered exhaustively representative but include all identified in the 5-year project, and thereby describe the breadth of those identifiable within the project scope. All are (a) automatic in their application to data, and (b) provide near-real-time outputs allowing quick understanding and action from decision makers. Stemming from engineering work many analytics are applicable across context, where data-type available is the enabler. Utility of analytic is dependent on the context and should be assessed on a case-by-case basis dependent on priority and apparent issues. Further detail is available from the authors on all analytics and their validation.

Table 2. Analytics of low-level digital files explored
Full size table

To identify the elements of work performance to which the output of each analytic relates, all were classified by the CSF categories and if their output provided a description of underlying activity. This is shown in Table 3.

Table 3. Coverage of analytics against CSFs and activity. Headers indicate CSF categories - Act: Activity, Ind: Individual, Des: Design, Pro: Process, Man: Management, Info: Information, Env: Environment, Per: Personnel, Sta: Stakeholder, Res: Resource. M - metadata only; B – both metadata and content attributes must be studied.
Full size table

Through this meta-analysis a summation of the inferencing power of each analytic technique is produced, with a particular focus on the capabilities of analyses applied to different data-types against the types of CSF to which they relate. In so doing this work clarifies the manner in which such analyses may support understanding of work performance. Following categorization, the capabilities of analytics with respect to the data inputs required and the CSFs to which they relate can be summarized, thereby allowing contextualization to data held within PDM systems (see Table 4).

Table 4. Analytic capability of extension to PDM systems.
Full size table

5 Discussion and Conclusion

From Table 3, a summation can be drawn of the possibility for extension to analytic capability within PDM systems, based on automatic and near-real-time analysis of DFs held within (see Table 4). From Table 4, several observations can be made about the capability of analytics based on low-level data held within PDMs to support interpretation of work performance.

There is high capability to monitor work performance via automatic and real-time analytics of DFs, including those held within PDM systems. Much benefit may be derived from addition of meta-data only analytics applied to existing data, while further benefit can be derived from inclusion of content analysis. The integration of such presents a significant area for future work in PDM system development. This has impact on integration into PDM systems in that the system must be able to fully open all DFs that it contains and parse content for analysis.

All 25 analytics provide description of activity, and as such demonstrate the potential for PDM data to provide automatic, detailed, and real-time monitoring capability. The breadth of description of analytic outputs is particularly exciting, describing many and varied aspects of engineer activity.

Data held within current PDM systems allows analysis of process-based and design-based CSFs, but with little capability to evaluate CSFs related to person and context categories. Extension to communication data enhances evaluation of person categories, but not context. As each of these areas is of vital importance to high performance this suggests a weakness in analysis as applied to PDM systems – it will not capture performance entirely.

Extension of PDM systems to capture and analyse communications sent as part of engineering work has great potential to inform of performance in person-based CSFs and extend the analytic capability of PDM systems generally. These specifically include areas of performance in which PDM systems are unable to inform.

Many analytics inform multiple CSFs, thereby providing a means through which limited implementation may provide broader information and understanding.

5.1 Benefits and Challenges in Implementation of Analytics

In context of management of engineering work through implementation of such analytics, several benefits and opportunities exist. First, this work demonstrates (1) opportunity for extension of analyses based on current PDM data, (2) opportunities for extension of PDM analytic capability through content-analysis, and (3) opportunity for extension via communication data capture and analysis. Each of these creates opportunity to address key challenges in engineering management. The applicability to PDM systems is highly significant, in that their near-ubiquitous implementation and data-neutrality of analytics build on infrastructure already in existence. Second, the broad categories of CSF against which analytics inform show the usefulness of such an approach in generating broad understanding. This is particularly true where single analytics relate to multiple CSFs, and through the emergent effect of simultaneous analysis of many aspects of engineering work. Third, the autonomy of each analytic provides significant capability, in that the investigative effort of managers may be reduced while simultaneously increasing their understanding of work performance. Fourth, analytics may be applied directly to data currently held within PDM systems, initially to basic meta-data created as part of the DF creation and modification process, and later through expansion of PDM capabilities to analysis of DF content directly.

There remain, however, several challenges to full implementation. First, as with the methods of implementation and data capture of PDM systems themselves, analytics applied must be selected and validated on a case-by-case basis. While general applicability to simple data-types exists, the uniqueness in data-types across engineering industry introduces significant complication. Further, as high performance may manifest in different ways for each unique work situation, the interpretation of performance from any analytic must be contextualized to the specific situation by managers before actions may be taken. Next, while usefulness of such analytics has been demonstrated, capability given current PDM data is limited primarily to CSFs concerning the Design and Process, hence away from the Persons involved in work and the situation Context. This presents both a difficulty and an opportunity; for full analysis of work performance PDM data capture must be expanded, and that capture and analysis of ongoing communications provides an avenue by which this may be best achieved. Finally, application of such analytics face technical challenges in their implementation that must be overcome. These include such well-documented issues as the formation of dictionaries for linguistic analysis (as required by email and report content analysis), issues associated with data privacy and security in autonomous analysis (i.e. for email), consistency in input data structure and standardisation across industry and analytic implementations, and complexity and demonstration of validity of big-data approaches.

5.2 Conclusion

Application of analytics such as those studied have broad capability to support and enhance the capabilities of PDM systems, particularly in that many have direct applicability to the data PDM systems currently capture. This work has shown the opportunity for immediate and detailed analysis of performance utilizing only existing PDM data, and the potential for extension of capability using content analysis and the additional capture of communications sent as part of engineering work. These analytics have strength in autonomy and speed, with potential for near real-time provision of information that has previously been unavailable to managerial decision makers.