R&D performance evaluation and analysis under uncertainty: the case of Chinese industrial enterprises

Abstract

Data Envelopment Analysis (DEA) is a widely adopted non-parametric technique for evaluating R&D performance. However, traditional DEA models often struggle to provide reliable solutions in the presence of data uncertainty. To address this limitation, this study develops a novel robust super-efficiency DEA approach to evaluate R&D performance under uncertain conditions. Using this approach, we analyze the R&D performance of industrial enterprises across 30 Chinese provincial regions from 2018 to 2022. The empirical results reveal a notable decline in R&D performance during 2018–20, driven by external shocks such as trade conflicts and the pandemic, followed by a gradual recovery post-2020, a trend that remains consistent under varying levels of data perturbation. Regional analysis highlights substantial disparities in R&D performance across Chinese regions. Comparative analysis further demonstrates the proposed model’s advantages in feasibility and computational efficiency. Based on the empirical analysis, we provide several policy implications. While rooted in the Chinese context, this paper contributes both methodologically through its robust DEA framework for handling uncertainty, and empirically by offering valuable insights into improving R&D performance in diverse national and organizational settings.

1. Introduction

Industry constitutes a critical pillar of the national economy, playing a pivotal role in job creation and elevating living standards. Given its large population, China places considerable emphasis on industrial advancement. The country has emerged as the world’s largest manufacturing nation, driven by robust government-led industrialization initiatives (Aslam et al. 2021). In 2020, China accounted for 35% of global manufacturing output, approximately three times that of the USA (12%), six times that of Japan (6%), and nine times that of Germany (4%).¹ Despite these achievements, China’s advanced industries currently lag behind developed countries such as the USA and Japan (Han et al. 2017). Increasing R&D investment is a critical strategy for China to narrow the gap with developed countries in high-tech sectors (Zheng, Wang and Bao 2024). From 2012 to 2022, China’s R&D expenditure has steadily grown, escalating from 1.03 trillion yuan to 3.08 trillion yuan, marking an average annual growth rate of 16.11%.² Significantly, more than 60% of this investment annually originates from industrial enterprises above designated size (IEDSs).³ Given the significant R&D investments by Chinese industrial enterprises (CIEs), evaluating their R&D performance is crucial for guiding future policies and enhancing China’s competitiveness in high-tech sectors.

However, assessing and analyzing CIEs’ R&D performance remains challenging, especially in today’s highly uncertain global environment. The US-China trade war has profoundly impacted China’s industrial development, disrupting the innovation environment and intensifying uncertainties surrounding the input-output ratio of innovation in these enterprises (Chen, Zhang and Miao 2023). Meanwhile, the COVID-19 pandemic has heightened the risk of cash flow disruptions, prompting enterprises to adopt conservative operational strategies and delay or reduce high-risk innovation activities (Zhuang, Luo and Li 2023). Although the pandemic’s immediate effects have lessened, ongoing virus mutations and the potential for future health crises sustain a climate of uncertainty. Moreover, the rapid evolution and adoption of emerging digital technologies are fundamentally reshaping the landscape of R&D activities in industrial enterprises, significantly influencing their investment decisions (Wang et al. 2024). These uncertainties render the external business environment highly volatile and unpredictable, leading to significant challenges for companies in formulating strategic plans and allocating resources. Under such circumstances, the data observed or collected by decision-makers often contains significant biases, making it difficult to accurately reflect the true operational status of the enterprise (Lin and Lu 2023).

Beyond the above external environmental uncertainties, the intrinsic complexities of enterprise data collection and R&D activities pose additional challenges for performance assessment. In practice, enterprise statistical data typically represents observations at specific points in time, and these values may fluctuate depending on when the data is collected (Wei, Ma and Ji 2024). Such temporal variability, coupled with measurement errors and reporting inconsistencies, introduces considerable uncertainty in recorded data (Lin and Lu 2024). The uncertainty is further amplified by the intrinsic characteristics of R&D activities, including extended development cycles, unpredictable innovation outcomes, and variable market responses to new technologies (Ye, Paulson and Khanna 2024). For CIEs, these challenges are particularly significant due to rapid technological changes and dynamic policy environments.

The preceding analysis indicates that R&D data from CIEs exhibits significant imprecision and uncertainty. Performance evaluation methods that fail to account for such data uncertainty may generate biased results, potentially leading to suboptimal policy recommendations and inefficient resource allocation decisions. This underscores the critical need for developing a robust approach that can effectively evaluate R&D performance while explicitly incorporating data uncertainty into the analytical process.

Data Envelopment Analysis (DEA) is a mathematical programming approach used to assess the relative efficiency of decision-making units (DMUs) (Charnes, Cooper and Rhodes 1978). Over the past two decades, DEA has emerged as a widely adopted method for evaluating R&D performance in the literature (Lee, Park and Choi 2009; Karadayi and Ekinci 2019; Yue, Gao and Suo 2020; Chen, Liu and Zhu 2022; Aristovnik et al. 2023; Zhao, Pei and Yang 2023; Chen et al. 2024). Despite its nonparametric advantages, DEA has notable limitations. Firstly, traditional DEA provides efficiency scores between 0 and 1 for each DMU, which can obscure performance differences among efficient DMUs. Secondly, DEA relies on mathematical programming models, making the optimal solution highly sensitive to data accuracy. However, data uncertainty is an inherent challenge (Hatami-Marbini and Arabmaldar 2021; Hadi-Vencheh et al. 2024). Even minor data disturbances can violate DEA model constraints, leading to unreliable efficiency scores for DMUs (Toloo, Mensah and Salahi 2022; Li et al. 2024). Therefore, given the high uncertainty surrounding R&D activities in CIEs, enhancing the robustness of DEA is critically important.

To address the first limitation of traditional DEA methods, a straightforward and effective solution is to employ the super-efficiency DEA (SEDEA) model. SEDEA can assign super-efficiency scores >1 to efficient DMUs, thereby distinguishing them (Andersen and Petersen 1993). Regarding the second limitation, developing uncertain DEA models is a worthwhile endeavor. Unlike other uncertain DEA methods such as Stochastic DEA and Fuzzy DEA, Robust DEA (RDEA) applies robust optimization (RO) techniques that do not rely on strict assumptions like membership functions or probability distributions. Additionally, the conservatism of RDEA solutions can be adjusted according to decision-makers’ preferences, ensuring it avoids excessive conservatism seen in methods like Interval DEA (Arana-Jiménez et al. 2021). These attributes position RDEA approach as an advanced method for addressing data uncertainty in DEA (Hatami-Marbini, Arabmaldar and Asu 2022; Arabmaldar, Sahoo and Ghiyasi 2023; Arabmaldar et al. 2024). However, research on robust SEDEA models remains limited (Sadjadi et al. 2011; Arabmaldar, Jablonsky and Hosseinzadeh Saljooghi 2017).

Despite the advantages of SEDEA in efficiency discrimination and RDEA in uncertainty handling, few studies have attempted to integrate both approaches into a unified framework for R&D performance evaluation. To bridge this gap, we first construct a non-radial robust SEDEA method utilizing the RO technique. Given that the proposed robust SEDEA model cannot differentiate inefficient DMUs, a binary variable is introduced to integrate the robust standard DEA model with the robust SEDEA model. This integrated approach simultaneously addresses data uncertainty and enables comprehensive DMU ranking. Subsequently, the integrated robust DEA model is applied to assess the R&D performance of CIEs across 30 Chinese provinces from 2018 to 2022. Building upon efficiency evaluation, we provide a detailed analysis of efficiency trends and regional disparities. While this study focuses on CIEs, the challenges of R&D performance assessment under uncertainty are not unique to China. Therefore, the insights derived from analyzing CIEs could offer valuable references for evaluating and improving R&D performance in industrial sectors of other countries facing similar uncertainties.

Table 1 highlights the distinctions and unique features of this study compared to previous literature. Notably, no prior research has introduced a non-radial robust SEDEA model to assess the R&D performance of industrial enterprises under uncertainty.

This paper not only advances the methodological frontier of efficiency measurement but also provides policymakers with a more reliable tool for R&D resource allocation optimization and strategic planning in industrial enterprises. Specifically, our contributions can be summarized as follows:

1. Our approach provides a practical and scalable theoretical framework for assessing R&D performance in uncertain environments. The framework addresses the limitations of traditional DEA methods by combining robust optimization with super-efficiency technique and offers flexibility for application across diverse institutional and national contexts.

2. By introducing a binary variable, the robust standard DEA model is integrated with the robust SEDEA model. This integration enhances our model’s computational efficiency and reduces decision-making time for policymakers.

3. Employing the proposed model, we evaluate the R&D performance of industrial enterprises across 30 Chinese provinces from 2018 to 2022. The findings reveal several generalizable insights about R&D performance determinants that are valuable for policymakers in various national contexts, especially in countries with uncertain R&D environments.

The rest of the paper is structured as follows. Next section introduces two relevant models: the standard Enhanced Russell Measure (ERM) and the super-efficiency ERM. Section 3 develops an integrated robust SEDEA model using the RO technique. Section 4 provides the empirical analysis. Model comparison is performed in Section 5 to illustrate the superiority of our proposed approach. Finally, Section 6 summarizes the paper and provides managerial insights.

2. Preliminaries

In DEA, models can be categorized as radial and non-radial. Compared to non-radial DEA models, radial DEA models exhibit several distinct limitations (Liu, Xu and Xu 2023): Firstly, radial DEA models often face difficulties in choosing between input-oriented and output-oriented approaches in many situations. Secondly, radial DEA models fail to adequately consider the contribution of each input and output to efficiency, thus often resulting in overestimation of efficiency scores. Furthermore, several benchmark tests have demonstrated that non-radial DEA models can provide more accurate efficiency estimates (Kohl and Brunner 2020). Based on these considerations, this paper adopts non-radial DEA as the basis for developing the RDEA model.

The Slack-based measure (SBM) and ERM are two non-radial DEA models commonly used in the literature, which are equivalent (Wu et al. 2015). However, the equation constraints in SBM model will restrict the feasible domain of the robust model and may even lead to infeasibility issues. Therefore, our proposed robust SEDEA model is developed based on ERM.

2.1 Standard ERM

Note that although the above standard ERM can provide an efficiency score for DMUs, it cannot distinguish between efficient DMUs.

2.2 Super-efficiency ERM

Equation (2) eliminates the constraint of efficiency scores being less than or equal to 1 by excluding evaluated efficient DMUs from the reference set. Therefore, we can obtain a complete ranking of DMUs using equation (1) and equation (2).

3. Methodology

This section first describes the details of budgeted uncertainty set in robust optimization. Then, we show how to use the budgeted uncertainty set to construct a robust counterpart of SupERM. Finally, we construct an integrated robust ERM model.

3.1 Robust SupERM

3.1.1 Budgeted uncertainty set

Assume that uncertain inputs and outputs can be expressed as $x∼ij=xij+αijxx^ij$ and $y∼rj=yrj+αrjyy^rj$⁠, where $x^ij=eixij$ and $y^rj=eryrj$ are defined as the maximum deviations from the nominal values $xij$ and $yrj$⁠. The pre-set parameters $ei$ and $er$ are the percentage of perturbation, representing the extent to which the uncertain data deviates from its nominal value. The true values $x∼ij$ and $y∼rj$ come from the symmetric intervals $xij-x^ij,xij+x^ij$ and $yrj-y^rj,yrj+y^rj$⁠, respectively.

Herein, we further define the random variables $αijx=x∼ij-xijx^ij$ and $αrjy=y∼rj-yrjy^rj$⁠, which are symmetrically bounded in $[-1,1]$ (Bertsimas and Sim 2004). Let $Jix=jx^ij≥0$ and $Jry=jy^rj≥0$ denote the index sets corresponding to inputs and outputs characterized by uncertainty, respectively. Then, $DMUo$ is uncertain if $o∈Jix∪Jry∪Jkz$⁠. With respect to the aggregate disturbances, we define $∑j∈Jixαijx≤Γix$ and $∑j∈Jryαrjy≤Γry$⁠, where $Γix$ and $Γry$ denote the degree of conservatism of a robust solution in RO, also known as the budget of uncertainty. As a result, $Γix$ and $Γry$ take values in $[0,Jix]$ and $[0,Jry]$⁠, respectively, where $⋅$ denotes the number of elements in the set. Note that the robust counterpart of the envelope DEA model is built from an optimistic perspective (Toloo, Mensah and Salahi 2022). Therefore, $Γix$ and $Γry$ in this paper represents the degree of optimism of the decision maker. As $Γix$ and $Γry$ become larger, the DMU’s robust efficiency score will be higher.

The above set maintains the advantage of preserving the linearity of the problem compared to other uncertainty sets, making it widely utilized in the RO literature.

Here, we utilize a simple example designed by Arabmaldar, Sahoo and Ghiyasi (2023) to demonstrate the impact of budgeted uncertainty set on efficiency assessment. Three DMUs with single input and single output are considered, denoted by A, B and C, respectively. The dataset is shown in Table 2.

Figure 1 shows the efficiency frontiers for different $Γ$ under CRS. If $Γ=0$⁠, it means B is deterministic. Thus, the efficiency frontier is the ray OQ, i.e. $y=78x$⁠. The efficiency score for each DMU can be calculated using $87*yx$⁠. In this situation, A is the only efficient DMU. If $Γ=0.43$⁠, the efficiency frontier maintains the ray OQ unchanged. However, B becomes efficient, while A and C remain unchanged. If $Γ$ increases to 1, then the new efficiency frontier becomes the ray OT, which can be expressed as $y=x$⁠. In this scenario, B is the only efficient DMU. The efficiencies of A and C are $78≈0.87$ and $1013≈0.76$⁠, respectively.

3.1.2 Robust counterpart

Equation (2) is reliant on precise data. Any disruption to the data could render the model’s solution unattainable. In other words, we require the solution of equation (2) to remain feasible in the face of data fluctuations. To tackle this challenge, we employ the robust optimization technique to address data uncertainty.

In equation (7), the auxiliary variables $pix$⁠, $qijx(j≠o)$ and $qiox$ (⁠$pry$⁠, $qrjy(j≠o)$和$qroy$⁠) are introduced to determine the optimal values of uncertain input (output) variables relative to $Γix$(⁠$Γry$⁠). It is worth noting that the term $Γixpix+∑j∈Jix,j≠oqijx+qiox$ is added to the input constraints, while $-Γrypry-∑j∈Jry,j≠oqrjy-qroy$ is added to the output constraints. This indicates that $Γix$ and $Γry$ together with the auxiliary variables determine the robustness of the model solution. Consequently, they enable the production frontier of the evaluated DMU to shift between its worst-case and best-case scenarios (Hatami-Marbini et al. 2022).

Given the decision maker’s optimism level, equation (7) allows us to distinguish between efficient DMUs under uncertainty. However, this model fails to yield efficiency scores for inefficient DMUs. Therefore, we need to employ the robust ERM model to compute the efficiency scores of inefficient DMUs. To streamline the process, the next section will introduce an integrated robust ERM model for directly obtaining efficiency scores for all DMUs.

3.2 Integrated robust ERM

If the evaluated DMU is inefficient, then $π=0$ and the constraints of Group (I) will operate. In this case, equation (10) becomes equation (9), returning the same efficiency score as equation (9). If the evaluated DMU is efficient, then $π=1$ and the constraints of Group (II) will operate. In this case, equation (10) is equivalent to equation (7).

Equation (10) can be transformed into a linear program through the application of the Charnes–Cooper transformation technique (Charnes and Cooper 1962).

4. Empirical analysis

In this paper, the CIEs of 30 Chinese provinces are regarded as DMUs. Then, we employ the integrated Robust ERM model to assess the R&D performance of Chinese IEDSs from 2018 to 2022. Through analyzing this 5-year dataset, the study aims to reveal the trends and regional differences of CIEs’ R&D performance, thus providing insights into enhancing industrial enterprises’ R&D performance under uncertainty.

4.1 Indicators and data

DEA does not impose specific restrictions on input and output indicators. However, if the choice of input and output indicators is not in line with the evaluation objectives, it will reduce the credibility of the efficiency evaluation results. Therefore, it is essential to include core indicators related to the evaluation objectives in the measure, while respecting the principles of data availability and completeness. Based on these considerations and in conjunction with the frameworks of existing studies, the following indicators are selected for this study:

In terms of inputs, the basic elements of R&D activities are labor and capital (Hermanu et al. 2024). Referring to Khoshnevis and Teirlinck (2018), we select Full-time Equivalent of R&D Personnel (⁠$x1$⁠, man-years) to measure labor input, followed by R&D expenditures (⁠$x2$⁠, billion RMB) to measure capital input. Additionally, following Chen, Liu and Zhu (2020), expenditure on developing new products (⁠$x3$⁠, billion RMB) is chosen as the third input variable.

In terms of output, the number of patent applications is a commonly used metric for measuring the output of R&D activities (Liu et al. 2020). Therefore, the number of invention patent applications (⁠$y1$⁠, piece) is selected as the primary output variable in this study. The reason for choosing patent applications rather than granted patents is that ungranted patents also consume labor and funds and may have potential value. Moreover, there is a significant positive correlation between the number of patent applications and the number of patents granted. It is worth emphasizing that the completion of R&D activities requires successful commercialization. Thus, selecting relevant economic benefit indicators as outputs of R&D activities is imperative. Following Zhong et al. (2011), we further employ sales revenue of new products (⁠$y2$⁠, billion RMB) and prime operating revenue (⁠$y3$⁠, billion RMB) as measures of the economic benefits.

The indicator data are collected from the ‘China Science and Technology Statistical Yearbook’ compiled by the National Bureau of Statistics of China.⁴ Due to data limitations, this study ultimately selects 30 provinces in mainland China as the research objects. Table 3 shows the descriptive statistics of these input and output variables. The full dataset can be found in the Supplementary Material of this paper. Although these statistical indicators are presented as precise values, they contain uncertainty due to measurement errors, temporal variations, and the volatile R&D environment. Therefore, this study employs robust DEA methods to handle such data uncertainty for reliable R&D performance evaluation.

4.2 Analysis of efficiency results

Before running equation (10), it is necessary to set the value of the uncertainty budget and the level of data perturbation. Following previous studies (Hatami-Marbini and Arabmaldar 2021; Arabmaldar, Sahoo and Ghiyasi 2023), we assume that all data is subject to uncertainty and that the data perturbation level is uniform, i.e. $ei=er=e$⁠. Furthermore, we simplify the setting of the uncertainty budget by assuming $Γix=Γry=Γ$⁠. It is worth noting that our proposed robust model is highly flexible. In practice, decision-makers can adjust the level of perturbation and the uncertainty budget according to their optimistic preferences.

Since our model is built on an optimistic perspective, a larger data perturbation will yield a more optimistic assessment. Given that China currently faces primarily negative uncertainties, overly optimistic decisions are inappropriate. Therefore, we set $e=1%$⁠, which is a conservative perturbation for CIEs with large input and output sizes. In addition, following Bertsimas and Sim (2004), we set $Γ=1+Φ-11-en$⁠, where $Φ-1⋅$ is the inverse function of the standard normal distribution and $n$ is the number of uncertain variables. Bertsimas and Sim (2004)’s approach can ensure the robust solution of equation (10) is feasible with high confidence. Table 4 presents the R&D efficiencies of CIEs and their rankings (in parentheses) for 30 regions in China from 2018 to 2022.

4.2.1 Analysis from a nationwide perspective

Figure 2a illustrates the evolution of the average R&D efficiency of CIEs at the national level. From 2018 to 2020, there was a steady annual decline in average R&D efficiency, decreasing from 0.784 in 2018 to 0.747 in 2020. During this period, uncertainties in the trade environment, exacerbated by the US-China trade war, prompted CIEs to adopt a more cautious approach toward R&D investment. Particularly affected were enterprises facing technological restrictions imposed by the USA, including disruptions in supply chains due to limitations on importing critical technologies and raw materials. Additionally, tariffs and trade barriers contributed to a decline in CIEs’ competitiveness in the international market, thereby impacting profit margins and exacerbating declines in R&D efficiency.

The onset of the COVID-19 pandemic in early 2020 further intensified disruptions in global supply chains, prompting many CIEs to scale back or suspend production and R&D activities significantly. In response, the Chinese government swiftly implemented multi-dimensional support policies. Fiscally, they introduced tax incentives, subsidies, and dedicated R&D funding while encouraging local governments to establish special relief funds. Financially, they strengthened credit support through measures such as increasing credit loans, lowering loan interest rates, and allowing loan extensions and renewals. In terms of innovation, they supported enterprise digital transformation, promoted internet platform services and intelligent manufacturing, and fostered integrated development among large, medium, and small enterprises. These policy interventions effectively stabilized CIEs’ operations and facilitated their adaptation to the new economic environment. Moreover, the government proposed a new development strategy emphasizing domestic circulation while integrating domestic and international markets. Consequently, from 2020 onwards, CIEs shifted their focus toward expanding in the domestic market. Another significant point is that the COVID-19 pandemic objectively accelerated the internal adoption of digital and intelligent manufacturing technologies within these enterprises, leading to a more effective allocation of research and development resources. These factors collectively drove a significant recovery in CIEs’ R&D efficiency, which rebounded markedly from 0.747 in 2020 to 0.851 in 2021 and remained at a relatively high level of 0.846 in 2022.

Analyzing the distribution of efficiency helps to gain a deeper understanding of the dynamics of CIEs’ R&D efficiency. Figure 2b depicts the kernel density curve, quantiles, and scatter distribution of efficiency from 2018 to 2022. The distribution curve of efficiency exhibited a prominent bimodal characteristic in 2018, with a relatively narrow peak above and a wider one below. This suggests that only a small portion of regions had higher R&D efficiency, while most regions were clustered at lower efficiency levels. With the impact of the US-China trade war and the pandemic, the R&D efficiency of CIEs continued to deteriorate. Consequently, the peak representing relatively higher efficiency began to move downward and gradually merged with the lower peak. By 2020, the density curve of CIEs’ R&D efficiency had evolved into a unimodal feature. After 2020, some regions’ CIEs have managed to stem the decline in efficiency and achieve significant improvement. This led to a resurgence of a double-peak pattern in the efficiency density curve, indicating a resurgence in regional disparities in R&D efficiency. This underscores the importance of a detailed examination of regional variations in CIEs’ R&D efficiency to identify the factors driving these disparities.

Overall, the above empirical analysis of CIEs during 2018–22 reveals how industrial enterprises’ R&D efficiency is impacted by and responds to major external shocks. While CIEs experienced declining R&D efficiency during 2018–20 due to international trade frictions and the global pandemic, the effectiveness of their recovery strategies—including policy support, market reorientation, and digital transformation—demonstrates valuable lessons for enhancing industrial R&D resilience across different national contexts.

4.2.2 Analysis from a regional perspective

Analyzing regional efficiency disparities not only enables low-performing regions to learn from successful practices in high-efficiency areas, but also provides valuable insights for R&D policy formulation. Since the reform and opening-up, China’s economic development has undergone profound changes. Existing research has predominantly focused on the regional disparities among China’s traditional eastern, central, and western regions, overlooking significant variances within each region. For instance, the eastern region has developed economic centers such as Shanghai, as well as economically backward regions such as Hainan. Therefore, to avoid drawing over-generalized conclusions, this section analyzes regional differences in efficiency under the perspective of eight comprehensive economic zones (ECEZs).

Following Cheng et al. (2024), we further subdivide the 30 provinces into ECEZs, as shown in Table 5. Figure 3 illustrates the evolution of the average R&D efficiency of ECEZs.

Based on Fig. 3 and Table 5, we present the following analysis.

Among ECEZs, the East Coast and the Middle Yellow River comprehensive economic zones do not reject the null hypothesis. This indicates that the development of CIEs within these regions is relatively balanced. Despite the insignificant distribution differences in both regions, the underlying reasons are distinct. As can be seen from Fig. 3, the former demonstrates generally high and balanced R&D efficiency, whereas the latter shows uniformly low R&D efficiency. The East Coast comprehensive economic zone stands as China’s most developed economic and technological hub. Its ample capital, advanced technology, and abundant human resources provide a solid foundation and support for CIEs within this region. Additionally, the regional policies implemented by the Chinese government for the integrated development of the Yangtze River Delta promote the efficient flow and sharing of R&D resources within the East Coast, contributing to its balanced development. In contrast, constrained by weak economic foundations, traditional industrial structures, insufficient policy support, and a shortage of talent resources, the Middle Yellow River comprehensive economic zone exhibits lower R&D efficiency.

Except for the two comprehensive economic zones mentioned above, the null hypothesis is rejected in the other six comprehensive economic zones. This indicates that regional development in China still exhibits significant imbalance. Furthermore, it should be recognized that the reasons for low efficiency vary across different regions. Since our model can differentiate efficient DMUs, provinces with lower R&D efficiency can always use the best-performing DMU within the same comprehensive economic zone as a benchmark for efficiency improvement.

The above findings highlight several potential pathways for enhancing regional R&D efficiency in industrial enterprises. First, the success of high-performing regions demonstrates the critical role of establishing comprehensive innovation ecosystems, encompassing capital availability, technological infrastructure, and human capital development. Second, policy frameworks promoting regional integration and resource sharing, as exemplified in the East Coast comprehensive economic zone, offer a replicable model for fostering balanced development across regions. Third, the identification of region-specific benchmarks through efficiency analysis provides a practical approach for targeted improvement strategies, allowing regions to learn from contextually relevant best practices rather than pursuing one-size-fits-all solutions. Such multi-dimensional improvement paths can be particularly valuable for regions seeking to address R&D efficiency gaps while accounting for their unique developmental contexts.

4.3 Sensitivity analysis

The level of uncertainty in the data has a substantial impact on the efficiency results. However, there is no way to determine a precise level of data perturbation. To test the sensitivity of CIEs’ R&D performance to the level of data perturbation, this section additionally considers two cases, $e=3%$ and $e=5%$⁠. Figure 4 compares the trends in the average R&D efficiency of CIEs under the three data perturbations. As depicted in Fig. 4, the efficiency trends exhibit consistency across varying data perturbations. This indicates that the constructed model is robust enough to generate consistent results despite data uncertainty. Moreover, this consistency can also indicate that the main factors affecting the R&D efficiency of CIEs are consistent under different data perturbations, such as trade wars, supply chain disruptions, and the impact of epidemics. This further underscores the significant impact of these uncertainties on CIEs’ R&D efficiency.

To further validate the robustness of our analysis findings, Fig. 5 presents a scatter plot depicting the correlation of efficiency rankings across different levels of data perturbation. The plot reveals a notable positive correlation in efficiency rankings across varying degrees of data perturbation. However, this correlation appears to diminish with higher levels of data perturbation. This phenomenon occurs because obtaining an accurate model solution becomes more challenging as data uncertainty increases. Consequently, the volatility of efficiency rankings also rises under these conditions.

5. Model comparison

To demonstrate the superiority of our model, this section provides a comparative analysis focusing on feasibility and computational scale.

5.1 Feasibility analysis

Existing robust SEDEA models predominantly employ radial approaches, which may encounter infeasibility issues under VRS (Seiford and Zhu 1999). To investigate this phenomenon in the context of our study, we apply the classical robust SEDEA method proposed by Arabmaldar, Jablonsky and Hosseinzadeh Saljooghi (2017) to our dataset under VRS. The results, presented in Table 6, indicate that the model of Arabmaldar, Jablonsky and Hosseinzadeh Saljooghi (2017) indeed experiences infeasibility for certain DMUs under VRS. In contrast, Table 4 demonstrates that our proposed method is not only feasible but also yields lower efficiency scores. This can be attributed to our non-radial and non-oriented approach, which accounts for maximum inefficiency in each input and output. Consequently, our method provides decision-makers with more objective and reliable performance assessments.

5.2 Computational scale analysis

In the previous modeling section, we integrated the Robust SupERM and Robust ERM models using a binary variable. To demonstrate the necessity of this integration, we first compare the size of the models, which can be measured by the number of constraints and decision variables (Tone, Toloo and Izadikhah 2020). Table 7 shows the number of constraints and decision variables for Robust SupERM [equation (8)], Robust ERM [equation (10)] and Integrated Robust ERM [equation (11)].

Figure 6 illustrates how the number of decision variables and constraints changes with the number of DMUs in different models. For simplicity, we set $m=s=3$⁠. As shown in Fig. 6, the number of decision variables and constraints increases significantly with the number of DMUs in all models. Additionally, our model has nearly half the number of decision variables and constraints compared to the Robust SupERM + Robust ERM model, which leads to a lower computational burden.

6. Conclusion and implications

6.1 Conclusion

In recent years, global industrial and supply chains have faced considerable uncertainty due to various factors such as trade wars, geopolitical risks, and pandemics. This volatility has created an increasingly complex and unpredictable R&D environment, making it challenging to measure the true performance of R&D activities using precise data. To address this challenge, this study develops a non-radial robust super-efficiency DEA approach to evaluate R&D performance under data uncertainty. The comparative analysis of the models demonstrates the advantages of our proposed model in terms of feasibility and computational efficiency. While this paper demonstrates the model’s application using Chinese industrial enterprises, the findings could offer valuable implications for countries seeking to develop R&D performance improvement strategies under uncertain environments.

Firstly, the empirical analysis reveals how Chinese industrial enterprises’ R&D performance responds to major external shocks. The observed pattern—an efficiency decline during 2018–20 followed by a post-2020 recovery—demonstrates the interplay between external challenges (trade conflicts, the pandemic) and response mechanisms, including policy support, market reorientation, and digital transformation. The effectiveness of these response strategies in the Chinese context offers valuable lessons for other countries in maintaining R&D resilience during periods of significant external disruption. The consistent results across various levels of data perturbation further validate our model’s reliability in uncertain environments.

Secondly, the regional analysis provides insights into the role of innovation ecosystems in shaping R&D performance. The contrasting experiences of different regions—particularly between the technologically advanced East Coast and the Middle Yellow River zone, which is traditionally industrial—highlight how factors such as technological infrastructure, human capital, and policy frameworks collectively influence R&D performance. These findings suggest that improving industrial R&D performance requires a holistic approach addressing both hard infrastructure (e.g. technological facilities) and soft capabilities (e.g. human capital and policy support).

6.2 Implications

Drawing on the findings, we propose two key implications with relevance beyond the Chinese context:

Firstly, establishing a resilient R&D management system under globalization is essential for firms to address external uncertainties. The empirical evidence demonstrates that in response to external shocks such as trade conflicts and pandemics, firms can restore R&D efficiency through various mechanisms, including institutional support, strategic market repositioning, and digital transformation initiatives. These findings indicate the necessity of implementing systematic risk response mechanisms that encompass risk monitoring systems, resource allocation optimization, and digital transformation strategies. Such mechanisms enable firms to sustain R&D activities and maintain operational effectiveness in dynamic environments, thereby fostering resilient innovation capabilities.

Secondly, fostering synergistic innovation ecosystems is crucial for sustainable R&D performance. The analysis reveals that regional disparities in R&D performance primarily derive from the comprehensive strength of innovation ecosystems, encompassing technological infrastructure, human capital accumulation, and institutional support. This finding suggests two critical implications: firms should strategically embed themselves within regional innovation networks to leverage external innovation resources; meanwhile, policymakers should adopt an integrated approach to develop both tangible infrastructure and intangible capabilities. Through the establishment of industry-university-research collaboration mechanisms, knowledge flows and technological spillovers can be facilitated, ultimately cultivating a sustainable innovation ecosystem.

6.3 Limitations and future directions

Despite the contributions of this study, several limitations warrant attention in future research. Firstly, given the growing importance of environmental concerns, future studies could incorporate undesirable outputs to analyze green R&D performance. Secondly, exploring how various types of external shocks impact R&D efficiency across different national contexts could enhance our understanding of industrial resilience. Lastly, investigating how different policy frameworks influence R&D efficiency could provide valuable insights for policymakers globally in designing effective industrial innovation policies.

Supplementary data

Supplementary data are available at Research Evaluation Journal online.

Funding

This work was supported by the Postgraduate Research & Practice Innovation Program of Jiangsu Province (no. KYCX23_0409) and the China Scholarship Council Program (no. 202306830166).

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data can be found in the supplementary materials.

Notes

References