Accuracy of Whitebark Pine and Limber Pine Identification by Forest Inventory and Analysis Field Crews

Abstract

Accurate identification of whitebark and limber pine has become increasingly important following the 2022 listing of whitebark pine as a threatened species under the Endangered Species Act. However, morphological similarities make identification of the two species difficult where ranges overlap. Using a genetic test that differentiates whitebark and limber pine, we compared field identification by Forest Inventory and Analysis field crews with genetic identification for needle samples from 371 trees. Field identifications were 100% correct for the 76 samples collected from outside regions of species’ range overlap. A total of 83% of the field identifications were correct in regions of range overlap (89% for large trees, 88% for saplings, and 78% for seedlings). Field-identified samples were correct 60% of the time for limber pine and >99% for whitebark pine. Random forests analysis revealed that identification accuracy is influenced by crew experience, large (≥ 12.7cm diameter) limber or whitebark pines recorded by field crews on the plot, elevation, Julian day of sample collection, and habitat type. We found that whitebark pine has likely been underestimated, and limber pine overestimated, within their overlapping ranges. We provide insights on improving accuracy of future monitoring where these species overlap.

Study Implications: Accurate identification of whitebark pine is critical for monitoring this threatened species, yet distinguishing whitebark from limber pine can be difficult. Genetic analysis determined accuracy of field identification by Forest Inventory and Analysis (FIA) crews was 83% where the species’ ranges overlap. Virtually all individuals identified as whitebark pine were genetically confirmed to be whitebark pine, although nearly 40% of individuals identified as limber pine were actually whitebark pine. Thus, previous data underestimated whitebark and overestimated limber pine abundance in the species’ range overlap. These results quantify reliability of FIA data for whitebark pine assessments and identify areas for improvement.

High-elevation five-needle pines, which include whitebark pine (Pinus albicaulis Engelm.) and limber pine (Pinus flexilis James), play important roles in high-mountain ecosystems across western North America. These species provide a host of ecological and cultural benefits, including serving as keystone or foundational species (Tomback et al. 2011), providing wildlife food and habitat, creating and sustaining biodiversity, facilitating succession, regulating snowpack melt and runoff, reducing soil erosion, and contributing to aesthetic and spiritual values (Steele 1990; Tomback and Achuff 2010; Tomback et al. 2001, 2011; Windmuller-Campione and Long 2016).

Declines in whitebark and limber pine have been observed throughout most of their ranges over the past several decades (Burns et al. 2023; Goeking and Izlar 2018; Goeking and Windmuller-Campione 2021; Tomback et al. 2001, 2011), with higher mortality rates in recent years (Burns et al. 2023; Goeking and Windmuller-Campione 2021). White pine blister rust (caused by Cronartium ribicola Fisch.), mountain pine beetles (Dendroctonus ponderosae Hopkins), altered fire regimes, forest succession, climate change, and interactions among these factors have caused declines in both species (Burns et al. 2023; Campbell et al. 2011; Cleaver et al. 2015; Gibson et al. 2008; Keane et al. 2022; Tomback and Achuff 2010; Tomback et al. 2011). Because of these ongoing threats, whitebark pine was listed as threatened under the Endangered Species Act by the US Fish and Wildlife Service in December 2022 (US Fish and Wildlife Service 2022). Goeking and Windmuller-Campione (2021) noted that limber pine population trends are comparable to the trends exhibited by whitebark pine less than 10 years earlier.

Reliable inventory data are required for scientific studies and for monitoring the status of these species (Keane et al. 2022; Tomback and Sprague 2022). However, the similarity between whitebark and limber pine needles, bark, and growth form make it very difficult to distinguish the two species based on vegetative characteristics alone (Alongi et al. 2019; Arno and Hammerly 2007; Hansen et al. 2021). When present, pollen and seed cones of whitebark and limber pine are sufficiently distinct in color, morphology, and persistence that the two species can be reliably distinguished (Arno and Hoff 1989; Lesica 2012; McCaughey and Schmidt 2001). However, cone-bearing trees of both species are absent from many sites due to widespread mortality (Keane et al. 2022; Tomback and Achuff 2010), branch dieback (i.e., branch death) caused by white pine blister rust (Cleaver et al. 2015; McKinney and Tomback 2007; Tomback and Achuff 2010), seed predation, and unsuitable weather conditions (Hansen et al. 2021). Additionally, these slow-growing species can take 20–40 years to produce cones, and large cone crops do not typically occur in whitebark pine until trees are at least 60 to 80 years old (Arno and Hammerly 2007; Krugman and Jenkinson 1974). Thus, cones are often absent from younger trees.

The challenge of identifying young trees is exacerbated in disturbed areas, where both species may be pioneering, and in other forest types without five-needle pines in the overstory. In both circumstances, long-distance seed dispersal by Clark’s nutcrackers (Nucifraga columbiana Wilson) may lead to whitebark pine or limber pine regeneration in these community types if conditions are suitable (Lorenz et al. 2011; Tomback and Linhart 1990; Tomback et al. 2011). Nutcrackers typically disperse seeds up to a few kilometers, routinely transport seeds 12 to 22 km (Tomback and Linhart 1990), and may disperse seeds as far as 32 km (Lorenz et al. 2011), often over large elevation gradients (Lorenz et al. 2011; Tomback and Linhart 1990; Tomback et al. 2011). This dispersion results in regeneration of whitebark and limber pines on sites that may not have trees of these same species in the overstory (Goeking and Izlar 2018; Goeking and Windmuller-Campione 2021; Goeking et al. 2019; Windmuller-Campione and Long 2016). Climate and substrate are also unreliable indicators for species identification, as their effects on species occurrence vary geographically (Arno and Weaver 1990; Hansen-Bristow et al. 1990; McCaughey and Schmidt 2001; Weaver 2001). Additionally, co-occurrence and elevational overlap between whitebark and limber pines has been documented in many areas (Arno and Hoff 1989; Hansen et al. 2021; Resler and Tomback 2008; Weaver 2001).

The need to better understand these declining and important species has led to several approaches for addressing identification challenges. For example, some regional monitoring protocols require crews to establish plots in alternate locations if whitebark and limber pine cannot be distinguished in the field at the initial plot location (Greater Yellowstone Whitebark Pine Monitoring Working Group 2011). Whitebark and limber pines are lumped as “high-elevation five-needle pines” in results from USDA Forest Service (Forest Service) Forest Health Protection aerial surveys (Hayes 2016). Tomback et al. (2005) required field crews to be familiar with cone-based species identification in geographic areas with both species, and Resler and Tomback (2008) used needle morphology to distinguish the two species, which requires specialized equipment and knowledge. These methods ensure certain levels of data quality, meet study objectives, and use available resources for identification. However, these approaches are not realistic or reliably effective for field campaigns using probability-based species-specific sampling for all tree ages and species, and new resources (i.e., the genetic testing method detailed in Alongi et al. 2019) have recently become available.

The Forest Service Forest Inventory and Analysis (FIA) conducts a probability-based, statistically representative sample of whitebark and limber pine occurrence that is frequently used for modeling species distribution and responses to climate change (e.g., Bell et al. 2014; Schoettle et al. 2022; Warwell et al. 2006; Windmuller-Campione and Long 2016). The FIA data constitute the primary data source throughout whitebark pine’s range in the United States and were cited in the decision to list whitebark pine as threatened (US Fish and Wildlife Service 2022). FIA data continue to be used by the Forest Service to assess impacts of management on whitebark pine and by researchers who are developing future range projections that will guide restoration efforts. This ongoing reliance on FIA data for whitebark pine assessments assumes that species identification is accurate. Although FIA conducts expert checks for quality control (Gray et al. 2012), in situ quality control is hampered when species cannot be conclusively identified in the field. To improve quality control, we analyzed an existing sample of whitebark and limber pine needles collected from FIA plots using an economical genetic test that differentiates the two species (Alongi et al. 2019).

The objectives of this study were to (1) assess identification accuracy for whitebark pine and limber pine by Rocky Mountain Research Station (RMRS)-FIA field crews, (2) identify factors that predict when misidentification is most likely to occur, and (3) describe steps that can improve future FIA identification of these two species. We hypothesized that difficulty in distinguishing whitebark and limber pine when cones are absent would lead to misidentification, and we predicted that these errors would be more common for seedlings. By examining patterns in misidentification, we identified additional causes of identification errors and provide recommended procedures to improve future data quality.

Materials and Methods

Study Area

Our study area encompassed regions of whitebark and limber pine range overlap within the states of Montana, Idaho, Wyoming, and Nevada. To define these regions, we first created comprehensive range maps for each species. Maps were built by joining location data from FIA plots containing whitebark and limber pine (as described in Burrill et al. 2023) with digitized range maps developed by the Whitebark Pine Ecosystem Foundation (2014 and 2019) and Forest Service Forest Health Protection (Forest Health Technology Enterprise Team 2023). We applied a 35 km buffer around each comprehensive range map to encompass the maximum seed dispersal distance of Clark’s nutcrackers (Lorenz et al. 2011). Using the intersect tool in ArcGIS, we isolated the area where the buffered ranges overlap. Finally, we eliminated any areas that fell outside the footprint of RMRS-FIA (figure 1). The resulting area excluded range overlap in central and southern Sierra Nevada in California, the Wallowa Mountains of northeastern Oregon, and a band that runs through southeastern British Columbia and southwestern Alberta to about 54° N (Tomback et al. 2011). The area of species’ range overlap, hereafter referred to as the study area, covered approximately 75% of whitebark pine range within the RMRS-FIA data collection states and included 89% of RMRS-FIA plots with whitebark pine trees (>12.7 cm diameter). A total of 50% of limber pine range within the RMRS-FIA data collection states and 50% of RMRS-FIA plots with limber pine trees (> 12.7 cm diameter) were in the study area.

Data Collection on FIA Plots

The FIA plot grid consists of approximately one plot per 2,428 ha, comprising a probability-based, statistically representative sample across all land cover types, forest types, and ownership groups (Bechtold and Patterson 2005; Burrill et al. 2023). Each year, 10% of FIA plots within the RMRS-FIA footprint are targeted for visit and numerous site, stand, and tree variables are measured (USDA 2010). Each FIA plot consists of four 7.3 m radius subplots, covering 672 m² total. All standing live and dead trees ≥12.7 cm diameter (hereafter referred to as ‘large trees’) in the subplot are identified to species and numerous measurements are taken. Nested within each subplot is a 2.1 m radius microplot, wherein field crews measure all saplings (≥2.5 cm and <12.7 cm diameter) and record the number of seedlings per species (Pinus spp. seedlings are <2.5 cm diameter and >15.2 cm height) (USDA 2010). Diameters are measured at breast height (1.37 m above ground level) for whitebark pine and limber pine (see USDA 2010 for more detail).

Needle Samples

From 2010 to 2012, RMRS-FIA field crews identified all tree species present on each subplot and collected one tissue sample per species from the subplot. Samples were collected from individuals with the following order of preference: (1) large trees on the subplot or saplings on the microplot, (2) seedlings on the microplot, (3) large trees immediately adjacent to the subplot or saplings and seedlings located off the microplot but on the subplot, and (4) saplings or seedlings immediately adjacent to the subplot. This method resulted in at least one sample for each field-identified species found on each plot, with a maximum of four samples for each species per plot. Each five-needle pine sample consisted of at least one fascicle of five needles. Samples were placed in envelopes with a small amount of silica gel cat litter and stored in cardboard boxes at the RMRS-FIA laboratory in Ogden, Utah, until 2020. Although all species were sampled, this study opportunistically used samples from field-identified whitebark and limber pines.

A total of 371 samples from 205 plots had conclusive genetic test results. Of these, 295 samples were collected from 165 plots located within our study area (figure 1). These 295 samples represent about 2% of the live whitebark and limber pine trees, saplings, and seedlings measured on FIA plots located within the study area. The field identifications of the remaining seventy-six samples collected outside of the study area were verified as 100% accurate using the genetic identification tool described below and were therefore excluded from our analyses of identifications errors.

Accuracy of Species Identification

To test species identification by field crews, genetic analyses were conducted in 2020 and 2021 by the Plant Sciences and Plant Pathology Department laboratory at Montana State University using the genetic identification tool detailed in Alongi et al. (2019). This tool uses two chloroplast loci to expeditiously distinguish needle samples of whitebark and limber pine trees. This approach is justifiable, as opposed to a genomic approach, for several reasons. The first is that introgressive hybridization between these two species is not known to occur. Ericson (1965), Bingham (1973), and Critchfield (1986) report cursory to uncertain evidence for whitebark and limber pine putative hybrids, either from single trees or small population samples, and these include results from artificial crosses. These studies provide no evidence of introgressive hybridization between whitebark and limber pine. Second, evidence of introgression between species typically involves delimitation of a hybrid zone that lies in the region of contact between the geographical distributions of two putative parental species, as exemplified by limber pine and southwestern white pine (Pinus strobiformis Engelm.; Menon et al. 2018) and whitebark pine and sugar pine (Pinus lambertiana Douglas; Liston et al. 2007). No such hybrid zone has been detected in the region of overlap for whitebark and limber pine (e.g., as mapped by Little 1971; Ulrich et al. 2023). Within this region of overlap, Alongi et al. (2019) and follow-up analyses found complete congruence between limber and whitebark pine trees identified using seed-cones and chloroplast haplotypes, these comparisons mostly involving blind tests. Thirdly, our approach would be excessively encumbered by a genomic approach to distinguishing whitebark and limber pine. Genomic analyses have been performed on limber pine (Liu et al. 2016a) and whitebark pine (Liu et al. 2016b; Syring et al. 2016). However, both limber and whitebark pine have large genomes (30.5 Gbp [Lui et al. 2016a] and ~27 Gbp [Syring et al. 2016]) that could not be affordably analyzed at a population level (Syring et al. 2016).

We determined the species identification accuracy for each sample by comparing the field identification written on the sample envelope (referred to as “field-identified”) to the genetic identification (referred to as “lab-identified”). Accuracy was then calculated as the percentage of samples that were correctly field-identified. Of the 295 study area samples, 181 were sampled from large trees, saplings, or seedlings that were measured as a standard part of plot data collection (categorized as “tally” samples), and 114 were sampled from large trees located off the subplot or saplings or seedlings located off the microplot (“nontally”).

To further understand patterns in misidentification, we created a confusion matrix containing data from all samples, and for each size class (seedlings, saplings, large trees, and unknown size). Confusion matrices display “actual” versus “predicted” values for two or more classes (in this study lab-identified vs. field-identified) and identify errors of commission and omission. For each confusion matrix, we calculated false-positive error rates, sensitivity, and specificity (James et al. 2013) for both whitebark and limber pine:

False-positive error rates identify errors of commission (Type 1 error), e.g., the proportion of limber pine trees that are incorrectly field-identified as whitebark pine. In our study, sensitivity for a species indicates the ability of FIA field crews to correctly field-identify a sample that is lab-identified as that species. Specificity for a given species indicates the field crews’ ability to correctly field-identify a sample that is not the given species (i.e., it is lab-identified as the other species). Because our study examined only two species, the specificity of one species was equal to the sensitivity of the other.

Predictors of Identification Accuracy

To identify predictors of misidentification errors, we developed a list of twelve potential predictors based on guidance given to RMRS-FIA field crews and on a literature review (Table 1). These predictors were grouped into three types of variables: (1) observer-related variables, (2) tree-level variables associated with the presence of cones, and (3) site-level variables that influence species distribution patterns or the likelihood of encountering both species (Table 1).

Crew leader experience was calculated as the number of plots with whitebark or limber pine in the study area that a crew leader had measured at the time of sample collection. We aggregated habitat type, which describes the potential (late seral or climax) natural vegetation based on climate, geography, or other site characteristics (Pfister et al. 1977), to the series level (Burrill et al. 2023), resulting in five habitat type series. We aggregated field-recorded forest type (i.e., FLDTYPCD in Burrill et al. 2023), which indicates the predominant live tree species or group of species in the stand (USDA 2010), to six forest-type groups (Burrill et al. 2023). Whitebark pine seed zones defined by Mahalovich and Hipkins (2011) encompass areas of similar environmental conditions and allowed comparison of accuracy rates among areas differing in geology, topography, climate, and vegetation. The thirteen samples collected outside of mapped seed zones were placed into the category of “outside.” We aggregated lithology types from the State Geological Map Compilation database (Horton et al. 2017) to five groups. We determined whether a sample was collected east or west of the Continental Divide by using spatial analyses on a geodatabase feature class of sample locations and a shapefile of the Continental Divide (U.S. Geological Survey 2002). We initially performed analyses using actual elevation and using equivalent elevation, which accounts for the latitudinal range of our study area (Windmuller-Campione and Long 2016), but we report only results based on actual elevation due to lack of significant differences between the two metrics.

To identify important predictors of identification accuracy, we developed a random forests model in R package randomForests (R Core Team 2016) using a classification threshold of 0.5 (James et al. 2013), with 500 trees for evaluation (parameter ntree = 500 and mtry = 3). We used ten-fold cross-validation to evaluate model performance. We chose the random forests model because it provides intuitive interpretation of variable importance, is robust to collinearity and interaction among predictor variables, and makes no a priori assumptions about the distributions of response or predictor variables (Cutler et al. 2007). Significant predictors were identified by their corresponding decrease in random forests model accuracy (i.e., not identification accuracy) and Gini index (Cutler et al. 2007; James et al. 2013). Mean decrease in accuracy measures the contribution of each variable to the overall accuracy of the model, whereas the Gini index is a measure of node purity (i.e., how accurate is each decision point, or node) within the classification tree (James et al. 2013). The Gini index method of calculating variable importance results in a strong preference for continuous variables or those with many categories (Strobl et al. 2007).

To complement modeling, we examined the relationship between predictor variables and identification accuracy. We compared field versus lab-identified whitebark pine presence across different predictor variable values or categories to better understand the circumstances in which misidentification is most likely to occur. Wilcoxon rank sum tests (for continuous predictor variables) and χ² tests of association (for categorical predictor variables) were performed to test for significant associations between identification accuracy and the predictor variables (Zar 1996) (critical p-value = .05 for each method). We calculated Cramer’s V scores to determine the strength of association with each categorical predictor variable. Cramer’s V scores quantify the strength of the association among two categorical variables, which in our analysis were done using χ² tests, where the values of V range from zero (no association) to one (perfect association) (Cohen 1988). Values less than or equal to 0.3 indicate a weak association, those between 0.4 and 0.5 a medium association, and those greater than 0.5 a strong association. To better understand effects of crew experience, we compared cumulative accuracy for the field crew leader collecting a given sample to that crew leader’s experience as of that date. Cumulative accuracy was calculated as mean accuracy of all samples across all plots that a crew leader had collected up to and including each sampling date.

Results

Field Identification Accuracy

Genetic analyses of the 295 needle samples collected within the study area revealed that 218 samples came from whitebark pines, and 77 samples came from limber pines. Overall field identification accuracy was 83% (Table S1 provides details for each misidentified sample). Greater than 99% of samples identified as whitebark pine by field crews were genetically confirmed as whitebark pine. The primary source of error was misidentification of whitebark pines as limber pines, indicated by the 40% false-positive rate for limber pine (Table 2), which meant that field crews detected 77% of sampled trees genetically verified as whitebark pines. In contrast, only a single limber pine was misidentified as a whitebark pine. Although identification accuracy was similar for large trees (89%) and saplings (88%), it was notably lower for seedlings (78%). More than half of seedlings identified as limber pine by field crews were whitebark pine, and only 73% of whitebark pine seedlings were correctly identified (Table 2).

Predictors of Identification Accuracy

Our random forests model correctly classified 88% of observations, based on the predictors in Table 1, and had a Cohen’s Kappa value of 0.53, which is generally considered to indicate moderate agreement between actual and predicted values (Cohen 1988). The two indices used to evaluate relative variable importance (mean decrease in accuracy and Gini index) produced different results. Both metrics indicated that crew leader experience had the most impact on identification accuracy. However, large limber pine trees recorded by field crews on the plot and large whitebark pine trees recorded by field crews on the plot had the second and third largest variable importance scores based on mean decrease in accuracy. Elevation and Julian day had the second and third largest variable importance scores based on the mean decrease in Gini index (figure 2a). The method of calculating variable importance for Gini index likely contributed to the increased relative importance of the three continuous variables (see Methods section for details).

The results of χ² tests of association (Table 3) and Wilcoxon rank sum tests detected significant relationships between identification accuracy and several variables also identified as important by the random forests model (see Table 1 for predictor variable descriptions). For categorical variables, significant associations with identification accuracy were detected for large whitebark pine trees recorded by field crews on the plot, large limber pine trees recorded by field crews on the plot, lithology type, seed zone, forest type group, and size class. Cramer’s V scores were highest for large whitebark pines recorded by field crews on the plot (0.346) and large limber pines recorded by field crews on the plot (0.262), suggesting moderate and weak associations, respectively, with identification accuracy. Despite its moderate importance in the random forests model, habitat type was not significantly associated with identification accuracy according to the χ² test of association. Among our continuous predictors, Wilcoxon rank sum tests indicated that only crew leader experience had a significant effect on identification accuracy (p < .001).

Patterns in Misidentification

We quantified the likelihood of misidentification with respect to three types of variables: observer-related, tree-level, and site-level variables. Analysis of observer-related variables indicated that the experience level of the crew leader was an important predictor of identification accuracy. For correctly identified samples, crew leaders had completed an average of forty-nine whitebark/limber pine plots in the study area, whereas for incorrectly identified samples, crew leaders had completed an average of twenty-six plots (p < .001, Welch’s t-test). Our ability to test whether accuracy increased as individual crew leaders gained experience was limited by low sample sizes from many crew leaders (81% of crew leaders collected samples from fewer than ten plots). However, a broad range of experience across all crew leaders revealed a strong correlation (p < .001, Spearman rank correlation) between crew leader experience and cumulative accuracy, both among crew leaders and even for some individuals as their experience increased (figure 3). Crew leaders who had measured fewer than twenty-five plots had highly variable cumulative accuracy (56% correct ±50% SD), whereas more experienced crew leaders were more consistently accurate (91% correct ±29% SD). In contrast to crew leader experience, the other observer-related variable, tally versus nontally, had no significant effect on identification accuracy. This indicates that although some samples were collected from trees that were not included in the FIA dataset, the accuracy rates and trends were still representative of FIA data.

We assessed how cone presence influenced identification accuracy using two tree-level variables associated with the likelihood of cone presence: tree size class and Julian day. To accomplish this, we compared accuracy rates among size classes and tested for relationships between accuracy and Julian day. The clear differences in accuracy among samples collected from seedlings versus from large trees and saplings in our confusion matrices (Table 2) were not supported by the low relative importance of “Size_Class” in the random forests model (figure 2) and low Cramer’s V score (0.176) in the χ² test (Table 3). This may be due to similar accuracies for large trees and saplings, which likely dampened the effect of seedlings. Julian day, which is an indicator of cone phenology and development, was one of the most important predictors of identification accuracy in the random forests model. A partial dependence plot of Julian day versus identification accuracy revealed that the likelihood of correct identification was relatively low prior to mid-July, reached a peak from mid-August through late-September, and then declined (figure 2b).

The most important site-level predictors of identification accuracy were whether large trees of either species (“PIAL_Trees”) or (“PIFL_Trees”) were recorded by field crews on the plot, as confirmed by the results of the random forests model (figure 2), χ² tests of association (Table 3), and comparison of field-identified versus lab-identified species frequencies (Table 3). We found substantially higher identification accuracy for samples (regardless of tree size) collected from plots where the large trees were recorded as whitebark pine (>99% accurate), compared with those where they were recorded as limber pine (66% accurate). In addition, all fifty-six plots where field crews recorded large whitebark pine trees did actually contain lab-identified whitebark pine, whereas only thirty-two of forty-three plots where field crews recorded large limber pine trees actually contained lab-identified limber pine.

We detected associations between identification accuracy and several other site-level variables, although these were weaker than those for field-identified presence of large whitebark and limber pines. Identification accuracy rates varied widely between seed zones, from 91% accuracy in the Inland Northwest (INLA) seed zone to 72% in the Greater Yellowstone–Grand Teton (GYGT) zone (Table 3). Lower identification accuracy within the GYGT seed zone is also associated with sample collection by crew leaders who were less experienced in measuring whitebark and limber pine (figure 3).

Identification accuracy was highest in the whitebark pine and limber pine forest type groups (Table 3). Samples from whitebark pine forest types were dominated by whitebark pine (96% of samples lab-identified as whitebark pine) and contained just one misidentification. Similarly, samples from limber pine forest types were dominated by limber pine (93% of samples lab-identified as limber pine), and also contained just one misidentification. The forest type groups with the highest rates of misidentification were the lodgepole pine and the fir/spruce/mountain hemlock (Table 3).

Among habitat type groups, the whitebark pine-timberline and limber pine habitat series had the highest accuracy rates (96% and 88%, respectively), exhibiting a pattern similar to the whitebark pine and limber pine forest type groups. Identification accuracy was lowest (79%) in the subalpine fir series, with a large discrepancy between percent of samples that were lab-identified (98%) versus field-identified (77%) whitebark pine. We detected a strong positive association between whitebark pine occurrence and the presence of subalpine fir (Abies lasiocarpa [Hook.] Nutt.); whitebark pine was associated with both the subalpine fir habitat type series (98% of samples lab-identified whitebark pine) and the whitebark pine-timberline series (96% of samples lab-identified whitebark pine), which includes several habitat types that typically contain a subalpine fir component (Pfister et al. 1977). This association is congruous with our finding of high whitebark pine occurrence in the fir/spruce/mountain hemlock forest type; all sites with fir/spruce/mountain hemlock forest types had either subalpine fir or whitebark pine-timberline habitat types.

Misidentifications were concentrated where elevational overlap was greatest between the species: 63% of misidentifications were detected between 2,100 m and 2,500 m, despite only 41% of samples coming from this elevation range (figure 4). The concentration of misidentifications in this range resulted in a significantly higher mean elevation for field-identified limber pine samples (2,134 m) than that of lab-identified limber pine samples (1,973 m) (p = .013, Welch’s t-test). The field-identified elevation range of whitebark pine was not significantly different than the lab-identified range (p = .49, Welch’s t-test), but the shift in limber pine elevation suggests there is more elevational distinction between the species than was reflected in previously collected FIA data. We also observed that misidentification was most prevalent on plots just below 2,500 m (~8,200 ft) (figure 4 and figure 2d partial dependence plot), although only seven samples were lab-identified as limber pine above 2,500 m.

We detected a weak association (Cramer’s V = 0.210) between identification accuracy and lithology type (Table 3). We found that identification was 100% accurate within the carbonate sedimentary lithology type and 44% of samples were lab-identified whitebark pine. However, analysis of identification accuracy among lithology types proved problematic due to the coarse scale of the lithology data (1:1,000,000).

We also found that co-occurrence of whitebark and limber pines on the same plot was more common than was recorded by field crews; Of the seventy-nine plots that had more than one sample collected, eight plots had both whitebark and limber pine genetically identified (~10%), whereas field crews did not correctly identify that there were two species present on any plots. Because field crews did not detect more than one species per plot, identification errors occurred on all eight plots where co-occurrence was lab-identified.

Discussion

We found that trees identified as whitebark pine in the field were correctly identified more frequently than trees that were field-identified as limber pine. This strong error directionality may be due in part to the rapid deconstruction of whitebark pine seed cones for seed removal by dispersers (McCaughey and Schmidt 1990). Whitebark pine seed cones mature between mid-August and mid-September, after which they rapidly disintegrate as the seeds are harvested by Clark’s nutcrackers and other agents, leaving small remnants or no evidence of cones (Arno and Hoff 1989; McCaughey and Tomback 2001). In contrast, limber pine seed cones often remain present and intact (Tomback et al. 2011). When the two species co-occur, crews are more likely to see evidence of limber pine and may be more likely to assign that identification to all high-elevation five-needle pines in the area. Additionally, postpredation cone remnants on whitebark pine can be mistaken for persistent limber pine cones and lead to misidentification, even in stands that contain only whitebark pine. In addition to contributing to error directionality, cone phenology provides a potential explanation for the relationship of Julian day and identification accuracy (figure 2b); higher identification accuracy coincides with the time when mature pollen and seed cones are most likely to be present for whitebark pine. An alternative explanation of the Julian day relationship to identification accuracy is that field crews typically sample their highest elevation plots in August and September, and these plots are more likely to be dominated by whitebark pine (figure 4); 80% of samples collected from mid-July to late-September were lab-identified whitebark pine versus 55% and 57% before and after this period, respectively.

Our results support our expectation that identification errors would be more prevalent for seedlings than saplings and large trees due to the difficulty in distinguishing species if cones are absent. However, differences between the genetic sampling protocol and FIA plot measurements indicate that FIA seedling data may be more accurate than our study suggests, even in the study area (see Implications for users of FIA data section). Further, the higher identification accuracy for samples from all size classes in the presence of correctly identified large whitebark pine trees suggests that (1) there is a tendency of crews to assign any five-needle pine regeneration to the same species they recorded in the overstory and (2) higher identification accuracy rates for regeneration on plots with large whitebark pines recorded can be explained by higher identification accuracy rates for large whitebark pines (Table 2).

A related finding is that identification accuracy is lower in forest types other than whitebark pine or limber pine types. Two previous studies based on FIA data (Goeking and Izlar 2018; Goeking et al. 2019) documented the prevalence of whitebark pine within the fir/spruce/mountain hemlock and lodgepole pine forest type groups and noted particularly high densities of regeneration in stands dominated by lodgepole pine. Our results validate that seedlings identified as whitebark pine were virtually always correct, yet more than half of seedlings identified as limber pine were actually whitebark pine. Thus, although these earlier studies quantified abundant whitebark pine regeneration in multiple forest types, they likely underestimated the prevalence of whitebark pine seedlings.

The strong positive association we found between whitebark pine and subalpine fir indicates that in some regions where the ranges of whitebark and limber pines overlap, the ability of a site to support subalpine fir may serve as a strong predictor of the five-needle pine species present. Pfister et al. (1977) and Steele et al. (1981) found that limber pine occurrence was restricted to just a few of the driest habitat types capable of supporting subalpine fir in Montana and central Idaho, respectively. However, Steele et al. (1983) found that limber pine occurrence was less restricted within habitat types capable of supporting subalpine fir in eastern Idaho and western Wyoming. These resources indicate that although the whitebark pine - subalpine fir association may be useful for identification in some regions, it may be less reliable in other areas, particularly in the GYGT seed zone, and this relationship warrants more study.

Higher error rates in the GYGT seed zone, which is known for challenges in five-needle pine species distinction (Greater Yellowstone Whitebark Pine Monitoring Working Group 2011; Hansen et al. 2021), may result from the specific combination of lithologies, habitat types, and forest types found within this seed zone. We found higher representation of some lithology types, habitat type groups, and forest type groups that had lower identification accuracy rates in this seed zone than other zones (data not presented). Crew leader experience also had substantial influence on identification accuracy, although we limited the quantification of crew leader experience to measurements of plots in FIA’s annual inventory, and thus potentially excluded experience gained from other field work. We found that crew leaders in the GYGT zone had less experience; thus, experience and seed zone may be confounding factors.

Lithology helps determine moisture and nutrient availability and has been tied to whitebark pine and limber pine occurrence patterns (Hansen-Bristow et al. 1990; Weaver 2001; Weaver and Dale 1974). We predicted that identification accuracy might be low within the carbonate sedimentary lithology type because resources used by field crews (e.g., Pfister et al. 1977; Steele et al. 1981) mention an affinity for calcareous (i.e., carbonate) substrates by limber pine and not by whitebark pine, yet comprehensive reviews of species substrate preferences suggest this is true only in specific geographic areas (Hansen- Bristow et al. 1990; Weaver 2001). However, this prediction was not supported by our results; all samples collected from sedimentary (carbonate) lithology type were accurately identified. Although we found a weak overall association between identification accuracy and lithology type, the possibility that samples were collected from inclusions of different lithology types in the coarser scale lithology polygons make this association questionable. If site-specific lithology data were available for RMRS-FIA plots, this analysis would be more useful.

Although we did not examine co-occurrence as a predictor variable, our results indicate that the potential for identification errors markedly increases on sites where whitebark and limber pines co-occur, as noted by others (Greater Yellowstone Whitebark Pine Monitoring Working Group 2011; Hansen et al. 2021). Because sample collection was limited to a maximum of four samples of each species per plot, true co-occurrence rates are likely higher than the 10% rate we detected. The likelihood of co-occurrence indicates that definitive identification features (e.g., mature cones) on one tree cannot reliably be used to identify nearby trees. This implication poses additional challenges to field crews tasked with identifying trees of all ages in a fixed area and underscores the importance of providing additional identification tools (i.e., genetic analysis).

Implications for Users of FIA Data

Genetic analysis of whitebark and limber pine samples revealed misidentification rates that exceed FIA’s data quality standards. Although over 99% of samples identified as whitebark pine by field crews were actually whitebark pine, the misidentification of many whitebark pine as limber pine meant that field crews correctly identified large trees, saplings, and seedlings of whitebark and limber pine only 89%, 88%, and 78% of the time, respectively. FIA objectives are to correctly identify species ≥95% of the time for large trees and saplings and ≥85% for seedlings (USDA 2010). Although these misidentification rates are based on FIA field crew measurements, they are likely indicative of error rates from other field-based identification of whitebark and limber pines by trained monitoring crews.

These samples were analyzed opportunistically, relying on a previous sampling strategy designed to compare genetics of all tree species across broad scales; thus, sampling was not specifically intended to distinguish whitebark and limber pines. Because crews did not know these samples would be used to test identification accuracy, the effort devoted to identification of these samples is likely representative of RMRS-FIA identification. The geographic range of the original sampling protocol provided seventy-six whitebark and limber pine samples collected outside of the study area. Genetic analysis verified that field identification was 100% accurate for both species outside the study area, indicating that the implications of identification errors are confined to the area of range overlap.

The differences between genetic sample collection protocols and FIA field methods create some challenges for understanding implications for FIA data. Foliage collection was limited to one sample per subplot for each species identified in the field for a maximum of four samples per species per plot. The sample limit, paired with the common assumption that only one five-needle pine species was present on a plot, reduced the likelihood of identifying undetected species co-occurrence, including dispersed seedlings of a five-needle pine species not present in the overstory. Needle collection protocol for this study only called for sampling from seedlings when larger, potentially cone-bearing trees were absent from the subplot or when needles could not be reached on these trees, whereas RMRS-FIA protocol measures seedling species regardless of large tree presence. Field crews typically assume that whitebark and limber pine seedlings match the species of nearby large trees. Our results corroborate this assumption on plots with large whitebark pine, which account for approximately 65% of plots with whitebark or limber pine in the study area. This suggests that large tree presence likely increases accuracy on the majority of FIA plots. A total of 71% of plots that have whitebark seedlings in recently published FIA data (2010–2019) also have large trees on plot, whereas only 60% of plots with seedlings collected in this study had large trees present on plot. This indicates that whitebark and limber pine seedling identification is likely somewhat better throughout FIA data than our results suggest, even within the species’ range overlap. However, the 2010–2012 sampling design limits the ability to quantify seedling identification accuracy for these species in the broader sample of FIA data.

Despite sampling limitations, this study does help FIA data users understand whether misidentifications introduce substantial bias to estimates of whitebark and limber pine numbers derived from FIA data. Our results suggest that within our study area, whitebark pine abundance has been underestimated, whereas limber pine abundance has been overestimated. If we wish to quantify the impact of this bias, we must make several assumptions. First, we must assume that the selected samples are representative of the overall population of whitebark and limber pines on RMRS-FIA plots within our study area. In our study, this is likely true for large trees and saplings but not for seedlings, suggesting that our results should not be used to quantify the impact of misidentifications on estimations of seedling numbers. Second, we must assume that misidentification rates are similar between the 2010–2012 study period and other years of data collection.

The sample design only evaluated live trees; therefore, factors related to misidentification cannot directly inform whether bias in species identification would be substantially different for dead trees. However, some information can be inferred based on the methods of FIA data collection. For example, dead trees in the FIA dataset are either (1) trees that were dead at the time of initial plot establishment or (2) trees that were alive at a previous plot visit and have since died. Patterns of misidentification are likely similar to those of live trees for the latter because the initial identification was made when the tree was alive. Our results provide no evidence that live and dead whitebark pine trees were underestimated at different rates or that previously published estimates of live-to-dead ratios of whitebark pine trees (Goeking and Izlar 2018) are biased.

In addition to affecting estimates of tree numbers, bias resulting from species misidentifications has implications for the use of FIA data in species distribution modeling. Our results indicate that data used to develop species distribution models for whitebark pine on forestland likely include only a few cases where the trees are actually limber pines. However, these models are likely missing additional locations where whitebark pine was present but was misidentified as limber pine. Further, we found that misidentifications were not evenly distributed with respect to elevation and habitat type series, which are closely correlated with climatic conditions (Pfister et al. 1977; Steele et al. 1981). However, because field and lab-identified elevation ranges of whitebark pine samples were not significantly different, inferences based on FIA data concerning the elevation distribution of whitebark pine should remain valid. Given that we found that 40% of samples field-identified as limber pine were actually whitebark pine, species distribution modeling of limber pine based on FIA data likely contain more errors of commission (i.e., predicting limber pine presence where it does not occur) in areas of range overlap with whitebark pine. Additionally, significantly higher mean elevation for field-identified limber pine samples than that of lab-identified limber pine samples means that inferences based on FIA data about the elevation distribution of limber pine need to be reevaluated. Although some of these results provide assurance of data quality, they also indicate that it is important to implement tools that improve identification accuracy not only within FIA but also for any probability-based field studies of these two species in regions of range overlap.

Our results indicate ways the FIA program and other studies that rely on field-based species identification can improve identification accuracy. First, sample protocols should prioritize non-cone-bearing trees for genetic analyses that provide definitively accurate identification of these individuals (Alongi et al. 2019; Hansen et al. 2021). Genetic analyses have proven relatively economical; current laboratory costs are about $20 per sample (with added costs and time for sample collection, cataloging, and shipping). Needle sampling and laboratory verification could also be applied beyond the zone of overlap to validate these two species’ geographic ranges and potentially identify populations outside their currently known range. However, such an objective would require a larger sample size than used in the current study. Second, trends in crew leader experience suggest that training on whitebark pine versus limber pine identification led by experienced personnel could increase accuracy when cones are absent and genetic analyses is not possible, such as when a tree is dead, needles are out of reach, or cost is prohibitive. Suggested training would include how to use important site-level predictor variables such as forest type and habitat type and would emphasize that species identification should be made at the tree versus stand level, as co-occurrence is more likely than expected.

These two approaches for improving quality complement each other; as the data from genetic identification efforts provide a more comprehensive accounting of error and species occurrence patterns, training materials can be improved. Our results indicate that improved training may increase future data quality, even in studies that may not have the means for genetic identification. Based on these results, in 2022, RMRS-FIA implemented a protocol that aims to genetically identify all non-cone-bearing large trees, all saplings, and up to five seedlings per subplot of whitebark and limber pine. Improved data quality is important not only for FIA and other field-based monitoring programs, but also for improved accuracy of models and maps of seedling niches and climate change projections, all of which rely on field-based observations for calibration and validation.

Conclusions

Identification accuracy of whitebark and limber pines by RMRS-FIA field crews fell below FIA’s data quality standards for all tree size classes in our study area. With respect to whitebark pine, field crew identifications showed high specificity (<1% of limber pines were field-identified as whitebark pines) but relatively low sensitivity (about 25% of whitebark pines were not field-identified as such). This resulted in underestimation of whitebark pine abundance and overestimation of limber pine abundance. Species distribution modeling efforts calibrated with FIA data will include very few false presences of whitebark pine, although some sites with whitebark pine present will be omitted from calibration datasets. In contrast, species distribution models for limber pine will be influenced by a greater number of false positives that could negatively affect model accuracy.

Misidentifications were confined to regions of range overlap, implying that potential biases were limited to those geographic areas. Within regions of range overlap, sampling limitations make it difficult to quantify the true impact of biases on estimates of whitebark and limber pine numbers and ratios of dead to live trees. Despite sampling limitations, we found that co-occurrence of whitebark and limber pines was more common than recorded by field crews and that seedling identification was particularly challenging. Better understanding of patterns of misidentification gained through this study has helped RMRS-FIA develop a path toward improvement that focuses on needle collection from non-cone-bearing trees by crews that are trained to use multiple factors for species identification.

Supplementary Material

Supplementary material is available at Forest Science online.

Acknowledgments

We thank all RMRS-FIA field crews for collecting the data and samples that made this work possible. We also thank Montana State University Plant Sciences Department for providing laboratory supplies used for genetic analysis. We extend sincere gratitude the editors and anonymous reviewers, whose comments led to substantial improvements of this article.

Funding

This work was supported in part by the USDA Forest Service, Forest Inventory and Analysis Program at Rocky Mountain Research Station. The findings and conclusions in this publication are those of the authors and should not be construed to represent any official USDA or US Government determination or policy. This article was prepared in part by employees of the USDA Forest Service as part of official duties and is therefore in the public domain in the US.

Conflict of Interest

None declared.

Data Availability

Most data underlying this article can be made available upon request by emailing [email protected]. A small portion of the analysis relied on spatial data using FIA’s true plot locations and related variables which are legally protected as confidential information.

Literature Cited