A novel tobacco forecasting model by multiple sociodemographic strata in Australia

Abstract

Introduction

Over the past three decades, Australia has been a global leader in tobacco control, achieving one of the lowest daily smoking rates among high-income countries—estimated at 8.5%–10.2% in 2022–23 [1, 2]. Despite this gain, tobacco smoking remains a major cause of disease, responsible for 21000 deaths annually (13% of total deaths) [3]. Smoking rates and the associated disease burden vary significantly across sociodemographic factors such as geographic remoteness, socioeconomic status (SES) and Indigenous status. In 2022–23, daily smoking rates were estimated at 20.1% in remote areas, and 13.4% in the most disadvantaged SES strata [1]. Daily smoking in the Indigenous population was estimated at 28.8% in this period [4]. These differences between sociodemographic groups have persisted, and in some cases widened (e.g. in remote vs. urban areas [5, 6]), indicating that business-as-usual (BAU) tobacco control strategies have had insufficient equity focus.

To end the tobacco epidemic, several countries have set targets for a “commercial tobacco endgame”, aiming to reduce tobacco smoking rates to ≤5% between 2025 and 2040 [7–9]. Commercial tobacco products, being those produced and sold by the tobacco industry, differ from traditional or sacred tobacco used by Indigenous communities, e.g. in ceremonies [10]. Most countries, including Australia, have a single population-wide commercial endgame target, which overlooks the substantial variation in smoking rates across different sociodemographic groups and fails to address equity. The Australian Federal Government’s National Tobacco Strategy aims to reduce daily smoking prevalence to ≤5% by 2030 for the population overall [7]. A separate goal of 27% smoking prevalence by 2030 for First Nations peoples was set, but no date for when a similar ≤5% is to be reached. Specific goals for other population groups were not set. A major challenge in setting equity-driven goals is the lack of high-quality, disaggregated smoking prevalence data over time, with no single dataset providing smoking prevalence by all sociodemographic factors of interest. However, several disparate datasets with partial information could be combined to create a historical time series and project future smoking prevalence across population subgroups of interest. In Australia, smaller population groups such as those in remote areas are often underrepresented in surveys, limiting understanding of their smoking trends and likelihood of achieving the 2030 overall target. This study aims to address this knowledge gap, and demonstrate a method for joining disparate datasets, by answering the following questions:

1. How close is Australia to achieving ≤5% daily smoking prevalence by 2030 under BAU?

2. What are the projected smoking rates in Australia by 2053, differentiated by remoteness and SES categories?

3. What rate of decline in smoking rates among disadvantaged groups would be required to achieve equality in the 2030 endgame target?

Methods

Overview of the model

We defined BAU as the projected daily smoking prevalence based on extrapolating smoking trends observed in Australia over the past 20 years. This projection followed a two-step process: (1) a simulated annealing model was used to generate estimates of historic smoking data for 11 040 strata, accounting for six covariates: year (2001–2023), sex (male, female [non-binary and other gender identities were excluded due to lack of data]), age (eight categories: 15–17, 18–24, 25–34, 35–44, 45–54, 55–64, 65–74, 75+ years), remoteness (urban, regional, remote), SES (five quintiles of the SocioEconomic Indexes for Areas Index of Relative Socioeconomic Advantage and Disadvantage [11]), and Indigenous status (Aboriginal and/or Torres Strait Islander: yes/no); and (2) regression models were then applied to this fully disaggregated historic data to project future smoking trends by birth cohort from 2023 to 2053. Both steps were run in Python (version 3.10.14).

Input data

We extracted daily smoking prevalence estimates from four national surveys: the National Drug Strategy Household Surveys (NDSHS; conducted triennially from 2001 to 2022–23 [1]), the National Health Surveys (NHS; conducted every 2–4 years from 2001 to 2022 [2]), the National Aboriginal and Torres Strait Islander Health Surveys (NATSIHS; conducted in 2004–5, 2012–13, 2018–19, and 2022–23 [4]) and, the National Aboriginal and Torres Strait Islander Social Surveys (NATSISS; conducted in 2002, 2008, and 2014–15 [12]) (Table 1). The latter two datasets, which focus on the Indigenous population, are sufficiently similar to be combined, as previously done by Thurber et al. [6], and are referred to hereafter as the NATSIHS/NATSISS dataset.

Population estimates for the period 2001–2023 were obtained from the Australian Bureau of Statistics (ABS). To forecast population counts for 2024–2053, we used recent mortality data, along with ABS birth and migration projections. Population calculations are detailed in the Supplementary Methods (Sections S1.3 and S2.1, available as Supplementary Data at IJE online).

Part 1: generating historic prevalence estimates across sociodemographic strata

We applied a simulated annealing approach [31] to estimate smoking prevalence disaggregated by year, sex, age, remoteness, SES, and Indigenous status between 2001 and 2023, using the datasets in Table 1.

A detailed description of this approach is available in the Supplementary Methods. Briefly, simulated annealing is a probabilistic algorithm used to search for an optimal solution within a parameter space, based on pre-defined target(s) [32]. In successive iterations, the algorithm randomly selects a new ‘candidate’ point near the current point in the parameter space. If the candidate scores better than the current point against the targets, it is accepted. Otherwise, it may be accepted with a probability that decreases as the algorithm progresses, according to the ‘temperature’. The temperature is a value that reduces with each iteration, allowing wider exploration early on (to avoid trapping the algorithm in local minima) and converging on an optimal solution over time.

For our model, we started with a randomly selected dataset comprising 11 040 values of smoking prevalence (together representing a single point in the parameter space). We used multi-objective optimization [33], with the smoking prevalence strata combinations outlined in the last column of Table 1 as targets for the modelled dataset to optimize to. These targets were input either in raw prevalence form or as relative risks (RRs) to inform optimization of data-poor strata (e.g. using regional trends to approximate remote trends). In each iteration, a fully disaggregated (i.e. 11 040 values) candidate dataset was generated, collapsed into lesser strata using population counts and compared to the target datasets. The fit of the candidate to each target was calculated by summing the log-squared differences between the candidate and target values. Individual target scores were then summed into an aggregate score.

We applied additional rules to reflect theoretical expectations around smoking trends: smoking prevalence should follow an approximately log-linear trend over time, increase with remoteness (remote > urban), and decrease with increasing socioeconomic advantage (SES5 < SES1). These rules were applied across all strata, with exceptions where data strongly indicated otherwise (see Supplementary Methods  Section S1.4.2). Smoking prevalence was also constrained to not increase with birth cohorts above the 25–34 age category, as smoking uptake typically peaks before age 30 [34, 35].

Since the model had multiple targets from different sources and strata with varying population sizes, trade-offs were necessary when optimizing the dataset. To prevent large deviations in any strata, we applied differential weights to each target during the scoring process. Weights were updated iteratively after each model run until the algorithm produced a dataset that met the criteria for target fit. Further details on model validation are provided in the Supplementary Methods (Sections S1.4.1 and S1.5.1).

Part 2: regression forecasting

Using the final dataset from the optimization model, expanded to single year of age values using cubic spline interpolation, we ran 60 separate logistic regression models for each combination of sex, remoteness, SES, and Indigenous status, parameterized as

Calendar year was modelled as a continuous variable, and age with knots, $k$⁠, placed at two-year intervals from age 15 to 35 then 10-year intervals from 35 to 75 (resulting in $n=$15) to allow smoking prevalence to differ non-linearly by age. Greater variation was allowed for younger ages during which there is greater variability in smoking rates (with both smoking uptake, then cessation, occurring), while older ages (where smoking rates follow a downward trajectory reflecting net cessation and/or death due to tobacco-related disease) were attributed fewer knots for parsimony.

Age-by-calendar year interaction terms were not included in the logistic regression models. Thus, a common ‘shape’ of smoking prevalence by age was estimated for all 60 populations, with the $B2$ coefficient estimating the shift on that shape over time on the log odds scales, assuming a linear decline in smoking rates over time, consistent within each model. Each regression model was weighted by the population size for each age and calendar year (from 2001 to 2023). The weights corresponding to each year, $y$⁠, were then multiplied by $0.952023-y$ to encode the assumption that future birth cohorts are more likely to follow trends of recent years than older ones.

The predictions of smoking prevalence from these models were used to forecast smoking prevalence of each birth cohort as they age, to 2053.

Although Indigenous status was included in the model due to its strong association with smoking rates and interaction with remoteness and SES, we do not report forecasts by Indigenous status in this paper. This decision respects principals of data sovereignty and governance [36, 37] as none of the authors are Indigenous Australians. Future analyses, conducted with the collaboration and leadership of Indigenous researchers, will build on these preliminary results.

Results

At the population level, daily smoking rates for 15+ year olds are projected to decline from 9.6% in 2024 to 7.8% by 2030 under the BAU scenario. The endgame goal of ≤5% daily smoking prevalence is forecast to be achieved in 2043, with females reaching this target by 2040 and males by 2047 (Fig. 1).

Figure 1.

Daily smoking prevalence estimates by sex from 2022 to 2053 in Australia. The dashed black line depicts the population average (aggregated across all strata). The grey box indicates the tobacco endgame goal of 5% daily smoking prevalence.

Open in new tabDownload slide

Smoking prevalence over time across each remoteness and SES strata are shown in Fig. 2. By 2030, only two out of the 15 strata achieve the ≤5% target. The highest smoking rates remain in the remote, SES1 (i.e. most disadvantaged) strata, with smoking rates in this group approximately four times the national population average in 2030. Further forecasts disaggregated by strata are provided in the Supplementary Results.

Figure 2.

Daily smoking prevalence estimates by remoteness and socioeconomic status from 2022 to 2053 in Australia. The dashed black line in each panel depicts the population average (aggregated across all strata). The grey box indicates the tobacco endgame goal of 5% daily smoking prevalence.

Open in new tabDownload slide

Based on our projections, in order to achieve the tobacco endgame goal at the national population level by 2030, daily smoking needs to decrease by 10.3% annually, or 3.1 times the modelled BAU rate from 2024 onwards. For each sociodemographic group to achieve 5% by 2030, the necessary rate of decline increases with greater remoteness and socioeconomic disadvantage, as shown in Table 2. Our model estimates that to reach this target, smoking rates would need to decline by 2.1 times the modelled BAU rate annually in urban areas, 5.3 times the BAU rate in regional areas, and 12.5 times the BAU rate in remote areas. Among those in the most disadvantaged socioeconomic category (SES1), a decline 7.4 times the modelled yearly BAU rate is required. In contrast, those in the least disadvantaged socioeconomic category (SES5) are projected to reach the endgame target before 2024 (Table 2). For the remote, SES1 category, an annual increase by approximately 37-fold over the projected BAU rate of decline (or a 28% decrease per year), would be needed to reach 5% daily smoking by 2030 (Table 2).

Discussion

Our model forecasts smoking prevalence in Australia from 2023 to 2053, disaggregated by sex, age, remoteness, SES, and Indigenous status. The projections show that under BAU (i.e. the momentum of change in prevalence in recent years forecast into the future), Australia is unlikely to achieve the target of ≤5% national daily smoking prevalence, as set in the National Tobacco Strategy. National smoking prevalence is projected to reach 7.8% by 2030, and significantly higher rates are forecast for disadvantaged and remote population groups. For example, smoking prevalence in the most disadvantaged (SES1) and remote population is expected to remain as high as 34.6% by 2030, down only slightly from an estimated 36.2% in 2024 (Fig. 2).

Our findings highlight the urgent need for policy action to accelerate declines in smoking prevalence faster than that forecast under BAU in order to reach the endgame goal. To achieve the endgame target, smoking rates would need to decrease annually by 8.5% for females (2.5 times the BAU rate), and 11.8% for males (3.6 times the BAU rate), based on population averages. The required annual reductions are even greater for more disadvantaged groups and those living in regional and remote areas if inequities in future smoking-related disease burden are to be reduced (Table 2).

Stronger measures than the current tobacco control policies, such as excise tax increases, are necessary to address the inequality in smoking prevalence across sociodemographic groups. Puljević et al.’s recent scoping review of tobacco endgame strategies highlights the potential of new approaches, such as reducing the nicotine content of cigarettes to non-addictive levels (‘denicotinisation’) or limiting the availability of tobacco products by restricting the number of retailers [38]. Our forecasts can serve as a baseline for simulating the effectiveness of such policies compared to traditional incremental approaches, especially in reducing differences in smoking rates and resulting health inequities. It should be noted that while a more ambitious equity-focused endgame goal is needed in Australia, a 5% target for 2030 may not be feasible or appropriate for specific populations with smoking rates sitting well above the national average. Future modelling for these populations, such as Indigenous Australians, must be community-driven and consider their unique needs and preferences when selecting policies, as well as the policy target.

A 2023 systematic review of published tobacco control models reported that only four out of the 25 models identified globally explicitly considered smoking-related inequity, and when considered it was just for a single factor such as SES, income, education, or ethnicity [39]. Our model is the first to integrate multiple sociodemographic strata, using a simulated annealing optimization algorithm to address the absence of comprehensive data across covariates over time. While other methods (such as small area estimation or hierarchical Bayesian models) [40] can combine unit-level survey data from multiple sources, our approach was able to use aggregated data to estimate prevalence for heterogeneous population groups. The use of simulated annealing also allowed us to incorporate logical assumptions, such as higher smoking prevalence in remote areas. This approach can be applied in other settings where multiple factors contribute to smoking-related inequity and where data are incomplete.

Our findings differ from those of Wade et al. [41] who forecast smoking prevalence in Australia by sex and age, based on NDSHS smoking trends up to 2016. Their model estimated that Australia could reach 5% prevalence by 2039 [41]—four years earlier than our estimate. The difference may be due to our model’s structural assumptions, the inclusion of more recent data, and the integration of multiple datasets, including the NHS, which generally reports higher smoking prevalence than the NDSHS (see key survey differences in Supplementary Table S1).

Our model aimed to produce reasonable forecasts based on historic trends resulting from the impact of tobacco control policies over the past 20 years in Australia. This process involved several assumptions about the regression model structure, which were tested in sensitivity analyses (Supplementary Figs S18–S23). The results were sensitive to the choice of regression model, with the target being reached two years later at the population level under a log-linear rather than logistic regression forecasting framework. Removing recency weights and changing age-knot placement had minimal impact.

An additional analysis compared the final output with a previous version produced with the same methodology but prior to the 2022–23 datasets being available as targets. The exclusion of the most recent survey datasets in the simulated annealing model saw the endgame target forecast to be reached six years later than the updated model (Supplementary Figs S24 and S25). This is due to the steeper decline seen in population and (most) strata-level smoking rates from 2018–19 to 2022–23 than in previous timesteps, which might be due to some switching from smoking to vaping, or might be artefact of lower survey response rates since the COVID-19 pandemic [42, 43]. This is an important reminder that forecasts are just that—forecasts.

A related limitation of our approach is that we did not quantify uncertainty in the historic prevalence estimates generated by the optimization model. Future analyses could use high-performance computing to simulate thousands of model runs, drawing from the uncertainty in survey estimates (rather than just using the mean values within a single model run), and propagating this through to the regression forecasts. Additionally, one might use a machine learning ensemble of equations (not just logistic regression) to forecast from each of the optimized datasets. However, these processes would take large computing power and resources. Our primary aim in this article was to provide reasonable forecasts to inform policy, rather than fully quantify uncertainty of historic/future smoking prevalence.

There are other limitations to our modelling approach. While we had multiple datasets available as targets, there was still limited information on smoking rates by SES in remote areas and for the Indigenous population. We used logical assumptions based on known relationships to help smooth such areas of data sparsity. Additionally, we did not explicitly model the impact of e-cigarette use/vaping on future smoking rates as data were insufficient. Recreational vaping product sales have recently been banned in Australia [44, 45], meaning it is reasonable to assume that daily vaping rates (estimated at 3% in 2022–23 [1]) will remain constrained. Nevertheless, future analyses should examine this uncertainty explicitly, for example modelling the potential impacts on smoking rates under scenarios in which e-cigarette use continues to increase versus rapidly falls away. Greater consideration is likely also required in jurisdictions with more liberal access to e-cigarettes.

In summary, our forecasting model shows that without significant policy changes, Australia is unlikely to reach the 2030 tobacco endgame target. Achieving equity in this goal is particularly challenging for remote and disadvantaged populations compared to urban and high SES groups. Our methodology could be applied to other countries to better understand and address disparities in future smoking prevalence.

Ethics approval

Ethics approval was not required as deidentified, published data used. Guidelines from AIHW were followed to maintain confidentiality for unit-level NDSHS data received from the Australia Data Archive.

Author contributions

All authors contributed to study conceptualization. Samantha Howe and Tim Wilson constructed the model. Samantha Howe conducted model implementation. All authors contributed to interpretation of findings. Samantha Howe drafted the article. All authors contributed to editing the final article. Guarantor: Driss Ait Ouakrim.

Supplementary data

Supplementary data is available at IJE online.

Conflict of interest: None declared.

Funding

S.H. holds a University of Queensland Melbourne Training Scholarship and Research Higher Degree Top-Up Scholarship from the NHMRC Centre of Research Excellence on Achieving the Tobacco Endgame (GNT1198301). C.G. holds a NHMRC Grant (GNT1198301) and an ARC Future Fellowship (FT220100186). T.B., D.A.O., and T.W. are supported by the NHMRC Centre of Research Excellence on Achieving the Tobacco Endgame (GNT1198301).

Data availability

Data inputs were obtained from publicly available sources as cited in the text (Exceptions: unit-level NDSHS data from the ADA required formal request, NHS data from TableBuilder may require an institutional account for access). Additional model output and model code can be made available upon reasonable request.

References