In the context of Industry 4.0, predictive maintenance (PdM) emerges as a key component to optimize manufacturing processes by improving productivity and reducing costs. PdM harnesses the convergence of advanced technologies, such as the Internet of Things (IoT), real-time data analytics, and machine learning (ML) algorithms, to anticipate equipment failures before they occur. This enables industries to optimize maintenance plans, improve operational efficiency, and ensure the reliability of critical assets [1].

PdM mainly addresses three types of problems [2]: (1) anomaly detection, (2) failure diagnosis, and (3) prediction of the remaining useful life (RUL). Anomaly detection focuses on identifying potential failures that have recently occurred or are about to happen. Failure diagnosis seeks to determine the root cause of a problem, typically using root cause analysis (RCA) techniques. And RUL aims at estimating the time or number of operation cycles remaining before a system or component reaches the end of its operational life, making this one of the most challenging problems to be solved.

To address the problem of RUL prediction, two approaches are generally used: model-based methods and data-driven methods. The former relies on mathematical and physical principles and prior system knowledge to create predictive models. However, these methods can be complex, costly, and time-consuming due to the need for in-depth knowledge of the system. In contrast, data-driven methods explore the relationships between sensor data monitored and RUL values using historical data. But these, despite the numerous success cases mentioned in this survey [3], are also affected by certain inconveniences that hinder the building of accurate and reliable models while keeping the use of resources low. First, industrial assets operate under dynamically changing conditions, making it difficult to generalize prediction models [4]. Furthermore, the scarcity of labeled data poses a significant obstacle, as failures are rare events, and it is not always feasible to run the equipment until it fails to collect representative data [5]. In addition, when possible, the labeling work is labor-intensive, daunting, and time-consuming [3,6]. Finally, data sets are often unbalanced, with a high proportion of normal operating data and a low representation of failures, which can bias models [7].

In response to these limitations, recent advances in ML have introduced a range of innovative frameworks specifically designed to mitigate these challenges. One particularly effective strategy is the use of simulated or synthetic data, which enables the artificial replication of failure scenarios and the generation of full-lifecycle datasets in controlled environments. This approach not only reduces the reliance on costly and time-consuming real-world failure data but also facilitates model evaluation under diverse operational conditions without physically stressing the equipment [8]. Complementing this, semi-supervised learning methods—such as likelihood-based pseudo-labelling—have been developed to leverage large volumes of unlabelled sensor data while minimizing the need for manual annotation [9]. Hybrid approaches that integrate self-supervised and supervised learning techniques applied to incomplete lifecycle data have also demonstrated considerable effectiveness in extracting relevant degradation features from industrial environments [10]. Further strategies address the combined challenges of data scarcity and class imbalance through the integration of enhanced clustering techniques and synthetic oversampling [11]. To ensure model robustness across varying operational conditions, domain adaptation and transfer learning have been extensively explored to align data representations across multiple working environments, thereby improving generalizability [12]. Additionally, recent advances in uncertainty quantification have introduced probabilistic frameworks capable of estimating confidence in predictions, which is essential in safety-critical applications such as predictive maintenance [13]. Finally, an emerging direction combines several of these techniques within reinforcement learning-based frameworks. Recent studies have integrated synthetic data generation through data diffusion, Bayesian deep learning for uncertainty estimation, and active learning for sample selection, achieving significant improvements in prediction accuracy and reducing uncertainty by 15%–42% [14].

Building upon these trends, the necessity to enhance generalization capabilities and reduce labeling costs has catalyzed interest in n-shot learning paradigms, including few-shot, one-shot, and zero-shot learning. Few-shot learning typically leverages transfer learning (use a pretrained model [15]) or meta-learning techniques to train models that can recognize new classes using only a handful of labeled examples, whereas one-shot learning requires just a single labeled instance, and zero-shot learning, none.

Foundation models (FMs) are a class of deep learning (DL) models that are pretrained on vast amounts of data, thus equipped with a wide range of general knowledge and patterns. To this end, these models serve as a versatile starting point for various tasks across different domains. These models have proven to be useful in predicting future values in time series, classifying, detecting anomalies, filling in missing data, or generating synthetic time series that mimic real data for simulation tasks [15]. Additionally, they are valuable for interpreting model outcomes and identifying root causes [16].

In this paper, we explore the use of TSFM as a means to reduce the reliance on large labeled datasets and to enable the development of accurate predictive models with lower computational and energy demands [17]. TSFMs offer a robust, pretrained basis for extracting relevant features from time series data, allowing for the construction of simple, competitive predictors with minimal training data. By leveraging the representational power of TSFMs, we aim to promote model reuse, minimize dataset-specific preprocessing—often time-consuming and detrimental to generalizability—and enhance robustness in real-world industrial scenarios, where failure events are scarce.

More specifically, in this work, we experimentally assess the potential of a hybrid architecture that leverages the feature extraction strengths of TSFMs in conjunction with lightweight, resource-efficient ML algorithms. To this end, we design and conduct four experiments using the CMAPSS dataset to determine whether such a combination can:

Match or surpass the predictive accuracy typically associated with DL models in time series regression problems, and significantly reduce training times compared to deep architectures based on recurrent or attention-based mechanisms.

The performance of the proposed architecture is compared against 22 existing ML and DL models for RUL prediction on the CMAPSS dataset, selected from the recent literature.

This paper is organized as follows: Section 2 provides a background about artificial intelligence (AI) strategies followed to estimate the RUL, motivating the need and opportunity of using FM for this goal. Section 3 describes the methodology followed in this research. Section 4 details the experimentation conducted on the CMAPSS dataset, specifying the preprocessing tasks performed, the setting of models built, and their evaluation under the following metrics: RMSE, score, training time, and inference time. Section 5 discusses the findings of this study highlighting scenarios in which TSFM could be a promising alternative. Finally, Section 6 draws the conclusions of the paper and the next steps in our research.

Over the past decade, sensorisation and the adoption of AI techniques have revolutionized predictive maintenance, transforming the industrial sector significantly. The number of research surveys, both general and topic-specific, on PdM is vast, highlighting the importance of this field [2,18,19].

In the literature, we find PdM applications that employ all kind of AI approaches-supervised, unsupervised, and reinforcement learning-to analyze the large volume of data captured by real-time condition monitoring systems [20]. Within each paradigm, different algorithms from the ML and DL arena have been successfully applied. For instance, Li et al. [21] focused on the importance of accurately predicting the RUL of lithium-ion batteries using different algorithms. Their experimentation yielded that RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network) present a good performance and ability for information extraction; SVR (Support Vector Regression) and ELM (Extreme Learning Machine) exhibit a good online updating ability and fast prediction; and AR (Auto-Regression) was the simplest algorithm with acceptable accuracy. Others, such as Taşcı et al. [22] proposed a hybrid model for the RUL prediction before production lines stop. They used real-world high-dimensional data from IoT sensors, and among all the proposed methods, RF (Random Forests), an ensemble bagging method, turned out to perform best. Later, Dintén et al. [23] used transformers to assess their effectiveness in anomaly detection and failure prediction, finding that the hybrid transformer-GRU (Gated Recurrent Unit) configuration delivers the highest accuracy, albeit at the cost of requiring the longest computational time for training. Also, Kim et al. [24] successfully implemented a transformer-based model to predict the RUL of lubricant used in operational rolling bearings. As Yu et al. [25] pointed out, when using transformers directly for failure prediction, the results often fail to meet expectations due to the sparsity of failure events within the datasets, leading to a model that learns temporally sparse features that do not adequately represent the complexity of failure prediction scenarios.

Despite the successes achieved so far, the intrinsic characteristics of industrial time series—being inherently temporal and capturing the dynamics of complex systems and processes—pose two major challenges in building robust and accurate predictive models. First, the scarcity of labeled datasets, which are also highly imbalanced due to the low frequency of failure events [5,15], often makes it difficult to construct a reliable and robust predictor, as pointed out by [25]. Second, the specificity of the behavior of each industrial asset limits the ability of the models to generalize. As previously mentioned, this has led the scientific community to propose innovative solutions to tackle these challenges, being one of these to develop adaptive approaches capable of transferring knowledge across different systems and operating conditions. Here is where TSFMs come into play–large AI models inspired by LLMs such as BERT [26] and GPT-3 [27], but specifically adapted for time series analysis [28]. One of their key advantages is that these models can be fine-tuned or adapted to specific tasks with relatively small amounts of task-specific data (few-shot learning) [29].

Recently, LLMs combining time series with textual prompts have achieved promising performance in TSFM. One approach represents time series as patches and leverages pretrained models for prediction [15]. Another research direction focuses on aligning embeddings between time series and textual data. This is the case of TimeCMA [30] that proposes a framework that uses dual-modality coding, combining time series embeddings generated with a transformer with embeddings obtained from textual cues in a pretrained LLM, improving accuracy and reducing computational costs. The transformer time-series attention is local and variable-specific, whereas LLM textual attention is universal and captures global dependencies between variables. Another proposal is TGForecaster [31], a robust baseline model that fuses textual cues, such as channel descriptions and dynamic news, and time series data using cross-attention mechanisms. Another approach is taken by TimeLLM [29] that adapts LLMs for time series forecasting by transforming time series into text prototypes and using the prompt-as-prefix technique to guide the transformation with natural language instructions. Other strategies convert the numerical input and output into prompts, and the forecasting task is framed in a sentence-to-sentence manner, making it possible to directly apply language models for forecasting purposes, such as PromptCast [32]. Finally, other models entirely rely on textual information for forecasting [33].

Subsequently, LLMs designed exclusively for time series data emerged, with Moment [34] being an outstanding example. Moment is a family of high-capacity transformer models pretrained on extensive time series datasets from diverse domains using a masked time series prediction task. In contrast to TimeCMA and TimeLLM, which may struggle with modelling long-term dependencies and capturing multi-scale temporal variations, Moment takes advantage of advanced pretraining techniques on large time series corpora, resulting in more robust and generalisable representations. Furthermore, its masked time series prediction task enhances the model’s ability to handle missing and noisy data, challenges frequently encountered in industrial applications. These capabilities enable Moment to surpass previous models in accuracy and adaptability across various time series analysis tasks, including forecasting, classification, and anomaly detection. Moreover, while the majority of the TSFM focuses on time series forecasting, Moment also offers representational learning, which allows users to obtain the values of the latent space where the most important characteristics of the time series are captured, making it a good feature extraction method that requires no training. This was the main reason for selecting this TSFM for our benchmark.

Another multipurpose model that covers forecasting, classification, regression or embedding generation is TOTEM [35]. TOTEM explores time series unification through discrete tokens instead of patches as used in allM4TS [36] and LLM4TS [37]. Its VQVAE (Vector Quantized Variational Autoencoders) backbone learns a fixed codebook of tokens over a multi-domain corpus of time series data independently from the training of any downstream model. This disentangles the choice of data representation from the choice of task-specific architecture and permits the learning of representations from a large, diverse set of data, which aids in zero-shot generalization. However, as far as we know, the authors have not yet published a pretrained version of the model.

Modern DL architectures such as transformers are revolutionizing the world of AI and time series predictions thanks to the multi-head self-attention mechanism introduced in [38], which can capture really complex non-linear relationships in the data and retain long-term relationships in sequences. However, the training of this kind of architecture requires a vast amount of data and extremely powerful specific hardware (such as GPUs, NPUs, or TPUs), making it very costly and time-consuming when compared to traditional ML models. This makes these models more difficult, when not impossible, to implement in fields like PdM, where the failure events are scarce in the datasets and cost reduction is of utmost importance.

Recently, TSFM have emerged, offering zero-shot forecasting and embedding generation for time series data. These models can help overcome the need to train these complex architectures from scratch. In this paper, we aim to experimentally validate whether a hybrid architecture that combines the feature extraction capabilities of TSFM with lightweight and less resource-intensive ML models can achieve the following goals:

Improve or, at least, maintain the prediction performance achieved by DL models in time series regression tasks.

To test our proposal, we designed the following benchmark. First, we chose a well-known RUL dataset, CMAPSS. Then, we selected four ML models from different paradigms: Random Forest, Light Gradient Boosting Machine, Support Vector Regressor, and a simple 1-hidden-layer Perceptron Neural Network. Next, we defined three scenarios to build a predictor with each model: 1.

Using the full original data set.

The 10% subset was selected based on a heuristic, experience-driven approximation, considering that the smallest dataset contains data of about 100 engines. Training with labeled data of 10 engines was deemed sufficient to simulate a realistic few-shot learning scenario while ensuring enough variability for model evaluation (see Experiment 3 in Section 4.5.3).

The predictive performance of the models is evaluated according to the RMSE and the score achieved with the test dataset. Furthermore, eighteen DL and ML models found in the literature for RUL prediction on this dataset were selected to do an honest comparison. Additionally, we measure the training time and inference time to know the computational cost and power consumption of each model.

Section 4 describes step-by-step the process followed and shows the measured metrics. Next, Section 5 discusses the results and draws the main insights extracted from the benchmark.

First, we graphically illustrate the process used to evaluate our approach with the CMAPSS dataset, from the data preparation phase to the result analysis (see Fig. 1). Based on this figure, we then describe the experimentation step by step.

CMAPSS is a well-known synthetic benchmark dataset for the RUL prediction of a type of jet engine released by NASA for the PHM08 data challenge. It contains 4 sub-datasets composed by 26 numerical columns and a target variable representing the RUL. These variables are measured once every operational cycle, and the RUL column represents the number of operational cycles before the failure of the engine. The 4 sub-datasets contain some differences, such as varying working conditions or fault modes. These differences between the sub-datasets are summarized in Table 1.

The aim of this study has been to evaluate the use of TSFM for the RUL prediction on the CMAPSS dataset. We compare the performance of four ML models trained with embeddings generated by the Moment against both the same models trained with the original data and DL models chosen from recent literature. Next, we will answer the questions posed in Section 4.5 based on the results obtained in our benchmark.

The first question asked about the performance improvement achieved by ML techniques using embeddings. The experimentation shows that regarding RSME, this improvement is only achieved with models such as SVR on the most complex datasets, FD002 and FD004. However, it is not a beneficial strategy and may even degrade the performance of tree-based models, Random Forest and Gradient Boosting Machine, that are inherently capable of capturing non-linear relationships directly from the data. Regarding the score, models trained using the embeddings almost always obtain higher (worse) values, mainly due to the sensitivity of the formula to individual poor predictions.

Regarding the behavior of our proposal with respect to other models found in the literature, Table 4 shows that the ML models trained with embeddings offer competitive performance compared to simpler DL models and some hybrid models, especially in more complex data scenarios. However, hybrid attention-based DL models surpass the proposed models across all subsets, especially if a previous specific preprocessing is carried on (see row 16 and 22), albeit at the cost of longer training times and the need for larger datasets.

Another question asked whether models trained using the few-shot learning paradigm would achieve sufficient accuracy. This is where our proposal achieves the best results. Using embeddings improves the predictive performance of SVR and NN when training on a reduced training set, in our case using a 10% of the original training data. Notably, SVR achieves practically the same RMSE (less than a 4% of difference compared to using the full dataset) and outperforms simple DL models such as CNN and LSTM. Additionally, as demonstrated by the comparison using different dataset sizes, this effect becomes more noticeable as the size decreases.

Beyond performance benefits, the use of TSFM can be particularly valuable for organizations aiming to rapidly build and deploy ML models, as it eliminates the need for extensive manual feature engineering or dataset-specific preprocessing. By providing meaningful embeddings, TSFM simplifies the development pipeline, reduces dependency on domain expertise, and enables more scalable and generalizable solutions—aligning with the objectives of AutoML frameworks. This not only accelerates time-to-deployment but also facilitates experimentation across diverse applications with minimal adjustment [28], albeit at the cost of not always achieving the same level of predictive accuracy as highly specialized models.

Concerned with sustainability, we measure time and power consumption to train and use these models. ML models with TSFM embeddings significantly reduce training time compared to DL models while requiring less powerful hardware, and therefore, the pollutant emissions derived from the training are much lower. This is due to the lower complexity of the models and their ability to achieve competitive performance with fewer data samples.

In short, the combination of the Moment TSFM with classical models such as SVR creates a synergistic effect: Moment acts as a universal feature extractor, transforming multidimensional time series into latent representations that effectively preserve underlying degradation patterns. This enables simple models to focus on modelling the relationship between these embeddings and the RUL, eliminating the need for complex architectures or large volumes of labelled data. As a result, computational cost is significantly reduced without compromising predictive accuracy in low-labelled data scenarios. This approach circumvents the need to train DL models from scratch (e.g., LSTMs or transformers), which are often data- and resource-intensive. Instead, the embeddings generated by Moment leverage pre-trained knowledge in a zero-shot fashion and facilitate rapid adaptation to new contexts in few-shot learning setups, offering a practical and efficient solution for industrial scenarios with scarce labeled failure data.

Finally, a practitioner might reasonably consider that integrating the TSFM and the ML model into a unified training process could improve predictive accuracy. However, even though this could potentially yield better results, the limited amount of data in few-shot scenarios may not be enough to improve the performance of the TSFM. Moreover, given the complexity of TSFMs—with millions of parameters—the training process would require significantly more time and computational resources, making it more costly than most state-of-the-art models. This, indeed, would undermine the core motivation of our approach: to build lightweight, fast, and cost-effective predictive models by leveraging the representational power of TSFM without the burden of retraining them.

Limitations of the Study

The study presents certain limitations that should be noted. First, it focuses on the CMAPSS dataset, a synthetic benchmark for RUL prediction. While this synthetic dataset is widely used for comparing the performance of different algorithms in a controlled manner, this does not fully reflect the characteristics of a real environment where external factors influence the data, for example, generating random noise that can interfere in the model’s decision. Therefore, the results of this study may not be fully replicable in real scenarios, and experimentation with real-world datasets that present different common problems, such as strong noise, stationary features, or high proportions of missing data, should be performed.

One way to address this issue, while maintaining the FM-based approach, would be to fine-tune a large model using datasets that accurately capture the relevant characteristics of the problem (domain-aware adaptation), which, if not available, could be generated by means of data augmentation techniques. However, as this solution leads to more computational costs, as an initial step, we propose to experiment with other TSFMs for embedding generation when they are available.

It is also worth noting that the models were built using basic preprocessing techniques to maintain a general approach useful for any context, and to test the ability of FMs to absorb this task. However, the lack of this optimisation may reduce the accuracy of the proposed models.

Estimating the RUL remains an open challenge in predictive maintenance. Although several studies and surveys have been published in recent years addressing this issue from different approaches, the results are not definitive. It has been proven that the best predictors are achieved with hybrid DL models based on a combination of multidimensional attention layers and LSTMs. However, these models require long training times, large volumes of data, and expert knowledge for fine-tuning.

Both the versatility and effectiveness that FM have demonstrated in predictive and representational learning tasks [28] and the recent emergence of TSFM have led us to explore their applicability to RUL prediction. Our goal was to evaluate whether embeddings generated by TSFM are meaningful enough to enable the construction of competitive and simpler predictors with a reduced amount of training data and reduced preprocessing, thus lowering computational costs and processing time.

Our benchmark shows that training simple ML models, such as SVR or NN, with embeddings generated by Moment FM offers a viable alternative to complex DL models, particularly in few-shot scenarios where labeled data is scarce and computational resources are limited.

As future work, we plan to extend this experimentation to other artificial and real-world datasets and to explore the use of other FMs specifically designed for time series under a different architecture, such as TOTEM [35].