Large pre-trained models excel in zero/few-shot learning for language and vision tasks but face challenges in multivariate time series (TS) forecasting due to diverse data characteristics. Consequently, recent research efforts have focused on developing pre-trained TS forecasting models. These models, whether built from scratch or adapted from large language models (LLMs), excel in zero/few-shot forecasting tasks. However, they are limited by slow performance, high computational demands, and neglect of cross-channel and exogenous correlations. To address this, we introduce Tiny Time Mixers (TTM), a compact model (starting from 1M parameters) with effective transfer learning capabilities, trained exclusively on public TS datasets. TTM, based on the light-weight TSMixer architecture, incorporates innovations like adaptive patching, diverse resolution sampling, and resolution prefix tuning to handle pre-training on varied dataset resolutions with minimal model capacity. Additionally, it employs multi-level modeling to capture channel correlations and infuse exogenous signals during fine-tuning. TTM outperforms existing popular benchmarks in zero/few-shot forecasting by (4-40\%), while reducing computational requirements significantly. Moreover, TTMs are lightweight and can be executed even on CPU-only machines, enhancing usability and fostering wider adoption in resource-constrained environments. The model weights for reproducibility and research use are available at https://huggingface.co/ibm/ttm-research-r2/, while enterprise-use weights under the Apache license can be accessed as follows: the initial TTM-Q variant at https://huggingface.co/ibm-granite/granite-timeseries-ttm-r1, and the latest variants (TTM-B, TTM-E, TTM-A) weights are available at https://huggingface.co/ibm-granite/granite-timeseries-ttm-r2. The source code for the TTM model along with the usage scripts are available at https://github.com/ibm-granite/granite-tsfm/tree/main/tsfm_public/models/tinytimemixer

Selecting the right set of hyperparameters is crucial in time series forecasting. The classical temporal cross-validation framework for hyperparameter optimization (HPO) often leads to poor test performance because of a possible mismatch between validation and test periods. To address this test-validation mismatch, we propose a novel technique, H-Pro to drive HPO via test proxies by exploiting data hierarchies often associated with time series datasets. Since higher-level aggregated time series often show less irregularity and better predictability as compared to the lowest-level time series which can be sparse and intermittent, we optimize the hyperparameters of the lowest-level base-forecaster by leveraging the proxy forecasts for the test period generated from the forecasters at higher levels. H-Pro can be applied on any off-the-shelf machine learning model to perform HPO. We validate the efficacy of our technique with extensive empirical evaluation on five publicly available hierarchical forecasting datasets. Our approach outperforms existing state-of-the-art methods in Tourism, Wiki, and Traffic datasets, and achieves competitive result in Tourism-L dataset, without any model-specific enhancements. Moreover, our method outperforms the winning method of the M5 forecast accuracy competition.

Today’s highly instrumented systems generate large amounts of time series data from many different domains. In order to create meaningful insights from these data, techniques are needed to handle the collection, processing, and analysis at scale. The high frequency and volume of data that is generated introduces several challenges including data transformation, managing concept drift, the operational cost of model re-training and tracking, and scaling hyperparameter optimization.Incremental machine learning can provide a viable solution to handle these kinds of data. Further, distributed machine learning can be an efficient technique to improve performance, increase accuracy, and scale to larger input sizes.In this paper, we introduce a framework that combines the computational capabilities of Apache Spark and the workflow parallelization of Ray for distributed incremental learning. We conduct an empirical analysis of our framework for time series forecasting using the Walmart M5 dataset. The system can perform a parameter search on streaming data with concept drift producing a robust pipeline that fits high-volume data effectively. The results are encouraging and substantiate system proficiency over traditional big data analysis approaches that exclusively use either offline or online training.

A large number of time series forecasting models including traditional statistical models, machine learning models and more recently deep learning have been proposed in the literature. However, choosing the right model along with good parameter values that performs well on a given data is still challenging. Automatically providing a good set of models to users for a given dataset saves both time and effort from using trial-and-error approaches with a wide variety of available models along with parameter optimization. We present AutoAI for Time Series Forecasting (AutoAI-TS) that provides users with a zero configuration (zero-conf) system to efficiently train, optimize and choose best forecasting model among various classes of models for the given dataset. With its flexible zero-conf design, AutoAI-TS automatically performs all the data preparation, model creation, parameter optimization, training and model selection for users and provides a trained model that is ready to use. For given data, AutoAI-TS utilizes a wide variety of models including classical statistical models, Machine Learning (ML) models, statistical-ML hybrid models and deep learning models along with various transformations to create forecasting pipelines. It then evaluates and ranks pipelines using the proposed T-Daub mechanism to choose the best pipeline. The paper describe in detail all the technical aspects of AutoAI-TS along with extensive benchmarking on a variety of real world data sets for various use-cases. Benchmark results show that AutoAI-TS, with no manual configuration from the user, automatically trains and selects pipelines that on average outperform existing state-of-the-art time series forecasting toolkits.

In this paper, we describe an overarching ML system with a simple programming interface that leverages existing AI and ML frameworks to make the task of model exploration easier. The proposed system introduces a new programming construct namely pipeline graph (a directed acyclic graph) consisting of multiple machine learning operations provided by different ML repositories. End user uses the pipeline graph as a common interface for modeling different ML tasks such as classification, regression, and timeseries prediction, while enabling efficient execution on different environments (Spark, Celery and Cloud). We further annotated the pipeline graph with a hyper-parameter grid and an option to try-out a wide range of optimization strategies (i.e., Random, Bayesian, Bandit, AutoLearn, etc). Given a large pre-defined pipeline graph along with its hyper-parameters, we provided a general-purpose, scalable and efficient pipeline-graph exploration technique to provide the automated solutions to a variety of ML tasks. We compare our automated approach to several state-of-the-art automated AI systems and find that we achieve performance comparable to the best results, while often producing simpler pipelines using off the shelf components. Our evaluation suite consists of experiments on 60+ classifications and regressions datasets.

Fueled with growth in the fields of Internet of Things (IoT) and Big Data, data has become one of the most valuable assets in today's world. While we are leveraging this data for analyzing complex systems using machine learning and deep learning, a considerable amount of time and effort is spent on addressing data quality issues. If undetected, data quality issues can cause large deviations in the analysis, misleading data scientists. To ease the effort of identifying and addressing data quality challenges, we introduce DQA, a scalable, automated and interactive data quality advisor. In this paper, we describe the DQA framework, provide detailed description of its components and the benefits of integrating it in a data science process. We propose a programmatic approach for implementing the data quality framework which automatically generates dynamic executable graphs for performing data validations fine-tuned for a given dataset. We discuss the use of DQA to build a library of validation checks common to many applications. We provide insight into how DQA addresses many persistence and usability issues which currently make data cleaning a laborious task for data scientists. Finally, we provide a case study of how DQA is implemented in a realworld system and describe the benefits realized.

Word, sentence and document embeddings have become the cornerstone of most natural language processing-based solutions. The training of an effective embedding depends on a large corpus of relevant documents. However, such corpus is not always available, especially for specialized heavy industries such as oil, mining, or steel. To address the problem, this paper proposes a semi-supervised learning framework to create document corpus and embedding starting from an industry taxonomy, along with a very limited set of relevant positive and negative documents. Our solution organizes candidate documents into a graph and adopts different explore and exploit strategies to iteratively create the corpus and its embedding. At each iteration, two metrics, called Coverage and Context Similarity, are used as proxy to measure the quality of the results. Our experiments demonstrate how an embedding created by our solution is more effective than the one created by processing thousands of industry-specific document pages. We also explore using our embedding in downstream tasks, such as building an industry specific classification model given labeled training data, as well as classifying unlabeled documents according to industry taxonomy terms.

Understanding a mobile user's behavior, e.g., to infer if she is exercising in a gym or dining in a restaurant, is the key to a variety of applications. However, in many real-world scenarios, precisely determining user visitation is extremely challenging due to the uncertainty present in mobile location updates, where errors can be hundreds of meters or even more. We consider the location uncertainty circle determined by the reported location coordinates as the center and the associated location error as the radius. Such a location uncertainty circle is likely to cover multiple location categories, especially in densely populated areas. Worse still, in many cases, mobile users are anonymous, and we have no access to their personal information or other labeled data, which compels us to develop an unsupervised learning approach to solve this problem. Using a user-time-location category tensor, we capture the user behavior and propose a novel tensor factorization framework to accurately infer the location categories visited by mobile users. This framework leverages several key observations including the negative-unlabeled nature of the data and the intrinsic correlations between users. Also, the proposed algorithm can predict where users are even in the absence of location information. To efficiently solve the proposed framework, we propose a parameter-free and scalable optimization algorithm by effectively exploring the sparse and low-rank structure of the tensor. Our empirical studies show that the proposed algorithm is both effective and scalable: it can solve problems with millions of users and billions of location updates, and also provide superior prediction accuracies on real-world location update and check-in datasets.

Increasing use of mobile apps which capture location information has led to wide availability of spatio-temporal data. This paper details our recent efforts on using such data to understand mobile user behaviors in terms of their interaction with apps. Specifically, we aim to mine the association between users' app open (AO) behaviors and their waiting times associated with some transport modes. Here, the transport mode is derived based on speed information measured from a user's location update data, without using any additional map data. One particular case study that we conducted and report here is to understand if users tend to access the app more often while they are waiting at airports. Using a two-week period of a particular iPhone app data from a major U.S. Retailer, the study shows that the app open rate (AR) of air travelers measured during their airport-dwelling time is 8x higher than their AR at other locations. Moreover, for the same group of travelers who have AOs at both airports and other locations, their AR at airports is 45x higher than that at other locations and times. Findings drawn from this study can be applied to assist the definition of geofences by retailers to improve targeting of customers at the right locations, and consequently improve the success of marketing campaigns.

There is a growing interest in leveraging geo-spatial data to provide location-aware services. With a large amount of collected geo-spatial data, a crucial step is to identify important "base" locations (e.g., home or work) and understand users' behavior at these locations. In this paper, we propose an unsupervised collaborative learning approach to identifying home and work locations of individuals from geo-spatial trajectory data. Our approach transforms user trajectory records into intuitive and insightful user-location signatures, clusters these signatures, and then identifies location types based on cluster characteristics. This clustering model can be used to identify base locations for new users. We validate this approach using Open Street Map and Foursquare location tags and obtain an accuracy of 80%.

Location-awareness is becoming increasingly important in wireless networks. Indoor localization can be enabled through wideband or ultra-wide bandwidth (UWB) transmission, due to its fine delay resolution and obstacle-penetration capabilities. A major hurdle is the presence of obstacles that block the line-of-sight (LOS) path between devices, affecting ranging performance and, in turn, localization accuracy. Many techniques have been proposed to address this issue, most of which make modifications to the localization algorithm. Since many localization algorithms work with distance or angle estimates, rather than received waveforms, information inherent in the wideband waveform is lost, leading to sub-optimal ranging error mitigation. To avoid this information loss, we present a novel approach to mitigate ranging errors directly in the physical layer. In contrast to existing techniques, which detect the non-line-of-sight (NLOS) condition, our approach directly mitigates the bias incurred in both LOS and non-LOS conditions. In particular, we apply two classes of non-parametric regressors to form an estimate of the ranging error. Our work is based on, and validated by, an extensive indoor measurement campaign with FCC-compliant UWB radios. The results show that the proposed regressors provide significant performance improvements in various practical localization scenarios, compared to conventional approaches.

Subset diversity (SSD) techniques, which select and combine the signals from a subset of the available diversity branches, are important for practical wireless systems. This paper characterizes the performance loss, or signal-to-noise ratio (SNR) penalty, of one SSD system with respect to another. Both ideal and non-ideal channel estimation are considered, and the analysis is valid for the important case of arbitrary two-dimensional signal constellations. Expressions are given for the asymptotic SNR penalty, for both small and large SNR, for all the comparisons considered. Additionally, we develop bounds and approximations to quantify the performance of one system in terms of another for all SNRs of interest. Furthermore, for some signal constellations, we derive the exact SNR penalty of a non-ideal system with respect to an ideal system, as well as the exact penalty associated with two non-ideal systems with varying degrees of estimation energy. The SNR penalty enables the assessment of system sensitivity to channel estimation energy, combining architecture, and signal constellation.

This paper investigates the number of multipath components in realistic wireless indoor channels. A measurement campaign was performed to collect channel realizations in the Stata Center on the MIT campus. We develop an algorithm with low computational complexity to extract multipath components from the measured waveforms. Based on the results obtained from the measured data, we quantify the behavior of the number of paths with respect to both center frequency and bandwidth.

Sensor networks can benefit greatly from location-awareness, since it allows information gathered by the sensors to be tied to their physical locations. Ultra-wide bandwidth (UWB) transmission is a promising technology for location-aware sensor networks, due to its power efficiency, fine delay resolution, and robust operation in harsh environments. However, the presence of walls and other obstacles presents a significant challenge in terms of localization, as they can result in positively biased distance estimates. We have performed an extensive indoor measurement campaign with FCC-compliant UWB radios to quantify the effect of non-line-of-sight (NLOS) propagation. From these channel pulse responses, we extract features that are representative of the propagation conditions. We then develop classification and regression algorithms based on machine learning techniques, which are capable of: (i) assessing whether a signal was transmitted in LOS or NLOS conditions; and (ii) reducing ranging error caused by NLOS conditions. We evaluate the resulting performance through Monte Carlo simulations and compare with existing techniques. In contrast to common probabilistic approaches that require statistical models of the features, the proposed optimization-based approach is more robust against modeling errors.

Ultra-wide bandwidth (UWB) transmission is a promising technology for indoor localization due to its fine delay resolution and obstacle-penetration capabilities. However, the presence of walls and other obstacles introduces a positive bias in distance estimates, severely degrading localization accuracy. We have performed an extensive indoor measurement campaign with FCC-compliant UWB radios to quantify the effect of non-line-of-sight (NLOS) propagation. Based on this campaign, we extract key features that allow us to distinguish between NLOS and LOS conditions. We then propose a nonparametric approach based on support vector machines for NLOS identification, and compare it with existing parametric (i.e., model-based) approaches. Finally, we evaluate the impact on localization through Monte Carlo simulation. Our results show that it is possible to improve positioning accuracy relying solely on the received UWB signal.

Diversity techniques play a key role in modern wireless systems, whose design benefits from a clear understanding of how these techniques affect system performance. To this aim we propose a simple class of bounds, whose parameters are optimized, on the symbol error probability (SEP) for detection of arbitrary two-dimensional signaling constellations with diversity in the presence of non-ideal channel estimation. Unlike known bounds, the optimized simple bounds are tight for all signal-to-noise ratios (SNRs) of interest. In addition, these bounds are easily invertible, which enables us to obtain bounds on the symbol error outage (SEO) and SNR penalty. As example applications for digital mobile radio, we consider the SEO in log-normal shadowing and the SNR penalty for both maximal ratio diversity, in the case of unequal branch power profile, and subset diversity, in the case of equal branch power profile, with non-ideal channel estimation. The reported lower and upper bounds are extremely tight, that is, within a fraction of a dB from each other.

Diversity techniques will play a key role in next generation wireless communication systems, thus system design will benefit from a clear understanding of how these techniques affect system performance. To this aim we propose simple bounds, optimized within a given class, on the symbol error probability (SEP) in the presence of non-ideal channel estimation for arbitrary two-dimensional signaling constellations. Unlike known bounds, the optimized simple bounds are tight for all signal-to-noise ratios (SNRs). In addition, these bounds are easily invertible, which enables us to obtain bounds on the symbol error outage (SEO) and SNR penalty. As an example application for digital mobile radio, we consider the SEO in log-normal shadowing for both maximal ratio combining with unequal branch power profile and subset microdiversity. The reported lower and upper bounds are extremely tight, that is, within a fraction of a dB from each other.

In modern wireless systems employing diversity techniques, combining all the available diversity branches may not be feasible due to complexity and resource constraints. To alleviate these issues, subset diversity (SSD) systems have been proposed. Here, we develop a framework for evaluating the symbol error probability for antenna SSD, where the signals from a subset of antenna elements are selected and combined in the presence of channel estimation error. We consider independent identically distributed Rayleigh fading channels and use an estimator structure based on the maximum likelihood (ML) estimate which arises naturally as the sample mean of Nppilot symbols. The analysis is valid for arbitrary two-dimensional signaling constellations. The expressions give insight into the performance losses of non-ideal SSD when compared to ideal SSD. Due to estimation error, these losses occur in branch combining as well as in branch selection. However, our analytical results show that the practical ML channel estimator still preserves the diversity order of an ideal SSD system with Ndbranches. Finally, we investigate the asymptotic signal-to-noise ratio penalty due to estimation error.

In this paper, we derive the asymptotic symbol error probability (SEP) of antenna subset diversity (SSD), where the signals from a subset of antenna elements are selected and combined in the presence of channel estimation error. The analysis is valid for arbitrary two-dimensional signaling constellations. We investigate the asymptotic SNR penalty, or performance loss between this system and an ideal system, caused by estimation error. We also compare this SNR penalty to that of a SSD system operating with perfect selection, but imperfect combining. We consider independent identically distributed (i.i.d.) Rayleigh fading channels and use an estimator structure based on the maximum likelihood (ML) estimate which arises naturally as the sample mean of Np pilot symbols. In both cases our analytical results show that the practical ML channel estimator still preserves the diversity order of an ideal SSD system with Nd branches.

We present a framework for evaluating the bit error probability of N/sub d/-branch diversity combining in the presence of non-ideal channel estimates. The estimator structure is based on the maximum likelihood (ML) estimate and arises naturally as the sample mean of N/sub p/ pilot symbols. The framework presented requires only the evaluation of a single integral involving the moment generating function of the norm square of the channel-gain vector, and is applicable to channels with arbitrary distribution, including correlated fading. Our results show that the diversity order of a system with practical channel estimation matches that of an ideal system operating in the same correlated fading environment, regardless of the number of pilot symbols used in the estimation process.