Various studies have pointed out that the performance of language models is poor in non-English or non-European languages. One of the factors affecting this performance is the effectiveness and suitability of the tokenization scheme used in the model. Indic scripts require multiple Unicode codepoints to represent a single visual unit to be encoded in the standard UTF-8 scheme. This paper investigates the effect of multiple tokenizers that use UTF-8 text input on the downstream performance of pretrained language models for Hindi and Marathi, languages written in \textit{Devanāgari} script. We present the intrinsic performance of the tokenizers using Fertility, Rényi Efficiency and Percentile Frequency, and report the extrinsic performance of monolingual and multilingual models on question-answering tasks, using an automated parts-of-speech and sentence similarity based evaluation framework, and on word-level tasks such as grapheme-to-phoneme conversion and transliteration. We propose a grapheme cluster tokenizer for the script which shows performance better than or competitive with other popular tokenizers. We also find that the Rényi Efficiency metric is highly correlated to downstream performance on question answering.

A technique for comparing spaceborne microwave radiometer brightness temperatures (Tb) is described in the context of the upcoming National Aeronautics and Space Administration Global Precipitation Measurement (GPM) mission. The GPM mission strategy is to measure precipitation globally with high temporal resolution by using a constellation of satellite radiometers logically united by the GPM core satellite, which will be in a non-sun-synchronous medium inclination orbit. The usefulness of the combined product depends on the consistency of precipitation retrievals from the various microwave radiometers. The Tb calibration requirement to achieve such consistency demands first that Tb's from the individual radiometers be free of instrument and measurement artifacts and, second, that these self-consistent Tb's will be translated to a common standard (GPM core) for the unification of the precipitation retrieval. The intersatellite radiometric calibration technique described herein serves both the purposes by comparing individual radiometer observations to radiative transfer model (RTM) simulations (for “self-consistency” check) and by using a double-difference technique (to establish a linear calibration transfer function from one radiometer to another). This double-difference technique subtracts the RTM-simulated difference from the observed difference between a pair of radiometer Tb's. To establish a linear inter-radiometer calibration transfer function, comparisons at both the cold (ocean) and the warm (land) end of the Tb's are necessary so that, using these two points, slope and offset coefficients are determined. To this end, a simplified calibration transfer technique at the warm end (over the Amazon and Congo rain forest) is introduced. Finally, an error model is described that provides an estimate of the uncertainty of the radiometric bias estimate between comparison radiometer channels.

Passive microwave (PMW) satellite-based precipitation over land algorithms rely on physical models to define the most appropriate channel combinations to use in the retrieval, yet typically require considerable empirical adaptation of the model for use with the satellite measurements. Although low-frequency channels are better suited to measure the emission due to liquid associated with rain, most techniques to date rely on high-frequency, scattering-based schemes since the low-frequency methods are limited to the highly variable land surface background, whose radiometric contribution is substantial and can vary more than the contribution of the rain signal. Thus, emission techniques are generally useless over the majority of the Earth's surface. As a first step toward advancing to globally useful physical retrieval schemes, an intercomparison project was organized to determine the accuracy and variability of several emissivity retrieval schemes. A three-year period (July 2004-June 2007) over different targets with varying surface characteristics was developed. The PMW radiometer data used includes the Special Sensor Microwave Imagers, SSMI Sounder, Advanced Microwave Scanning Radiometer (AMSR-E), Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Advanced Microwave Sounding Units, and Microwave Humidity Sounder, along with land surface model emissivity estimates. Results from three specific targets in North America were examined. While there are notable discrepancies among the estimates, similar seasonal trends and associated variability were noted. Because of differences in the treatment surface temperature in the various techniques, it was found that comparing the product of temperature and emissivity yielded more insight than when comparing the emissivity alone. This product is the major contribution to the overall signal measured by PMW sensors and, if it can be properly retrieved, will improve the utility of emission techniques for over land precipitation retrievals. As a more rigorous means of comparison, these emissivity time series were analyzed jointly with precipitation data sets, to examine the emissivity response immediately following rain events. The results demonstrate that while the emissivity structure can be fairly well characterized for certain surface types, there are other more complex surfaces where the underlying variability is more than can be captured with the PMW channels. The implications for Global Precipitation Measurement-era algorithms suggest that physical retrievals are feasible over vegetated land during the warm seasons.

This letter presents an empirical correction for a time-varying radiometric calibration error (2.5 K peak to peak) present in the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) 1B11 calibrated brightness temperature (Tb) data product. The Tberror is modeled as a function of the spacecraft orbit position and the Sun elevation above the orbit plane (solar beta angle) based on the observed systematic differences between the TMI 10-GHz vertically polarized (10-V) channel oceanic Tb's as compared to a radiative transfer model with collocated numerical weather model environmental parameters. Assuming a slightly emissive reflector, the errors in the other TMI channels are estimated from the 10-GHz error model. This method is evaluated using before- and after-correction comparisons of 1B11 data over the ocean and Amazon, and results show that this method effectively eliminates the time-varying calibration error. This correction technique will be implemented in version 7 of the TRMM 1B11 data product, which is scheduled to be released in 2010.

Recent intersatellite radiometric comparisons of the Tropical Rainfall Measurement Mission Microwave Imager (TMI) with polar orbiting satellite radiometer data and modeled clear-sky radiances have uncovered a time-variable radiometric bias in the TMI brightness temperatures. The bias is consistent with a source that generally cools during orbit night and warms during sunlight exposure. The likely primary source has been identified as a slightly emissive parabolic antenna reflector. This paper presents an empirical brightness temperature correction to TMI based on the position around each orbit and the Sun elevation above the orbit plane. The results of radiometric intercomparisons with WindSat and special sensor microwave imager are presented, which demonstrate the effectiveness of the recommended correction approach based on four years of data.

NASAs Global Precipitation Measurement (GPM) Mission will consist of a constellation of cooperative satellites with microwave radiometers to make global rainfall measurements. It is crucial that the brightness temperature (Tb) measurements from these satellite radiometers be consistent with each other. This may be achieved by the radiometric inter-comparison of near-simultaneous collocated oceanic measurements from pairs of satellites. In this paper, we describe two methods of performing such inter-comparisons. The first uses a radiative transfer model (RTM) to predict expected Tb differences between a pair of radiometer channels due to frequency and incidence angle differences, and then computes the unexplained relative radiometric bias between the channels over a wide variety of environmental conditions. The other approach uses a multi-variable linear regression built from RTM Tbsimulations of all frequency channels for V and H polarizations. These two approaches are used to compare external measurements between the TRMM Microwave Imager (TMI), WindSat (on Coriolis), and SSMI (on DMSP F-13 and F-14).

Launch pads have many cameras that record footage of the space vehicles and the surrounding areas during launches. The footage from many of these cameras is not stable due to camera vibration during ignition and lift-off from the rocket thrust. Aside from making the footage difficult to watch, these instabilities in the background make it harder for viewers to notice any debris that may have fallen from the vehicle, or any other foreign objects. In this paper, we present a method for the automatic stabilization and detection of any debris from such footage. We first stabilize the background by registering consecutive frames and subsequently detect and track moving objects that could be potentially dangerous debris.