With the growing complexity and dynamism of cloud environments, users of operations management solutions are facing a critical headache of "event storms". Understanding and prioritizing reactions to such high volumes of noisy recommendation content for various tasks is beyond the capacities of human operators. This significantly degrades the resolution metrics of performance issues and optimization of infrastructures and applications. We have devised a novel streaming clustering algorithm for processing alerts and discovering Alert Episodes with their evolution tracked in time and space. It is based on the principles of the classical density-based clustering DBSCAN. We learn Unknown Problems applying this algorithm to low-level events within the VMware Aria Operations manager. Those episodes might typically be out of alert definitions coverage and explain new types of emerging incidents. Our solutions with different hyperparameters are prototyped and integrated into the production. We share experimental insights from an internal environment with interesting alert episodes learned and unknown problems of alarms/symptoms discovered with a self-explainable story on where the source of the performance issue stays and how it evolved into a larger problem situation affecting several objects and hierarchy layers. The constructs we introduce help reduce user efforts in making sense of events' waves and perform troubleshooting with relevance. Our solution can be refactored into an independent event management service for cloud operations.

We examine the determination of abnormality of streamed data using the statistical structure of the meta-data associated with it. The vital need for such a subject within a heterogeneous log based environment in real-time comes from the fact that most cloud based applications will use text-based logging as a means of reporting application behavior. The sheer volume of such logs makes retrospective analysis infeasible due to large processing and storage requirements. Our approach is based on conversion of the original data stream into meta-data (graph) and revealing the dominating (normal) statistical patterns within it. Real-time analysis of the stream compared with the meta-data model determines the degree of anomaly of the current data. The resulting graph also reveals the fundamental structure (“behavioral footprint”) of the data beyond the sources (physical or virtual devices) and processes.

We study the problem of multiple hypothesis testing (HT) in view of a rejection option. That model of HT has many different applications. Errors in testing of M hypotheses regarding the source distribution with an option of rejecting all those hypotheses are considered. The source is discrete and arbitrarily varying (AVS). The tradeoffs among error probability exponents/reliabilities associated with false acceptance of rejection decision and false rejection of true distribution are investigated, the optimal decision strategies are outlined. The special case of discrete memoryless source (DMS) is also discussed. An interesting insight that the analysis implies is the phenomenon (comprehensible in terms of supervised/unsupervised learning) that in optimal discrimination within M hypothetical distributions one permits always lower error than in deciding to decline the set of hypotheses. Geometric interpretations of the optimal decision schemes and bounds in multi-HT for AVS's are given.

The problem of multiple hypothesis testing (HT) for arbitrarily varying sources (AVS) is considered. The achievable error probability exponents (reliabilities) region is derived, optimal decision schemes are described. The result extends the known ones by Fu and Shen and by Tuncel. The Chernoff bounds for AVS binary and M-ary HT are specified via indication of a Sanov theorem for those sources.

Optical orthogonal codes (OOC) are special classes of codes for applications in code division multi-access (CDMA) transmission using an optical carrier. We show a new relationship between constant weight cyclically permutable codes (CPC) and OOC's which can be investigated further. The main result ties together the CPC's and the solution of an algebraic system of congruences. For a particular case, there is a recurrence relationship for the code cardinality. More general cases need to be elaborated in coming research. We discuss also so-called super OOC's, which are codes with stronger correlation properties, and show several examples of such codes derivable from CPC's designed by applying the method proposed.

The Shannon rate-distortion function for arbitrary varying source (AVS) coding subject to error exponent or reliability criterion E > 0 is set up. In other words, the E-optimal (asymptotic lossy coding) rate R ≥ 0 for AVS under given requirement on reconstruction fidelity Δ ≥ 0 is specified. Using the already known terminology we call that interdependence the AVS rate-reliability-distortion function. We indicate the maximum error exponent as a function of the coding rate and distortion threshold in AVS coding as well. In zero-distortion case, our general formulas specialize the ones derived by Fu and Shen from their hypothesis testing analysis for AVS.

Some important properties of the rate-reliability-distortion function of discrete memoryless source (DMS) are established. For the binary source and Hamming distortion measure this function is derived and analyzed. Even that elementary case suffices to show the nonconvexity of the rate-reliability-distortion function in the reliability (error exponent) argument.

A source coding problem is considered for a robust description system with one encoder and many decoders. The rate-reliability-distortion function is specified. An example is given showing the distinction in the calculation of the rate-reliability-distortion function as compared with the rate-distortion function.