While current machine-learning-based AI techniques have been spectacularly successful, their present applications still leave many important open questions – for example, how to make their results more reliable or, at least, how to gauge how reliable is each AI recommendation. In this paper, we argue that to fully answer these questions, we need to go beyond the current AI techniques, and that in this development, systems-, human-, and cybernetics-based ideas not only naturally appear, they seem to provide a way to the desired answers.

Elon Musk's successful “move fast and break things” strategy is based on the fact that in many cases, we do not need to satisfy all usual constraints to be successful. By sequentially trying smaller number of constraints, he finds the smallest number of constraints that are still needed to succeed - and using this smaller number of constrains leads to a much cheaper (and thus, more practical) design. In this strategy, Musk relies on his intuition - which, as all intuitions, sometimes works and sometimes doesn't. To replace this intuition, we propose an algorithm that minimizes the worst-case cost of finding the smallest number of constraints.

In many practical situations, we need to process data under fuzzy uncertainty: we have fuzzy information about the algorithm’s input, and we want to find the resulting information about the algorithm’s output. It is known that this problem can be reduced to computing the range of the algorithm over several (A) alpha-cuts of the input. However, a straightforward application of this idea requires A times longer computation time than each range estimation – and for complex data processing algorithms, each range computation is already time-consuming. In this paper, we show how to compute all the desired ranges much faster.

In many applications of intelligent computing, we need to choose an appropriate function – e.g., an appropriate re-scaling function, or an appropriate aggregation function. In applications of intelligent techniques, the problem of selecting an optimal function is usually too complex or too imprecise to be solved analytically, so the best functions are found empirically, by trying a large number of alternatives. In this paper, we show that in many such cases, the resulting empirical choice can be explained by natural invariance ideas. Example range from applications to building blocks of intelligent techniques – such as aggregation (including hierarchical aggregation) and averaging – to method-specific (polynomial fuzzy approach, pooling and averaging in deep learning) and domain-specific application, such as describing relative position of 2D and 3D objects, gauging segmentation quality, and perception of delay in public transportation.

Most of our decisions are based on the notion of similarity: we use a decision that helped in similar situations. From this viewpoint, it is important to have, for each pair of situations or objects, a numerical value describing similarity between them. This is called a similarity measure. In some cases, the only information that we can use to estimate the similarity value is some natural distance measure d(a,b). In many such situations, empirical data shows that the similarity measure 1/(1+d) is very effective. In this paper, we provide two explanations for this effectiveness.

In the late 1990s, researchers analyzed what distinguishes great companies from simply good ones. They found several features that are typical for great companies. Interestingly, most of these features seem counter-intuitive. In this paper, we show, on the qualitative level, that from the algorithmic viewpoint, many of these features make perfect sense. Some of the resulting explanations are simple and straightforward, other explanations rely on complex not-well-publicized results from theoretical computer science.

In many practical situations, we need to reconstruct the dependence between quantities x and y based on several situations in which we know both x and y values. Such problems are known as regression problems. Usually, this reconstruction is based on positive examples, when we know y - at least, with some accuracy. However, in addition, we often also know some examples in which we have negative information about y - e.g., we know that y does not belong to a certain interval. In this paper, we show how such negative examples can be used to make the solution to a regression problem more accurate.

Teaching cybersecurity means teaching all possible ways how software can be attacked - and how to fight such attacks. From the usual pedagogical viewpoint, a natural idea seems to be to teach all these ways one by one. Surprisingly, a completely different approach works even better: when the class is divided into sparring mini-teams that try their best to attack each other and defend from each other. In spite of the lack of thoroughness, this approach generates good specialists - but why? In this paper, by analyzing a simple mathematical model of this situation, we explain why this approach work - and, moreover, we show that it is optimal in some reasonable sense.

Most multi-layer neural networks used in deep learning utilize rectified linear neurons. In our previous papers, we showed that if we want to use the exact same activation function for all the neurons, then the rectified linear function is indeed a reasonable choice. However, preliminary analysis shows that for some applications, it is more advantageous to use different activation functions for different neurons - i.e., select a family of activation functions instead, and select the parameters of activation functions of different neurons during training. Specifically, this was shown for a special family of squashing functions that contain rectified linear neurons as a particular case. In this paper, we explain the empirical success of squashing functions by showing that the formulas describing this family follow from natural symmetry requirements.

Often, we use original expert estimates to compute estimates of related quantities. In many practical situations, it is desirable to know how accurate is the resulting estimate. There are many techniques for computing this accuracy: we can use simple probabilistic ideas and we can use simple fuzzy ideas. Strangely enough, these two reasonable techniques lead to drastically different results. Which of them is correct? Our practical tests show that none of these two methods is perfect: probabilistic approach usually underestimates uncertainty, while the fuzzy approach overestimates it. This looks similar to many cases that motivated Zadeh to promote the idea of soft computing – a combination of different uncertainty techniques. To get a more adequate combination technique, we analyzed the general problem of combining accuracy estimates and came up with a 1-parametric family of techniques that contains probabilistic and fuzzy as particular cases – and that indeed works better on several practical examples that each of the original two techniques.

Fuzzy logic is normally used to describe the uncertainty of human knowledge and human reasoning. Physical phenomena are usually described by probabilistic models. In this paper, we show that in extremal conditions, when the concentrations are very large, some formulas describing physical interactions become fuzzy-type. We also show the observable consequences of such fuzzy-type formulas: they lead to bursts of gravitational waves.

To process huge amounts of data, one possibility is to combine some data points into granules, and then process the resulting granules. For each group of data points, if we try to include all data points into a granule, the resulting granule often becomes too wide and thus rather useless; on the other case, if the granule is too narrow, it includes only a few of the corresponding point – and is, thus, also rather useless. The need for the trade-off between coverage and specificity is formalized as the principle of justified granularity. The specific form of this principle depends on the selection of a measure of specificity. Empirical analysis has show that exponential and power law measures of specificity are the most adequate. In this paper, we show that natural symmetries explain this empirically observed efficiency.

If all we know about normalized fuzzy sets is which set is a subset of which, will we be able to detect crisp sets? It is known that we can do it if we allow all possible fuzzy sets, including non-normalized ones. In this paper, we show that a similar detection is possible if we only allow normalized fuzzy sets. We also show that we can detect type-1 fuzzy sets based on the subsethood ordering of normalized interval-valued fuzzy sets.

In the usual [0,1]-based fuzzy logic, the actual numerical value of a fuzzy degree can be different depending on a scale, what is important - and scale-independent - is the order between different values. To make a description of fuzziness more adequate, it is reasonable to consider interval-valued degrees instead of numerical ones. Here also, what is most important is the order between the degrees. If we have only order between the intervals, can we, based on this order, reconstruct the original numerical values - i.e., the degenerate intervals? In this paper, we show that such a reconstruction is indeed possible, moreover, that it is possible under three different definitions of order between numerical values.

One of the main techniques used to de-noise and deblur signals and images is regularization, which is based on the fact that signals and images are usually smoother than noise. Traditional Tikhonov regularization assumes that signals and images are differentiable, but, as Mandelbrot has shown in his fractal theory, many signals and images are not differentiable. To de-noise and de-blur such images, researchers have designed a heuristic method of ℓp-regularization. ℓp-regularization leads to good results, but it is not used as widely as should be, because it lacks a convincing theoretical explanation - and thus, practitioners are often reluctant to use it, especially in critical situations. In this paper, we show that fuzzy techniques provide a theoretical explanation for the ℓp-regularization. Fuzzy techniques also enables us to come up with natural next approximations to be used when the accuracy of the ℓp-based denoising and de-blurring is not sufficient.

In some cases, a membership function μ(x) represents an unknown number, but in many other cases, it represents an unknown crisp set. In this case, for each crisp set S, we can estimate the degree μ(S) to which this set S is the desired one. A natural question is: once we know the values μ(S) corresponding to all possible crisp sets S, can we reconstruct the original membership function? In this paper, we show that the original membership function μ(x) can indeed be uniquely reconstructed from the values μ(S).

In situations when several participants collaborate with each other, it is desirable to come up with a fair way to divide the resulting gain between the participants. Such a fair way was proposed by John von Neumann and Oscar Morgenstern, fathers of the modern game theory. However, in some situations, the von Neumann-Morgenstern solution does not exist. To cover such situations, we propose to use a fuzzy-inspired hierarchical version of the von Neumann-Morgenstern (NM) solution. We prove that, in contrast to the original NM solution, the hierarchical version always exists.

Fuzzy techniques are a successful way to handle expert knowledge, enabling us to capture different degrees of experts' certainty in their statements. To use fuzzy techniques, we need to describe experts' degrees of certainty in numerical terms. Some experts can provide such numbers, but others can only describe their degrees by using natural-language words like “very”, “somewhat”, “to some extent”, etc. In general, all we know about these word-valued degrees is that there is a natural partial order between these degrees: e.g., “very small” is clearly smaller than “somewhat small”. In this paper, we propose a natural way to transform such a partial order between degrees into numerical values.

To more adequately gauge the student's knowledge, it is desirable to take into account not only whether the student's answers on the test are correct or nor, but also how confident the students are in their answers. For example, a situation when a student gives a wrong answer, but understands his/her lack of knowledge on this topic, is not as harmful as the situation when the student is absolutely confident in his/her wrong answer. In this paper, we use the general decision making theory to describe the best way to take into account the student's degree of certainty when evaluating the test results.

One of the main objectives of ecology is to analyze, maintain, and enhance the bio-diversity of different ecosystems. To be able to do that, we need to gauge bio-diversity. Several semi-heuristic diversity indices have been shown to be in good accordance with the intuitive notion of bio-diversity. In this paper, we provide a theoretical justification for these empirically successful techniques. Specifically, we show that the most widely used techniques - Simpson index - can be justified by using simple fuzzy rules, while a more elaborate justification explains all empirically successful diversity indices.

One of the most empirically successful tools for studying dependence between different quantities in econometrics is the tool of vine copulas. In this paper, we explain this empirical success by showing that the most widely used vine copulas are, in effect, the results of using the general fuzzy methodology. To be more precise, vine copulas correspond to a natural extension of the traditional fuzzy methodology, when we allow several different “and”-operations (t-norms), and some of these t-norms can be nonassociative.

In the traditional fuzzy logic, the experts' degrees of confidence in their statements is described by numbers from the interval [0,1]. These degree have a clear intuitive meaning. Somewhat surprisingly, in some applications, it turns out to be useful to also consider different numerical degrees - e.g., complex-valued degrees. While these complex-valued degrees are helpful in solving practical problems, their intuitive meaning is not clear. In this paper, we provide a possible explanation for the success of complex-valued degrees which makes their use more intuitively understandable - namely, we show that these degrees naturally appear due to the approximate nature of the traditional fuzzy methodology.

In the traditional fuzzy logic, the expert's degree of confidence d(A&B) in a complex statement A&B (or A V B) is uniquely determined by his/her degrees of confidence d(A) and d(B) in the statements A and B, as f&(d(A), d(B)) for an appropriate "and"-operation (t-norm). In practice, for the same degrees d(A) and d(B), we may have different degrees d(A&B) depending on the relation between A and B. The best way to take this relation into account is to explicitly elicit the corresponding degrees d(A&B) and d(A V B), i.e., to come up with a "double" fuzzy set. If we only elicit information about pairs of statements, then we still need to estimate, e.g., the degree d(A & B & C) based on the known values d(A), d(B), d(C), d(A & B), d(A & C), and d(B & C). In this paper, we explain how to produce such "and"-operations for "double" fuzzy sets — and how to produce similar "or"-operations.

Traditional probabilistic description of uncertainty is based on additive probability measures. To describe non-probabilistic uncertainty, it is therefore reasonable to consider non-additive measures. An important class of non-additive measures are possibility measures, for which μ(A ∪ B) = max(μ(A), μ(B)). In this paper, we show that possibility measures are, in some sense, universal approximators: for every ε > 0, every non-additive measure which satisfies a certain reasonable boundedness property is equivalent to a measure which is ε-close to a possibility measure.

An expert opinion describes his or her opinion about a quantity by using imprecise ("fuzzy") words from a natural language, such as "small", "medium", "large", etc. Each of these words provides a rather crude description of the corresponding quantity. A natural way to refine this description is to assign degrees to which the observed quantity fits each of the selected words. For example, an expert can say that the value is reasonable small, but to some extent it is medium. In this refined description, we represent each quantity by a tuple of the corresponding degrees. Once we have such a tuple-based information about several quantities x1, xm, and we know that another quantity y is related to x1 by a known relation y = f (x1,..., xm), it is desirable to come up with a resulting tuple-based description of y. In this paper, we describe why a seemingly natural idea for computing such a tuple does not work, and we show how to modify this idea so that it can be used.

In many practical situations, we encounter physical quantities like time for which there is no fixed starting point for measurements: physical properties do not change if we simply change (shift) the starting point. To describe knowledge about such properties, it is desirable to select membership functions which are similarly shift-invariant. We show that while we cannot require that each membership function is shift-invariant, we can require that the linear space of all linear combinations of given membership functions is shift-invariant. We describe all such shift-invariant families of membership functions, and we show that they are naturally related to the corresponding formulas of chemical kinetics.

In many applications, we know the function f (x1 xn), we know the intervals xi of possible values of each quantity xi, and we are interested in the range of possible values of y = f (x1 xn), this problem is known as the problem of interval computations. In other applications, we know the function f (x1 xn), we know the fuzzy sets Xi that describe what we know about each quantity xi, and we are interested in finding the fuzzy set Y corresponding to the quantity y = f (x1 xn), this problem is known as the problem of fuzzy computations. There are many efficient algorithms for solving these problems, however, most of these algorithms implicitly assume that each quantity xi can take any real value within its range. In practice, some quantities are discrete: e.g., xi can describe the number of people. In this paper, we provide feasible algorithms for interval, set, and fuzzy computations for such discrete inputs.

Students do not always spend enough time studying. How can we encourage them to study more? In this paper, we show that a lot depends on the grading policy. At first glance, the problem of grading may seem straightforward: since our objective is that the students gain the largest amount of knowledge and skills at the end of the class, the grade should describe this amount. We show, however, that it is exactly this seemingly straightforward grading policy that often leads to an unfortunate learning behavior. To improve the students' learning, it is therefore necessary to use a grading policy which goes beyond the straightforward approach. In this paper, we use fuzzy-motivated intuition to formulate selection of a grading policy as a precise optimization problem, and, in the first approximation, provide a solution to this optimization problem. This solution is in line with what experienced instructors are actually doing when grading the class.

It is well known that an arbitrary continuous function on a bounded set - e.g., on an interval [a; b] - can be, with any given accuracy, approximated by a polynomial or by a piece-wise polynomial function (spline). Usually, polynomials are described as linear combinations of monomials. It turns out that in many computational problems, it is more efficient to represent each polynomial as a Bernstein polynomial - e.g., for functions of one variable, a linear combination of terms (x - a)k· (b - x)n-k. In this paper, we provide a simple fuzzy-based explanation of why Bernstein polynomials are often more efficient than linear combinations of monomials, and we show how this informal explanation can be transformed into a precise mathematical explanation.

To enhance learning, it is desirable to also let students learn from each other, e.g., by working in groups. It is known that such groupwork can improve learning, but the effect strongly depends on how we divide students into groups. In this paper, based on a first approximation model of student interaction, we describe how to optimally divide students into groups so as to optimize the resulting learning. We hope that, by taking into account other aspects of student interaction, it will be possible to transform our solution into truly optimal practical recommendations.

We show that in many application areas including soft constraints reasonable requirements of scale-invariance lead to polynomial formulas for combining degrees (of certainty, of preference, etc.).

In the 1920s, Pavel Urysohn proved his famous lemma (sometimes referred to as “first non-trivial result of point set topology”). This lemma was instrumental in proving that under reasonable conditions, every topological space can be metrized. Motivated by the success of pseudo-metric spaces in General Relativity, Urysohn started working on an extension of his lemma and of the metrization theorem to (causality-)ordered topological spaces and corresponding pseudo-metrics. By the 1970s, general space-time versions of Urysohn's lemma and metrization theorem have been proven. However, the proofs of these results are not natural - they looks like clever tricks, not like a direct consequence of the definitions. Since one of the main objectives of this activity is to come up with useful applications to physics, we desire more natural versions of these proofs. In this paper, we show that fuzzy logic leads to such natural proofs.

Sometimes, the efficiency of a class is assessed by assessing the amount of knowledge that the students have after taking this class. However, this amount depends not only on the quality of the class, but also on how prepared were the students when they started taking this class. A more adequate assessment should therefore be value-added, estimating the added value that the class brought to the students. In pedagogical practice, there are many value-added assessment models. However, most existing models have two limitations. First, they model the effect of the class as an additive factor independent on the initial knowledge. In reality, the amount of knowledge learned depends on the amount of the initial knowledge. Second, the existing models are statistical, they implicitly assume that the assessment values are objective - and are subject to random measurement errors and noises. In reality, many assessment values are subjective. Thus, fuzzy techniques provide, in our opinion, a more adequate way of processing these values. In this paper, we describe how the use of fuzzy techniques can help us overcome both limitations of the existing value-added assessments.

In 1951, K.J. Arrow proved that, under certain assumptions, it is impossible to have group decision-making rules that satisfy reasonable conditions like symmetry. This Impossibility Theorem is often cited as a proof that reasonable group decision-making is impossible. We start our article by remarking that Arrow's result covers only those situations when the only information we have about individual preferences is their binary preferences between the alternatives. If we follow the main ideas of modern decision making and game theory and also collect information about the preferences between lotteries (i.e., collect the utility values of different alternatives), then reasonable decision-making rules are possible, e.g., Nash's rule in which we select an alternative for which the product of utilities is the largest possible. We also deal with two related issues: how we can detect individual preferences if all we have is preferences of a subgroup and how we take into account the mutual attraction between participants. © 2008 Wiley Periodicals, Inc.

One of the main applications of fuzzy techniques is to formalize the notions of "typical", "representative", etc. The main idea behind fuzzy techniques is that they formalize expert knowledge expressed by words from natural language. In this paper, we show that if we do not use this knowledge, i.e., if we only use the data, then selecting the most representative sample becomes a computationally difficult (NP-hard) problem. Thus, the need to find such samples in reasonable time justifies the use of fuzzy techniques.

Most modern physical theories are formulated in terms of differential equations. As a result, if we know exactly the current state of the world, then this state uniquely determines all the future events - including our own future behavior. This determination seems to contradict the intuitive notion of a free will, according to which we are free to make decisions - decisions which cannot be determined based on the past locations and velocities of the elementary particles. In quantum physics, the situation is somewhat better in the sense that we cannot determine the exact behavior, but we can still determine the quantum state, and thus, we can determine the probabilities of different behaviors - which is still inconsistent with our intuition. This inconsistency does not mean, of course, that we can practically predict our future behavior; however, in view of many physicists and philosophers, even the theoretical inconsistency is somewhat troubling. Some of these researchers feel that it is desirable to modify physical equations in such a way that such a counter-intuitive determination would no longer be possible. In this paper, we analyze the foundations for such possible theories, and show that on the level of simple mechanics, the formalization of a free will requires triple interactions - while traditional physics is based on pairwise interactions between the particles.

Computer modeling programs that generate three-dimensional (3-D) data on fine grids are capable of generating very large amounts of information. These data sets, as well as 3-D sensor/measured data sets, are prime candidates for the application of data compression algorithms. A very flexible and powerful compression algorithm for imagery data is the newly released JPEG 2000 standard. JPEG 2000 also has the capability to compress volumetric data, as described in Part 2 of the standard, by treating the 3-D data as separate slices. As a decoder standard, JPEG 2000 does not describe any specific method to allocate bits among the separate slices. This paper proposes two new bit allocation algorithms for accomplishing this task. The first procedure is rate distortion optimal (for mean squared error), and is conceptually similar to postcompression rate distortion optimization used for coding codeblocks within JPEG 2000. The disadvantage of this approach is its high computational complexity. The second bit allocation algorithm, here called the mixed model (MM) approach, mathematically models each slice's rate distortion curve using two distinct regions to get more accurate modeling at low bit rates. These two bit allocation algorithms are applied to a 3-D Meteorological data set. Test results show that the MM approach gives distortion results that are nearly identical to the optimal approach, while significantly reducing computational complexity.

According to the traditional probability theory, events with a positive but very small probability can occur (although very rarely). For example, from the purely mathematical viewpoint, it is possible that the thermal motion of all the molecules in a coffee cup goes in the same direction, so this cup will start lifting up. In contrast, physicists believe that events with extremely small probability cannot occur. In this paper, we show that to get a consistent formalization of this belief, we need, in addition to the original probability measure, to also consider a maxitive (possibility) measure

The problem of optimal bit rate allocation for 3-D JPEG2000 compression is dicussed. In this paper we consider data where the 2-D slices are compressed using JPEG2000, and the third dimension is decorrelated using the Karhunen-Loeve Transform. Here two new methods are proposed. The first approach is called the Rate Distortion Optimal (RDO) method and is based on Post-Compression Rate-Distortion (PCRD) optimization concept. The second approach is here called the Mixed Model (MM) approach and consists of extending the traditional high-resolution model with a region that is accurate for low bit rates. The proposed bit allocation methods are tested by applying them to Meteorological (Met) data. The specific data set used was generated by the Battlescale Forecast Model (BFM). The test results shows that these approach significantly reduces the computational complexity.

This paper focuses on the parameters selection that optimizes JPEG 2000 compression for meteorological data. In particular, a procedure for bit rate allocation for different slices of data is proposed. The selection is based on the variances of individual slices and a mixed model approximation of MSE. The solution of the constrained minimization problem for MSE is obtained using Lagrange multipliers. This procedure allows the incorporation of other constraints in this problem such as noninteger bit rates, limited range of possible bit rates, etc. Experimental results are given for all six variables. The reduction of MSE also led to the reduction of maximum absolute error in most cases.

The formal concept of logical equivalence in fuzzy logic, while theoretically sound, seems impractical. The misinterpretation of this concept has led to some pessimistic conclusions. Motivated by practical interpretation of truth values for fuzzy propositions, we take the class (lattice) of all subintervals of the unit interval [0, 1] as the truth value space for fuzzy logic, subsuming the traditional class of numerical truth values from [0, 1]. The associated concept of logical equivalence is stronger than the traditional one. Technically, we are dealing with much smaller set of pairs of equivalent formulas, so that we are able to check equivalence algorithmically. The checking is done by showing that our strong equivalence notion coincides with the equivalence in logic programming. © 1996 John Wiley & Sons, Inc.