A long standing problem of Gian-Carlo Rota for associative algebras is the classification of all linear operators that can be defined on them. In the 1970s, there were only a few known operators, for example, the derivative operator, the difference operator, the average operator and the Rota-Baxter operator. A few more appeared after Rota posed his problem. However, little progress was made to solve this problem in general. In part, this is because the precise meaning of the problem is not so well understood. In this paper, we propose a formulation of the problem using the framework of operated algebras and viewing an associative algebra with a linear operator as one that satisfies a certain operated polynomial identity. To narrow our focus more on the operators that Rota was interested in, we further consider two particular classes of operators, namely, those that generalize differential or Rota-Baxter operators. With the aid of computer algebra, we are able to come up with a list of these two classes of operators, and provide some evidence that these lists may be complete. Our search have revealed quite a few new operators of these types whose properties are expected to be similar to the differential operator and Rota-Baxter operator respectively.Recently, a more unified approach has emerged in related areas, such as difference algebra and differential algebra, and Rota-Baxter algebra and Nijenhuis algebra. The similarities in these theories can be more efficiently explored by advances on Rota's problem.

We describe results on enumerations of sets of Rota-Baxter words in a finite number of generators and a finite number of unary operators. Rota-Baxter words are words formed by concatenating generators and images of words under Rota-Baxter operators. Under suitable conditions, they form canonical bases of free Rota-Baxter algebras and are studied recently in relation to combinatorics, number theory, renormalization in quantum field theory, and operads. Enumeration of a basis is often a first step to choosing a data representation in implementation. Our method applies some simple ideas from formal languages and compositions (ordered partitions) of an integer. We first settle the case of one generator and one operator where both have exponent 1 (the idempotent case). Some integer sequences related to these sets of Rota-Baxter words are known and connected to other combinatorial sequences, such as the Catalan numbers, and others are new. The recurrences satisfied by the generating series of these sequences prompt us to discover an efficient algorithm to enumerate the canonical basis of certain free Rota-Baxter algebras. More general sets of Rota-Baxter words are enumerated with summation techniques related to compositions of integers.