SuperSQL is an extended SQL language, which brings out a rich layout presentation of a relational database with a particular query. This paper proposes SSstory, a storytelling system in a 3D data space created by a relational database. SSstory uses SuperSQL and Unity to generate a data video and add cinematic directions to the data video. Without learning special authoring tooling, users can easily create data videos with a small quantity of code.

In this work, we used the Web IndeX system, which converts words into hyperlinks in arbitrary Web pages, to implement a system for sharing annotations registered on keywords within a limited group. This system allows group members to view all the written annotations by simply mousing over the keyword when it appears on a web page, facilitating sharing information and awareness in collaborative research and work. In this study, we define our system as a sticky note type annotation sharing system. In contrast to the sticky note type, our system is positioned as a functional type, but it may display annotations that are not necessary since it allows browsing on any page. To improve this point, we propose a usability judgment method that shows only the most useful posts.

SuperSQL is an extension of SQL that allows you to structure the output of relational databases by writing your own queries and to express various layouts. However, this method is not suitable for data with high update frequency, such as stream data, because the information in the database refers to the data at the time of SuperSQL execution. In this study, we propose an implementation of a web page generation function that asynchronously updates a web page with the latest information for frequently updated data, using PipelineDB and SuperSQL, both of which are DBMSs capable of processing streams. You can specify the dynamic part of the stream by specifying the stream in the "decorator" which is a feature of SuperSQL. At the same time, you can specify "pull" and "push" in the stream decorator to select how the dynamic part is updated. This makes it possible to create a web page that displays the latest stock prices at any time in a page that displays a list of stock prices.

Most recommendation systems excessively pursue the recommendation accuracy and give rise to over-specialization. However, the existing recommendation systems research has not studied serendipity much. Hence, the serendipitous item recommendation has received more attention in recent years. The serendipitous recommendation of our research is not included in the area that the user predict easily but recommends the keywords that match the potential preferences. Potential preferences are those that are present in the user profile, which the user may not know. In this research, we recommend keywords that can express serendipity by intersecting the relation between keywords mainly. Furthermore, we propose the related page recommendation method on Web IndeX System for recommending linked pages related to these serendipitous keywords based on the user's potential preferences.

SuperSQL is an extended language of SQL. By structuring the output of relational databases, SuperSQL enables the user to generate various types of structured documents with various layouts which are not represented in SQL. There is a problem that the larger and more complicated the SuperSQL query is, the more difficult it is to detect errors and the more time is spent on debugging. In this study, we propose a system that automatically detects and corrects syntax errors in user queries. When a query parsing fails, the system reanalyzes the query and predicts a correction by using deep learning. To modify the query, we use recurrent neural network and attention mechanism. By presenting the predicted modifications to users, the burden of debugging can be reduced and the efficiency of user's work can be improved.

Web Index (WIX in short) is a system that achieves joining information resources on the Web. WIX replaces keywords in Web documents hyperlinks to other web pages based on a WIX file that a user chose. WIX file is a kind of a dictionary that have a set of WIX entries (keyword and target URL). Using WIX, users can join any Web contents and arbitrary dictionaries. In conventional WIX, matching and linking are executed only for fixed character strings between the keyword set and the input text. However, when a user wants to search for phrases like idioms, this matching system is not sufficient because of the declension of words, change of the verb tense, and so on. Therefore, we propose a phrasal pattern matching mechanism on WIX. This helps users easily find idiom expressions in the text on the web and get more information.

SuperSQL is an extension of SQL which generates data in various formats like HTML, PDF, XML, among many others. The same data is represented in different forms according to the user, due to which it is called a data representation and publishing language. This research is to provide help in testing the SuperSQL processor.

Recently, the push for open data has been very strong, and more and more sources, such as governments are sharing data such as weather records or demographic statistics. The Remote Table Access (RTA) system allows the easy publication of data from a relational database, and its use through SQL-like queries by remote users. Still, the data currently being shared as open data comes in many formats, not always directly integrable with relational databases, and many sources publish data as raw CSV, XML or even PDF files. These files then need to be downloaded, parsed, and integrated with the final user's data, often in a relational database. In this work, we present Table on Top (ToT) for CSV, an extension of the RTA system that allows the easy publication and access of data contained in CSV files through RTA.

When exploring data or communicating it to other people, data is currently visualized through flat diagrams, tables, graphs, etc. Visualization of data in three dimensions (3D) offers more immersive and intuitive representations of the data and, through the added dimension, allows for more compact representations. Still, when representing large amounts of data in 3D, a fine control of the layout becomes a must. Current tools for 3D visualization do not allow for easy and fine tuned control of this layout.

The Rule-based e-Mail eXchange system, also known as RMX, is an email transfer agent that transfers email based on user-defined delivery rules which are expressed as parameterized SQL queries. In this paper, we introduce insertion rules, which are also expressed as parameterized SQL queries based on the recipient's email address to derive values such as recipient's name or the list of purchase records to be embedded in the email body and header. By using the full expressive power of SQL, RMX allows arbitrary information in a relational database to be embedded at the moment of delivery. A straightforward implementation of insertion rules, however, invokes numerous SQL executions which are proportional both to the number of inserted items K and the number of recipients N. We have developed a query synthesis algorithm which derives a single SQL query from the delivery rule query and the K insertion rule queries. The 1 + N × K original query executions will be replaced by a single execution of the resulting query. Furthermore, we introduce a heuristic query simplification algorithm to reduce redundant references to the same relation and the joins between them generated by the naive synthesis algorithm. As an example, delivering 5000 emails each containing four embedded items with the straightforward implementation required 20001 SQL query executions that took 650.50 seconds to complete. This overhead was reduced to a single execution taking 304.69 milliseconds to complete with the naive query synthesis. The query simplification further reduced the time to 148.74 milliseconds by reducing four joins to one.

We study task composition in crowdsourcing and the effect of personalization and diversity on performance. A central process in crowdsourcing is task assignment, the mechanism through which workers find tasks. On popular platforms such as Amazon Mechanical Turk, task assignment is facilitated by the ability to sort tasks by dimensions such as creation date or reward amount. Task composition improves task assignment by producing for each worker, a personalized summary of tasks, referred to as a Composite Task (CT). We propose different ways of producing CTs and formulate an optimization problem that finds for a worker, the most relevant and diverse CTs. We show empirically that workers' experience is greatly improved due to personalization that enforces an adequation of CTs with workers' skills and preferences. We also study and formalize various ways of diversifying tasks in each CT. Task diversity is grounded in organization studies that have shown its impact on worker motivation [33]. Our experiments show that diverse CTs contribute to improving outcome quality. More specifically, we show that while task throughput and worker retention are best with ranked lists, crowdwork quality reaches its best with CTs diversified by requesters, thereby confirming that workers look to expose their “good” work to many requesters.

International audience

There are currently massive amounts of public data, also refereed to as open data, for example stock price data or weather data. However, such data is distributed in a variety of ways, such as downloadable files like CSV or XML files, or through API calls to web services. Each data source thus requires a specific workflow, making it a burden for the users to process and use this data. This barrier to use diminishes the openness of this data We thus propose the Remote Table Access (RTA) system, a simple and safe architecture for publishing, i.e. giving open read only access to relational data, and easily integrating it with the user's local data. RTA enables the user to query relational open data and their own local data seamlessly through a single SQL query. To allow this, we designed a three parties architecture featuring a client-side application, an optional server-side module and a "Public Table Library" (PTL). The client side application processes the RTA query and fetches the necessary data, the server side system acts as an agent between the remote database and the client, offering added security as well as scalability in terms of connections, and the PTL list all the published data and stores its access information. We implemented an early prototype of this architecture as a proof of concept. We validated it against two datasets, including data from the TPC-C benchmark and make it available1. Our results show the feasability of RTA and possible significant reduction of query processing time mainly because of the reduction on transmission volume by condition pushing and semijoin.

Virtual Reality is gaining traction as a medium. But generating virtual reality scenes is still a challenge as manipulating and arranging 3D objects is a complex task requiring the mastery of specialized tools. SuperSQL is an extension of the SQL query language that lets users format relational data into different kinds of structured documents. In this paper we propose an extension of SuperSQL that generates a virtual reality scene from information about 3D objects stored in a relational database. This system enables users with no knowledge of virtual reality tools to create 3D scenes such as a virtual museum using a simple SQL-like query.

Recently, infinite scrolling has become a trend in loading massive data in webpages. This feature allows contents of a webpage to be divided and loaded automatically as a user scrolls through the page. Integrating infinite scroll in webpages, however, can be a complex task. A developer needs to be proficient in client-side programming to design the webpages and server-side programming to be able to load data dynamically. In this study, we propose an approach based on SuperSQL to simplify the integration of infinite scroll in web pages. SuperSQL is an extension of SQL that outputs data extracted from a database in various types of structured documents such as HTML, directly as a result of a declarative query. We extend SuperSQL to implement infinite scrolling in an HTML query output. As a result, developing web pages with the infinite scroll feature can be achieved by running a single SuperSQL query.

Crowdsourcing has gained popularity in a variety of domains as an increasing number of jobs are "taskified" and completed independently by a set of workers. A central process in crowdsourcing is the mechanism through which workers find tasks. On popular platforms such as Amazon Mechanical Turk, tasks can be sorted by dimensions such as creation date or reward amount. Research efforts on task assignment have focused on adopting a requester-centric approach whereby tasks are proposed to workers in order to maximize overall task throughput, result quality and cost. In this paper, we advocate the need to complement that with a worker-centric approach to task assignment, and examine the problem of producing, for each worker, a personalized summary of tasks that preserves overall task throughput. We formalize task composition for workers as an optimization problem that finds a representative set of k valid and relevant Composite Tasks (CTs). Validity enforces that a composite task complies with the task arrival rate and satisfies the worker's expected wage. Relevance imposes that tasks match the worker's qualifications. We show empirically that workers' experience is greatly improved due to task homogeneity in each CT and to the adequation of CTs with workers' skills. As a result task throughput is improved.

As the use of crowdsourcing spreads, the need to ensure the quality of crowdsourced work is magnified. While quality control in crowdsourcing has been widely studied, established mechanisms may still be improved to take into account other factors that affect quality. However, since crowdsourcing relies on humans, it is difficult to identify and consider all factors affecting quality. In this study, we conduct an initial investigation on the effect of crowd type and task complexity on work quality by crowdsourcing a simple and more complex version of a data extraction task to paid and unpaid crowds. We then measure the quality of the results in terms of its similarity to a gold standard data set. Our experiments show that the unpaid crowd produces results of high quality regardless of the type of task while the paid crowd yields better results in simple tasks. We intend to extend our work to integrate existing quality control mechanisms and perform more experiments with more varied crowd members.

A lot of time and knowledge are necessary for the Mobile Web application development. In this study, we have implemented the Mobile Web application generation features for SuperSQL, which generates HTML files, JavaScript files, and server-side PHP files as the result of a query execution. This feature consists of 55 new functions of SuperSQL. In the evaluation, we have created three Mobile Web applications and compared the amount of lines of code with other popular technologies. As a result, the extended SuperSQL achieved 97% reduction of code amount compared to HTML / JavaScript / PHP and 89% reduction compared to Ruby on Rails / JavaScript.

Generation of modern web pages is a difficult task for novice programmers because requiring to be familiar with many programming languages, HTML, CSS, JavaScript, etc. SuperSQL, a project of Toyama laboratory, is an extension of SQL that automatically formats data retrieved from the database into various kinds of structured documents as an output of a query.

With the rapid spread of smartphones and tablets, it is becoming necessary for web developers to create responsive web pages which are visually appealing on devices of various sizes. However, building responsive UIs is a very challenging task, requiring deep knowledge of HTML and CSS. In this paper, we propose an approach to generate responsive web pages using SuperSQL, which is an extension of SQL that can format data retrieved from a database into various kinds of structured documents. Our approach applies the methodology of Bootstrap, a grid-based framework for front-end development, to generate responsive web pages from SuperSQL queries. By combining SuperSQL's capability of expressing complex layout structure with the systematic use of Bootstrap, we aim to establish an uncomplicated method of developing responsive web pages that do not require expertise in front-end web development.

On the rule-based email exchange system called RMX, similar to general mailing lists, anyone can send emails by sending to an address unique to RMX. However, there is a security problem that we cannot prevent spam emails and accidentally sending email that may cause a leak of information. In this paper, to resolve this problem, we propose to introduce the authorization mechanism that authorizes a sender to send emails based on authorization rules that the users set in advance into RMX. This mechanism enables sending limitation, and if a sender tries to send to unauthorized recipients, a warning message is returned to the sender.

In general, a search engine is used to obtain information about specified keywords of interest. However, users must go through the list of web pages presented by the search engine in order to find the page that meets the purpose. In order to reduce this burden, we propose Web Index (WIX), a hyperlink generation system that achieves joining information resources on the web. The WIX system replaces keywords that appear in Web documents on browser into hyperlinks to a specific web page group of the user's choice. However, when there are multiple URLs paired up with a keyword, there is a need to choose the web page that meets the user's purpose. In this paper, we propose WIX System and an architecture that decides and presents likely candidates for hyperlink destination based on similarity of URLs and the content of each candidate.

Due to the amount of work needed in manual sentiment analysis of written texts, techniques in automatic sentiment analysis have been widely studied. However, compared to manual sentiment analysis, the accuracy of automatic systems range only from low to medium. In this study, we solve a sentiment analysis problem by crowdsourcing. Crowdsourcing is a problem solving approach that uses the cognitive power of people to achieve specific computational goals. It is implemented through an online platform, which can either be paid or volunteer-based. We deploy crowdsourcing applications in paid and volunteer-based platforms to classify teaching evaluation comments from students. We present a comparison of the results produced by crowdsourcing, manual sentiment analysis, and an existing automatic sentiment analysis system. Our findings show that the crowdsourced sentiment analysis in both paid and volunteer-based platforms are considerably more accurate than the automatic sentiment analysis algorithm but still fail to achieve high accuracy compared to the manual method. To improve accuracy, the effect of increasing the size of the crowd could be explored in the future.

The Rule-based e-Mail eXchange system, also known as RMX, is a mail transfer agent that transmits emails based on user-defined rules, which are applied to a relational database. It is a novel way of sending email to various groups of people. However, the creation of the RMX operating environment is not simple and is relatively costly. Furthermore, in order to use the system, users usually configurate the system using a command line interface. Consequently, RMX users cannot perform intuitive operations. In this study, we develop the web application to easily provide RMX as a platform to users. As a result, users can intuitively utilize RMX with considerably reduced cost in the operating environment.

Recently, a demand for a mobile-friendly web page is rising, but creating and maintaining separate sites for mobile and desktop is costly for web developers. At our laboratory, we have been designing and developing SuperSQL, an extension of SQL that can generate HTML files that contain values stored in RDBs. In this paper, we propose an approach to generate both mobile and desktop versions of a web page with just one SuperSQL query. We believe our approach can facilitate the laborious work of creating and maintaining separate sites for mobile and desktop environments.

This paper introduces a pair of query editors for SuperSQL: SSedit and SSvisual. SSedit is a structured editor specialized for SuperSQL, which is mainly used to create a query statement from scratch. SSvisual is a WYSIWYG editor, which is mainly used to fine tune the layout and visual effects on HTML.

Web Index (WIX) is a hyperlink generation system that achieves joining information resources on the web. The WIX system takes a set of pairs of keywords and URLs written in XML (called WIX Files) and join it to the text content of a web page in order to transform keywords into hyperlinks to a specific web page group of the user's choice. In the previous WIX system, synonymous expressions of the same entity had to be directly added to the WIX File in order to generate hyperlinks for such expressions. In this paper, we propose a synonym processing mechanism, which generates the hyperlink for synonyms without adding them to the WIX File directly. We collected synonymous relations from the redirection function of the Japanese version of Wikipedia and constructed a synonym database for our system. We incorporated the information of the Synonym database into the automaton based on the Aho-Corasick algorithm used for the lexicographic matching process of the WIX system, and achieved to generate hyperlinks on synonymous expressions without barely changing the size of our WIX File database.

The World Wide Web has become an invaluable source of data. However, extracting useful information from the vastness of the web can become a challenge as depending on the amount of data needed, manual extraction or creation of web scraping programs may be necessary. These processes can be tedious and complicated. To address these, Ducky, a web wrapper that extracts data from web sources and translates them into structured data based on a user-defined configuration, has been developed. Ducky is able to extract data flexibly from various structured web pages, remove noise from extracted data and integrate multiple pages from different sites. In addition, the current version of Ducky automatically extracts data from Wikipedia and trendy keywords of Google and Yahoo.

SuperSQL is a database publishing/presentation extension of SQL that can generate various kinds of structured documents that contain values stored in RDBs. Thanks to a useful tools, it is becomes easily to creating web application. However, users need a lot of knowledge about various programing languages such as HTML, CSS, PHP, JavaScript, and so on to create Web applications.

Mailing lists are widespread tools to communicate and share information with each other. Especially, organizations maintain so many of them for collaborative works. Because of conventional mailing schemes, it requires constant administration from its initiation to its maintenance. In this paper, we propose a rule-based mailing system called RMX where e-mail is delivered based on rules and parameters, instead of recipients' bare e-mail address or manually maintained mailing lists. By using this rule-based mailing approach, the administrator need not manage mailing lists since it guarantees a single point of administration by involving the organization's database and rules defined in SQL.

Content Management Systems (CMS) have been used recently to facilitate dynamic website creation and enable division of labor in web authoring. However, even with the use of CMS, the hyperlinks in websites must be created for every page in the website by author's handwriting. Furthermore, in changing URL, you have to manually change all corresponded hyperlinks one by one. In this study, we develop a CMS plugin using the Web Index System, which allows automate creation of hyperlinks, to collectively manage hyperlinks of invariant words. As a result, the workload involved in the management of hyperlinks in a webpage is considerably reduced and ease of website maintenance is improved.

SuperSQL is an extension of SQL that automatically formats data retrieved from the database into various kinds of application data (HTML, PDF...). Current developments lead us to identify improvement points and remodel the design of the SuperSQL architecture. Among them, in the current SuperSQL version, the emptiness of one single relation leads to the emptiness of the entire table forming the output data. This is because the process handling the retrieval of desired data does not consider the schema representation of the data and thus does not identify independence between data lists. In this paper, we propose a new process of data retrieval based on a three layers model: the definition layer, the equivalence layer and the optimisation layer. As a result, our proposed architecture is able to manage empty sets and allows easier integration to support future developments.

The Rule-based e-Mail eXchange system, also known as RMX, is a mail transfer agent that transfers emails based on user-defined rules, which are applied to a relational database. It is a novel way of sending email to various groups of people, compared to conventional email. However, the creation of the RMX operating environment is not simple and is relatively costly. Furthermore, in order to use the system, users must have a knowledge of the database and invoke the system using a command line interface. Consequently, RMX users cannot perform operations intuitively. To address these concerns, we develop two web applications to easily provide RMX as a service and RMX as a platform. As a result, users can intuitively deploy mailing lists using RMX and incorporate RMX to other web applications and utilize RMX with considerably reduced cost in the operating environment.

To benefit from the invaluable data in the World Wide Web, manual extraction or creation of web scraping programs may be necessary. However, these processes can be tedious and complicated. To address these, we have proposed Ducky, which is a Web data extraction system including a web wrapper that extracts data from web sources and translates them into structured data based on user-defined data extraction rules. Ducky can extract data flexibly from various structured web pages, remove noise from extracted data and integrate data distributed to multiple pages from different sites. In this paper, we propose a browser GUI for Ducky. Instead of manually writing a configuration file, users can just click or point a cursor (mouse over) to objective elements. The users' actions are then automatically converted to data extraction rules and saved in a configuration file. Thus, we help users to extract the data by allowing intuitive operations and reduce users' burden in write the configuration file.

The World Wide Web has become a primary source of information. Therefore, extracting data from Web sources has become a key technology. In this paper, we introduce a semi-automatic system Ducky: including a Web Wrapper which extracts data from Web sources and translates them into structured data. In Ducky, by defining a configuration file consisting in several parameters (URL of the Web page, CSS selectors which locates the data to retrieve and so on.), users do not need to write Web scraping programs at all. The definition is simple, yet can extract data flexibly from various structured Web pages. Additionally, Ducky provides a Web API and various output data formats: XML, JSON, CSV. Finally, experimentations confirmed that Ducky can accurately extract data from 22 different structured Web sources.

PPX(pretty printer for XML) is a query language for XML database which has extensive formatting capability that produces HTML as the result of a query. In this paper, we report about a prototype implementation of PPX processor. In the experiment, a quick comparison shows that PPX requires far less description compared to XSLT or XQuery programs doing same tasks.

Sensor database systems need to provide both freshness of data and persistence to the incoming sensor streams. To provide persistence, a disk based logging method has been widely used, however it is not applicable for sensor streams because of its tardiness. In this paper, we propose the light neighbor write ahead logging protocol(L-WAL) for sensor streams. The L-WAL is a refinement of the neighbor-WAL (N-WAL). The L-WAL needs two network interfaces and applies a relaxed protocol rather than a two phase commit protocol. Since the relaxed protocol weakens the guarantee of logging successes, we have incorporated a repair system and checker system to enhance the guarantee. The result of experiments shows that the L-WAL is about 2.13 times faster than the N-WAL when the number of concurrent sensor streams is 250 and the L-WAL can enhance the persistence of data almost for free, while the N-WAL needs to pay high cost.

The star schema is often used in dimensional approaches applied to OLAP applications. The fact table in the star schema typically contains a huge amount of data. When some of the dimension tables are also very large, it may take too much time and storage to join the fact table with these dimension tables. The performance of the join algorithm becomes critical under such a condition. The fluent join is a join algorithm that operates on relations organized as multidimensional linear hash files. Like a merge join on relations which are already sorted on the joining key, its execution reads each page in the operand relations no more than once and does not create intermediate result files. Unlike sorting, the multi-dimensional linear hash can cluster records in several keys symmetrically. In this paper, the concept of the fluent join is applied to an OLAP system to cluster records in each table on the joining keys. As a result, the algorithm yields symmetric performances on joins with different dimension tables.

SuperSQL is an extension of SQL that allows query results presented in various media for publishing and presentations with simple but sophisticated formatting capabilities. SuperSQL query can generate various kinds of materials, for example, a LaTeX source file to publish query results in a nested table, HTML or Java source files to present the result on WWW browsers, and other media including MS-Excel worksheet, Tcl/Tk, O2C, etc. O2C is a data manipulation language of O2 and thus useful to migrate data in a relational database to an object oriented database.

The concept of parameterized view definition mechanism for relational database systems is presented. Primarily, it makes a single view definition serving for several different view instances according to the supplied actual parameters. Dynamic binding of parameters for the view allows the definition of recursive relations, such as transitive closure, without using extended operators or procedural constructs such as loops. In this paper, upward compatible syntax for SQL that incorporates view parameterization is proposed.

An encoding scheme (FLSOPC) is presented as a new data compression method. The generated fixed-length codes are preserving the order on the original data representations in the sense of semiorder preservation as defined in this paper. The FLSOPC employing binary sectioning assignment algorithm requires the code size that is linear to logarithm of the data cardinality. It is about 2.8 times that required by FLMB (fixed-length minimum bit) encoding when no knowledge about data is given a priori. This factor can be reduced to 2.1 if a half of data has been available as the initial load and approaches 1 when even more data is known.