We introduce VisualHandicaps, a novel benchmark environment for the systematic analysis of interactive text-based reinforcement learning (TBRL) agents by providing visual handicaps. Unlike previous TBRL environments, which focus on providing additional textual information to measure agent understanding of sequential natural language information, VisualHandicaps seeks to improve the generalization ability of RL agents using varying details of maps and textual information, allowing for the study and demonstration of robust planning and self-localization. We provide automatically generated variations and difficulty levels in our environment and show that an agent using our systematic visual handicaps along with textual observation generally outperforms previous methods (that use only textual handicaps) in terms of success rate and the number of steps required to reach the goal. We also provide a detailed analysis of each handicap, which we believe to be important findings for driving future improvements in RL agents on text-based applications.

Geospatial machine learning is of growing importance in various global remote-sensing applications, particularly in the realm of vegetation monitoring. However, acquiring accurate ground truth data for geospatial tasks remains a significant challenge, often entailing considerable time and effort. Foundation models, emphasizing pre-training on large-scale data and fine-tuning, show promise but face limitations when applied to geospatial data due to domain differences. Our paper introduces a novel image translation method, combining geospatial-specific pre-training with training and test-time data augmentation. In a case study involving the translation of normalized difference vegetation index (NDVI) values from synthetic aperture radar (SAR) images of cabbage farms, our approach outperformed competitors by 31% in a public competition. It also exceeded the average of the top five teams by 44%. We publish both our image translation method with baseline methods and the geospatial-specific dataset at https://github.com/IBM/SAR2NDVI.

High-resolution flood modeling is enabled by utilizing high-resolution input derived by remote sensing technologies such as Light Detection and Ranging (LiDAR) systems. However, there is a long-standing trade-off between the computational time and spatial resolution for a flood simulation. In this paper, we propose a novel deep learning-based geospatial encoder-decoder for flood modeling consisting of (i) accuracy-preserving coarse-graining of the input topography, (ii) simulating flood with the coarser model, and (iii) downscaling the simulated flood to super-resolution. Our experiments show that our approach accelerates flood simulation up to 50 times faster with 1/16 scale while MSE of 0.0179, which is 10.3% less than the baseline with bilinear interpolation. Especially, we observe 20.5% reduction of MSE on average for the 5% worst cases.

To improve the unsupervised training of geospatial foundation models, we propose a novel approach that prepares diverse and unbiased datasets by maximizing an information entropy of selected geospatial features. Our method involves the extraction of detailed metrics such as temperature and precipitation, which are then organized into clusters based on their similarities. The approach introduces a weighted sampling method that ensures the inclusion of representative data points, it gives preference to less frequent data by counting the number of similar geospatial data points to increase the diversity of the dataset. The result shows that the information entropy value of the proposed method is higher than that of the uniform random method. And the approach significantly improves the accuracy of the geospatial model by providing a balanced representation of the data. Our research highlights the potential benefits of optimizing geospatial data sampling, which can lead to improved model accuracy and expanded practical applications.

Foundation models are often pretrained on large datasets and have been valuable in improving efficiency of model training for language and visual processing tasks. An increasing amount of research has focused on foundation models for satellite data which would be beneficial for a wide range of applications such as climate impact modelling. As large quantities of unlabeled satellite data are collected daily, self-supervised learning methods such as image inpainting are key to pretraining these models. Reducing pretraining data required by improving learning efficiency could improve feasibility of implementation. This research proposes an augmentation based consistency loss to improve pretraining efficiency while enhancing downstream performance. Two variations of the proposed approach are evaluated by finetuning pretrained models on flood segmentation and multilabel land cover downstream tasks. Findings show that incorporating consistency loss can enhance downstream performance, although the degree of improvement depends on the downstream task. It is further demonstrated that the downstream improvements can be achieved even with reduced pretraining data.

Subhajit Chaudhury, Sarathkrishna Swaminathan, Daiki Kimura, Prithviraj Sen, Keerthiram Murugesan, Rosario Uceda-Sosa, Michiaki Tatsubori, Achille Fokoue, Pavan Kapanipathi, Asim Munawar, Alexander Gray. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

Flood risks have been increasing in recent years due to climate change. To provide flood risks through simulations near river areas, observational data of river discharges is required. However, the information of river discharges is sparse because of the limited availability of gauge stations. In this work, we use a combination of topographic features in location-scale with meteorological features in basin-scale to predict river discharges in ungauged locations along a river network. We utilize deep kernel learning as our core model and use non-stationary kernel formulation to enable spatiotemporal interpolation. We tested the performance in two different rivers using one year of collected data and compared it with several baselines. According to our experimental results, our proposed model improved Nash Efficiency Criterion (NSE) by 2.6%, normalized root mean-squared error (NRMSE) by 7%, and coefficient of determination (R2) by 275% compared with ordinary kriging as the baseline. The combination of features used in this experiment also improved NSE by 27.1%, NRMSE by 13.7%, and R2by 182.6%.

When providing the boundary conditions for hydrological flood models and estimating the associated risk, interpolating precipitation at very high temporal resolutions (e.g. 5 minutes) is essential not to miss the cause of flooding in local regions. In this paper, we study optical flow-based interpolation of globally available weather radar images from satellites. The proposed approach uses deep neural networks for the interpolation of multiple video frames, while terrain information is combined with temporarily coarse-grained precipitation radar observation as inputs for self-supervised training. An experiment with the Meteonet radar precipitation dataset for the flood risk simulation in Aude, a department in Southern France (2018), demonstrated the advantage of the proposed method over a linear interpolation baseline, with up to 20% error reduction.

Text-Based Games (TBGs) have emerged as important testbeds for reinforcement learning (RL) in the natural language domain.Previous methods using LSTM-based action policies are uninterpretable and often overfit the training games showing poor performance to unseen test games.We present SymboLic Action policy for Textual Environments (SLATE), that learns interpretable action policy rules from symbolic abstractions of textual observations for improved generalization.We outline a method for end-to-end differentiable symbolic rule learning and show that such symbolic policies outperform previous stateof-the-art methods in text-based RL for the coin collector environment from 5 -10x fewer training games.Additionally, our method provides human-understandable policy rules that can be readily verified for their logical consistency and can be easily debugged.1

Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.

Testing self-driving cars in different areas requires surrounding cars with accordingly different driving styles such as aggressive or conservative styles. Calibrating a driving model (DM) makes the simulated driving behavior closer to human-driving behavior, and enable the simulation of human-driving cars. Conventional DM-calibrating methods do not take into account that the parameters in a DM vary while driving. These "fixed" calibrating methods cannot reflect an actual interactive driving scenario. In this paper, we propose a DM-calibration method for measuring human driving styles to reproduce real car-following behavior more accurately. The method includes 1) an objective entropy weight method for measuring and clustering human driving styles, and 2) online adaption of DM parameters based on deep learning by combining Bayesian optimization and a gated recurrent unit neural network. We conducted experiments to evaluate the proposed method, and the results indicate that it can be easily used to measure human driver styles. The experiments also showed that we can calibrate a corresponding DM in a virtual testing environment with up to 26% more accuracy than with fixed calibration methods.

Subhajit Chaudhury, Daiki Kimura, Kartik Talamadupula, Michiaki Tatsubori, Asim Munawar, Ryuki Tachibana. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

This paper describes a framework called MaestROBe It is designed to make the robots perform complex tasks with high precision by simple high-level instructions given by natural language or demonstration. To realize this, it handles a hierarchical structure by using the knowledge stored in the forms of ontology and rules for bridging among different levels of instructions. Accordingly, the framework has multiple layers of processing components; perception and actuation control at the low level, symbolic planner and Watson APIs for cognitive capabilities and semantic understanding, and orchestration of these components by a new open source robot middleware called Project Intu at its core. We show how this framework can be used in a complex scenario where multiple actors (human, a communication robot, and an industrial robot) collaborate to perform a common industrial task. Human teaches an assembly task to Pepper (a humanoid robot from SoftBank Robotics) using natural language conversation and demonstration. Our framework helps Pepper perceive the human demonstration and generate a sequence of actions for UR5 (collaborative robot arm from Universal Robots), which ultimately performs the assembly (e.g. insertion) task.

Traffic congestion in developing cities like Nairobi, Kenya can be significantly impacted by the presence of Moving Traffic Obstacles (MTOs). These MTOs are events that temporarily exist on the road, moving with or against the direction of traffic at slower speeds. They include two-wheelers, pushcarts, animals, and pedestrians, which have quite different influence on traffic compared with static obstacles, such as potholes and speed bumps. As smartphones and supporting 3G infrastructures are wide spread even in developing countries, recent studies enabled frugal traffic obstacle data collection from smartphones in probe cars. Assuming the opportunistic, unevenly-distributed, sparse and errorful observation of traffic obstacles, we propose an MTO detection algorithm extending an image analysis technique called Probabilistic Hough Transform for collective observations as input. Based on our experiences with a small set of real-world data collected in a smartphone-based probe car project with Nairobi City County, we conducted experiments with simulated observation data to see the effectiveness of the algorithm.

Programmers who develop Web applications often use dynamic scripting languages such as Perl, PHP, Python, and Ruby. For general purpose scripting language usage, interpreter-based implementations are efficient and popular but the server-side usage for Web application development implies an opportunity to significantly enhance Web server throughput. This paper summarizes a study of the optimization of PHP script processing. We developed a PHP processor, P9, by adapting an existing production-quality just-in-time (JIT) compiler for a Java virtual machine, for which optimization technologies have been well-established, especially for server-side application. This paper describes and contrasts microbenchmarks and SPECweb2005 benchmark results for a well-tuned configuration of a traditional PHP interpreter and our JIT compiler-based implementation, P9. Experimental results with the microbenchmarks show 2.5-9.5x advantage with P9, and the SPECweb2005 measurements show 20-30 % improvements. These results show that the acceleration of dynamic scripting language processing does matter in a realistic Web application server environment. CPU usage profiling shows our simple JIT compiler introduction reduces the PHP core runtime overhead from 45 % to 13 % for a SPECweb2005 scenario, implying that further improvements of dynamic compilers would provide little additional return unless other major overheads such as heavy memory copy between the language runtime and Web server frontend are reduced.

PHP is a popular language for server-side applications. In PHP, assignment to variables copies the assigned values, according to its so-called copy-on-assignment semantics. In contrast, a typical PHP implementation uses a copy-on-write scheme to reduce the copy overhead by delaying copies as much as possible. This leads us to ask if the semantics and implementation of PHP coincide, and actually this is not the case in the presence of sharings within values. In this paper, we describe the copy-on-assignment semantics with three possible strategies to copy values containing sharings. The current PHP implementation has inconsistencies with these semantics, caused by its naïve use of copy-on-write. We fix this problem by the novel mostly copy-on-write scheme, making the copy-on-write implementations faithful to the semantics. We prove that our copy-on-write implementations are correct, using bisimulation with the copy-on-assignment semantics.

The performance of server-side applications is becoming increasingly important as more applications exploit the Web application model. Extensive work has been done to improve the performance of individual software components such as Web servers and programming language runtimes. This paper describes a novel approach to boost Web application performance by improving inter-process communication between a programming language runtime and Web server runtime. The approach reduces redundant processing for memory copying and the context switch overhead between user space and kernel space by exploiting the zero-copy data transfer methodology, such as the sendfile system call. In order to transparently utilize this optimization feature with existing Web applications, we propose enhancements of the PHP runtime, FastCGI protocol, and Web server. Our proposed approach achieves a 126% performance improvement with micro-benchmarks and a 44% performance improvement for a standard Web benchmark, SPECweb2005.

Web applications often use HTML templates to separate the webpage presentation from its underlying business logic and objects. This is now the de facto standard programming model for Web application development. This paper proposes a novel implementation for existing server-side template engines, FlyingTemplate, for (a) reduced bandwidth consumption in Web application servers, and (b) off-loading HTML generation tasks to Web clients. Instead of producing a fully-generated HTML page, the proposed template engine produces a skeletal script which includes only the dynamic values of the template parameters and the bootstrap code that runs on a Web browser at the client side. It retrieves a client-side template engine and the payload templates separately. With the goals of efficiency, implementation transparency, security, and standards compliance in mind, we developed FlyingTemplate with two design principles: effective browser cache usage, and reasonable compromises which restrict the template usage patterns and relax the security policies slightly but in a controllable way. This approach allows typical template-based Web applications to run effectively with FlyingTemplate. As an experiment, we tested the SPECweb2005 banking application using FlyingTemplate without any other modifications and saw throughput improvements from 1.6x to 2.0x in its best mode. In addition, FlyingTemplate can enforce compliance with a simple security policy, thus addressing the security problems of client-server partitioning in the Web environment.

PHP is well known as a programming language in the Web 2.0 era enabling agile server-side software development. It has officially supported SOAP messaging since version 5 through a C-based built-in library. In this paper we perform a thorough study of the capability of PHP as a Web service engine in both qualitative and quantitative aspects while comparing it with other Web service engines implemented in Java and C. We used Axis2 for this purpose as it is an open source web service engine whose implementation is available both in Java and C. We report that PHP as a web service engine performs competitively with Axis2 Java for Web services involving small payloads, and greatly outperforms it for larger payloads by 5-17 times. As the authors expected, Axis2 C performs best, but the experimental results demonstrate that PHP performance is closer to Axis2 C with larger payloads. This performance difference comes from the fact that the SOAP engine within the PHP runtime is implemented in C with a monolithic architecture, whereas Axis2 uses a more modular architecture for the flexible insertation of handlers for an assorted set of WS-* standards, and also that Axis2 uses a different data binding mechanism known as ADB (Axis2 Data binding). This paper is the first attempt to compare Web services engines implemented in PHP, Java and C, and the authors believe that this boosts the development of SOAP-based Web services in PHP by letting people know its decent performance score and high productivity characteristics.

There are large demands for re-engineering human-oriented Web application systems for use as machine-oriented Web application systems, which are called Web services. This paper describes a framework named H2W, which can be used for constructing Web service wrappers from existing, multi-paged Web applications. H2Ws contribution is mainly for service extraction, rather than for the widely studied problem of data extraction. For the framework, we propose a page-transition-based decomposition model and a page access abstraction model with context propagation. With the proposed decomposition and abstraction, developers can flexibly compose a Web service wrapper of their intent by describing a simple workflow program incorporating the advantages of previous work on Web data extraction. We show three successful wrapper application examples with H2W for real world Web applications

An enterprise service-oriented architecture is typically realized on a messaging infrastructure called an enterprise service bus (ESB). An ESB is a bus which delivers messages from service requesters to service providers. Since it sits between the service requesters and providers, it is not appropriate to use any existing capacity planning methodology for servers, such as modeling to estimate an ESB's capacity. There are programs which run on an ESB called mediation modules. Their functionalities vary and depend on how people use the ESB. This creates difficulties for capacity planning and performance evaluation. This paper proposes a performance evaluation methodology and techniques for ESBs. We actually run the ESB on a real machine while providing a pseudo-environment around it. In order to ease setting up the environment we provide ultra-light service requestors and service providers for the ESB under test. We show that the proposed mock environment can be set up with practical hardware resources available at the time of hardware resource assessment. Our experimental results showed that the testing results with our mock environment are equivalent to the results in the real environment

The poor performance of WS-security (WSS) processing is often a topic of concern and prevents its wider adoption. We focused on byte-level similarities in WSS messages and implemented a template-based WSS processor. Inside the processor an automaton is employed that matches the incoming messages and extracts signature values and/or encrypted values. WSS operations including XML canonicalization are performed against the extracted values, without costly XML parsing and traversal. This is more than twice as fast as the DOM-based WSS processor and our prior work with a stream-based processor.

Web services technology has emerged as a key infrastructure that enables business entities to interact with each other without any human inventions. In order for the technology to be widely used, especially in any field where a large volume of transactions may be processed, it is highly desirable that the Web services engine should tolerate such environments. In this paper, we present a novel approach for improving Web services performance. We first focus on the fundamental characteristics of the Web services in that the SOAP messages on the wire are mostly generated by machines and have a lot of similarities among the processed messages. By making use of these features and eliminating the redundant processing, we propose a new deserialization mechanism that reuses matching regions from the previously deserialized application objects from earlier messages, and only performs deserialization for a new region that would not be processed before. Through our experiments in this paper, we observed that our approach obtained a 288% performance gain (maximum) by incorporating the differential deserialization into the Axis SOAP engine.

XML (Extensible Markup Language) processing can incur significant runtime overhead in XML-based infrastructural middleware such as Web service application servers. This paper proposes a novel mechanism for efficiently processing similar XML documents. Given a new XML document as a byte sequence, the XML parser proposed in this paper normally avoids syntactic analysis but simply matches the document with previously processed ones, reusing those results. Our parser is adaptive since it partially parses and then remembers XML document fragments that it has not met before. Moreover, it processes safely since its partial parsing correctly checks the well-formedness of documents. Our implementation of the proposed parser complies with the JSR 63 standard of the Java API for XML Processing (JAXP) 1.1 specification. We evaluated Deltarser performance with messages using Google Web services. Comparing to Piccolo (and Apache Xerces), it effectively parses 35 % (106%) faster in a server-side use-case scenario, and 73 % (126%) faster in a client-side use-case scenario.

We discuss the design for an efficient response cache mechanism appropriate for the Web services architecture. The important feature of Web services is its interoperability between heterogeneous platforms. This interoperability is based on widely accepted standards such as XML, SOAP, and WSDL. We describe a response cache mechanism for Web services client middleware without any extensions to these standards so that the client can participate transparently in the existing Web services community. We propose three optimization methods to improve the performance of our response cache. The first optimization is caching the post-parsing representation instead of the XML message itself. The second is caching application objects. For this optimization, we show some copying processes that are dependent on the type of cached objects. The third optimization is for read-only objects. These methods reduce the overhead of XML processing or object copying. We have implemented a prototype of a response cache on Apache-Axis, and evaluated these optimization methods through experiments for Google Web services. Finally, based on the experimental results, we discuss the optimal configuration of these methods based on data types.

This paper presents an emerging tool for security configuration of service-oriented architectures with Web Services. Security is a major concern when implementing mission-critical business transactions and such concern motivated the development of Web Services Security (WS-Security). However, the existing tools for configuring the security properties of Web Services give a technology-oriented view, and only assist in choosing the data to encrypt and selecting an encryption algorithm. The users must construct their own mental models of how the security configurations actually relate to business policies. In contrast, the tool described here gives a simplified, business-policy-oriented view. It models the messaging with customers and business partners, lists various threats, and presents best-practice security patterns against the threats. A user can select among variations on the basic patterns according to the business policies, and then apply them to the messaging model through the GUI. The result of the pattern application is described in the Web Services Security Policy Language (WS-Security Policy).