Assertions have long been used to validate the functionality of software systems. Researchers and practitioners have extended them for validation of non-functional requirements, such as performance. This paper presents the implementation and application of the performance assertions in mobile device software. When applying performance assertions for such systems, we have discovered and resolved a number of issues in assertion specification, matching, and evaluation that were unresolved in previous research. The paper describes a simple, but effective framework geared towards mobile devices that allows specification and validation of real world performance requirements.

Performance profiling consists of monitoring a software system during execution and then analyzing the obtained data. There are two ways to collect profiling data: event tracing through code instrumentation and statistical sampling. These two approaches have different advantages and drawbacks. This paper proposes a hybrid approach to data collection that combines the completeness of event tracing with the low cost of statistical sampling. We propose to maximize the weighted amount of information obtained during data collection, show that such maximization can be performed in linear time or is NP-hard depending on the data collected and the collection implementation. We propose an approximation algorithm for NP-hard case. Our paper also presents an application of the formal approach to an example use case.

Performance is an important quality attribute that needs to be planned and managed proactively. Abstract models of the system are not very useful if they do not produce reasonably accurate metrics. Detailed models are time consuming and expensive to build as well as to simulate. In order to strike a right balance, a framework is proposed in this paper that takes advantage of the flexibility of abstract modeling and intricacies of detailed modeling. Performance is modeled and verified per use case using a hierarchical queuing model of the system. Each component job is represented through characterization functions and service requests. Characterization functions may be parametric regression models derived from job measurements on system level model. A co-design framework is used to simulate and measure the performance of software components. The use case simulator analyzes the performance and verifies the use case requirements.

Mobile phones serve as platforms for a variety of mobile applications including text and picture messaging as well as personal information management, including data synchronization with remote servers and desktop computers. They also host a range of communication-centered applications most of which have real-time constraints. To improve the performance of the mobile phone software, we focused on the runtime software architecture - a partition of all software functions into concurrent units and a scheduling policy that delivers the best possible service to the user with available resources. The units of concurrency in most products are operating system tasks. Thus, partition of software into tasks and allocation of functionality to tasks in the form of objects or functions are the most important decisions in the design of the runtime architecture (Ran et al., 2003). In this paper, we focus on the scheduling policy and its parameters. Some problems encountered in our project can be solved by a different task design. However, task redesign often is not feasible in industrial setting since it takes a long time and has high costs.

Making sense of runtime architecture for mobile phone software.

No abstract available.

Test suites are designed to validate the operation of a system against requirements. One important aspect of a test suite design is to ensure that system operation logic is tested completely. This is a difficult task. Code coverage tools support test suite designers by providing the information about which parts of source code are covered during system execution. Unfortunately, code coverage tools produce only source code coverage information. For a test engineer it is often hard to understand what the noncovered parts of the source code do and how they relate to requirements. We propose a generic approach that provides design coverage of the executed software, simplifying the development of new test suites. We demonstrate our approach on common design abstractions such as statecharts and structure diagrams. We implement the design coverage using tracing and a trace analysis framework. Using design coverage, test suites could be created faster by focussing on untested design elements.

Another concept of Third Eye is the tracing state. Tracing state is a set of event types generated in that state, other event types are filtered out and not reported. The system is always in a specific tracing state. Tracing states correspond to specifications. A program specification describes a set of constraints on events. The event types used in a specification have to be monitored to validate a trace against this specification. All event types contained in a specification and monitored for this specification form a tracing state. Tracing states also control the overhead of tracing on the executing system.The Third Eye framework includes modules for event type definition, event generation and reporting, tracing state definition and management, trace logging, query and browsing interfaces. Modules of event type definition, event reporting facility and tracing state controler are integrated with the software of the system under trace (SUT). The rest of the modules are independent from the SUT and can be deployed on a different execution platform to minimize the influence on system performance. Trace delivery for logging and analysis uses alternative interfaces to accommodate devices with different data storage and connectivity capabilities. We have implemented Third Eye framework prototype currently used by the Third Eye project team in collaboration with product development teams in Nokia's business units. We used Third Eye to test a number of software systems: the memory subsystem of one of Nokia's handsets, Apache Web Server, and WAP (Wireless Application Protocol) client. WAP is an industrial standard for applications and services that operate over wireless communication networks. We validated message sequences in this protocol by adding events in the functions that correspond to the protocol primitives and then checking whether the event sequence corresponds to the protocol message sequence. Events are mapped to Prolog facts and constraints are expressed as Prolog rules. Third Eye can be used for debugging, monitoring, specification validation, and performance measurements. These scenarios use typed events—a concept simple and yet expressive enough to be shared by product designers and developers. The Third Eye has an open architecture allowing easy replacement of third-party tools, including databases, analysis and validation tools. Third Eye is a practical framework for specification-based analysis and adaptive execution tracing of software systems.

Object relationships in modem software systems are becoming increasingly numerous and complex. Programmers who try to find violations of such relationships need new tools that allow them to explore objects in a large system more efficiently. Many existing debuggers present only a low-level, one-object-at-a-time view of objects and their relationships. We propose a new solution to overcome these problems: query-based debugging. The implementation of the query-based debugger described here offers programmers an effective query tool that allows efficient searching of large object spaces and quick verification of complex relationships. Even for programs that have large numbers of objects, the debugger achieves interactive response times for common queries by using a combination of fast searching primitives, query optimization, and incremental result delivery.