Abstract: Research in learning analytics (LA) has long held a strong interest in improving student self-regulated learning and measuring the impact of feedback on student outcomes. Despite more than a decade of work in this space very little is known around the contextual factors that influence the topics and diversity of feedback and assessment a student encounters during their full program of study. This paper presents research investigating the institutional adoption of a personalized feedback tool. The reported findings illustrate an association between the topics of feedback, student performance, year level of the course and discipline. The results highlight the need for LA research to capture feedback, assessment and learning outcomes over an entire program of study. Herein we propose a more integrated model drawing on contemporary understandings of feedback with current research findings. The goal is to push LA towards addressing more complex teaching and learning processes from a systems lens. The model posed in this paper begins to illustrate where and how LA can address noted deficits in education practice to better understand how feedback and assessment are enacted by instructors and interpreted by students.

Learning analytics approaches to profiling students based on their study behaviour remain limited in how they integrate temporality and change. To advance this area of work, the current study examines profiles of change in student study behaviour in a blended undergraduate engineering course. The study is conceptualised through complex dynamical systems theory and its applications in psychological and cognitive science research. Students were profiled based on the changes in their behaviour as observed in clickstream data. Measure of entropy in the recurrence of student behaviour was used to indicate the change of a student state, consistent with the evidence from cognitive sciences. Student trajectories of weekly entropy values were clustered to identify distinct profiles. Three patterns were identified: stable weekly study, steep changes in weekly study, and moderate changes in weekly study. The students with steep changes in their weekly study activity had lower exam grades and showed destabilisation of weekly behaviour earlier in the course. The study investigated the relationships between these profiles of change, student performance, and other approaches to learner profiling, such as self-reported measures of self-regulated learning, and profiles based on the sequences of learning actions.

Industry supervisors play a pivotal role in ongoing learner support and guidance within a work-integrated learning context. Effective provisional feedback from industry supervisors in work-integrated learning environments is essential for increasing a team’s metacognitive awareness and ability to evaluate their performance. However, research that examines the usefulness and type of feedback from industry supervisors for teams remains limited. In this study, we investigate the quality of provisional feedback by comparing the teams’ helpfulness rating of the feedback from two types of industry supervisors (i.e., clients and mentors), based on the feedback type (task, process, regulatory and self-level oriented) using learning analytics. The results show that teams rated the perceived helpfulness scores of clients and mentors as very useful, with mentors providing slightly more helpful feedback. We also found that mentors provide more co-occurrences of feedback classifications than clients. The overall results show that teams perceive mentor feedback as more helpful than clients and that the mentor targets feedback that is more beneficial to the teams learning than the clients. Our findings can aid in developing guidelines that aim to validate and improve existing or new feedback quality frameworks by leveraging backward evaluation data.

The conceptualisation of self-regulated learning (SRL) as a process that unfolds over time has influenced the way in which researchers approach analysis. This gave rise to the use of process mining in contemporary SRL research to analyse data about temporal and sequential relations of processes that occur in SRL. However, little attention has been paid to the choice and combinations of process mining algorithms to achieve the nuanced needs of SRL research. We present a study that 1) analysed four process mining algorithms that are most commonly used in the SRL literature – Inductive Miner, Heuristics Miner, Fuzzy Miner, and pMineR; and 2) examined how the metrics produced by the four algorithms complement each. The study looked at micro-level processes that were extracted from trace data collected in an undergraduate course (N=726). The study found that Fuzzy Miner and pMineR offered better insights into SRL than the other two algorithms. The study also found that a combination of metrics produced by several algorithms improved interpretation of temporal and sequential relations between SRL processes. Thus, it is recommended that future studies of SRL combine the use of process mining algorithms and work on new tools and algorithms specifically created for SRL research.

Students enrol into STEM programs with varying degrees of confidence with citing and referencing texts in their written work. Students often have an inclination to choose numbers over written language throughout schooling which means less opportunity to practice referencing and citation. This is compounded by large numbers of students for whom English is an additional language or who articulate from different cultural ways-of-doing. The Search, Evaluate, Integrate, Reference and Act Ethically (SEIRA) modules were developed to provide discipline-relevance to a confounding task. Data Analysis looking at the student engagement with the SEIRA site and subsequent student success provides an indication of the value of this approach to developing academic literacy across the STEM disciplines.

Feedback is a major factor of student success within higher education learning. However, recent changes - such as increased class sizes and socio-economic diversity of the student population - challenged the provision of effective student feedback. Although the use of educational technology for personalised feedback to diverse students has gained traction, the feedback gap still exists: educators wonder which students respond to feedback and which do not. In this study, a set of trackable Call to Action (CTA) links was embedded in two sets of feedback messages focusing on students' time management, with the goal of (1) examining the association between feedback engagement and course success and (2), to predict students' reaction to provided feedback. We also conducted two focus groups to further examine students' perception of provided feedback messages. Our results revealed that early engagement with the feedback was associated with higher chances of succeeding in the course. Likewise, previous engagement with feedback was highly predictive of students' engagement in the future, and also that certain student sub-populations, (e.g., female students), were more likely to engage than others. Such insight enables instructors to ask "why" questions, improve feedback processes and narrow the feedback gap. Practical implications of our findings are further discussed.

The temporal and sequential nature of learning is receiving increasing focus in Learning Analytics circles. The desire to embed studies in recognised theories of self-regulated learning (SRL) has led researchers to conceptualise learning as a process that unfolds and changes over time. To that end, a body of research knowledge is growing which states that traditional frequency-based correlational studies are limited in narrative impact. To further explore this, we analysed trace data collected from online activities of a sample of 239 computer engineering undergraduate students enrolled on a course that followed a flipped class-room pedagogy. We employed SRL categorisation of micro-level processes based on a recognised model of learning, and then analysed the data using: 1) simple frequency measures; 2) epistemic network analysis; 3) temporal process mining; and 4) stochastic process mining. We found that a combination of analyses provided us with a richer insight into SRL behaviours than any one single method. We found that better performing learners employed more optimal behaviours in their navigation through the course's learning management system.

The use of learning trace data together with various analytical methods has proven successful in detecting patterns in learning behaviour, identifying student profiles, and clustering learning resources. However, interpretation of the findings is often difficult and uncertain due to a lack of contextual data (e.g., data on student motivation, emotion or curriculum design). In this study we explored the integration of student self-reports about cognitive load and self-efficacy into the learning process and collection of relevant students' perceptions as learning traces. Our objective was to examine the association of traced measures of relevant learning constructs (cognitive load and self-efficacy) with i) indicators of the students' learning behaviour derived from trace data, and ii) the students' academic performance. The results indicated the presence of association between some indicators of students' engagement with learning activities and traced measures of cognitive load and self-efficacy. Correlational analysis demonstrated significant positive correlation between the students' course performance and traced measures of cognitive load and self-efficacy.

Learning analytics has the potential to detect and explain characteristics of learning strategies through analysis of trace data and communicate the findings via feedback. However, the role of learning analytics-based feedback in selection and regulation of learning strategies is still insufficiently explored and understood. This research aims to examine the sequential and temporal characteristics of learning strategies and investigate their association with feedback. Three years of trace data were collected from online pre-class activities of a flipped classroom, where different types of feedback were employed in each year. Clustering, sequence mining, and process mining were used to detect and interpret learning tactics and strategies. Inferential statistics were used to examine the association of feedback with the learning performance and the detected learning strategies. The results suggest a positive association between the personalised feedback and the effective strategies.

This paper aims to identify self-regulation strategies from students' interactions with the learning management system (LMS). We used learning analytics techniques to identify metacognitive and cognitive strategies in the data. We define three research questions that guide our studies analyzing i) self-assessments of motivation and self regulation strategies using standard methods to draw a baseline, ii) interactions with the LMS to find traces of self regulation in observable indicators, and iii) self regulation behaviours over the course duration. The results show that the observable indicators can better explain self-regulatory behaviour and its influence in performance than preliminary subjective assessments.

Despite strong interest in learning analytics (LA), adoption at a large-scale organizational level continues to be problematic. This may in part be due to the lack of acknowledgement of existing conceptual LA models to operationalize how key adoption dimensions interact to inform the realities of the implementation process. This paper proposes the framing of LA adoption in complexity leadership theory (CLT) to study the overarching system dynamics. The framing is empirically validated in a study analysing interviews with senior staff in Australian universities (n=32). The results were coded for several adoption dimensions including leadership, governance, staff development, and culture. The coded data were then analysed with latent class analysis. The results identified two classes of universities that either i) followed an instrumental approach to adoption - typically top-down leadership, large scale project with high technology focus yet demonstrating limited staff uptake; or ii) were characterized as emergent innovators - bottom up, strong consultation process, but with subsequent challenges in communicating and scaling up innovations. The results suggest there is a need to broaden the focus of research in LA adoption models to move on from small-scale course/program levels to a more holistic and complex organizational level.

Learning analytics research has often been touted as a means to address concerns regarding student retention outcomes. However, few research studies to date, have examined the impact of the implemented intervention strategies designed to address such retention challenges. Moreover, the methodological rigor of some of the existing studies has been challenged. This study evaluates the impact of a pilot retention program. The study contrasts the findings obtained by the use of different methods for analysis of the effect of the intervention. The pilot study was undertaken between 2012 and 2014 resulting in a combined enrolment of 11,160 students. A model to predict attrition was developed, drawing on data from student information system, learning management system interactions, and assessment. The predictive model identified some 1868 students as academically at-risk. Early interventions were implemented involving learning and remediation support. Common statistical methods demonstrated a positive association between the intervention and student retention. However, the effect size was low. The use of more advanced statistical methods, specifically mixed-effect methods explained higher variability in the data (over 99%), yet found the intervention had no effect on the retention outcomes. The study demonstrates that more data about individual differences is required to not only explain retention but to also develop more effective intervention approaches.

Student's learning happens where the learner is, rather than being constrained to a single physical or digital environment. It is of high relevance for the LAK community to provide analytics support in blended learning scenarios where students can interact at diverse learning spaces and with a variety of educational tools. This workshop aims to gather the sub-community of LAK researchers, learning scientists and researchers in other areas, interested in the intersection between ubiquitous, mobile and/or classroom learning analytics. The underlying concern is how to integrate and coordinate learning analytics seeking to understand the particular pedagogical needs and context constraints to provide learning analytics support across digital and physical spaces. The goals of the workshop are to consolidate the Cross-LAK sub-community and provide a forum for idea generation that can build up further collaborations. The workshop will also serve to disseminate current work in the area by both producing proceedings of research papers and working towards a journal special issue.

The amount of data extracted from learning experiences has grown at an astonishing pace both in depth due to the increasing variety of data sources, and in breath with courses now being offered to massive student cohorts. However, in this emerging scenario instructors are now facing the challenge of connecting the knowledge emerging from data analysis with the provision of meaningful support actions to students within the context of an instructional design.

An ongoing challenge for Learning Analytics research has been the scalable derivation of user interaction data from multiple technologies. The complexities associated with this challenge are increasing as educators embrace an ever growing number of social and content-related technologies. The Experience API (xAPI) alongside the development of user specific record stores has been touted as a means to address this challenge, but a number of subtle considerations must be made when using xAPI in Learning Analytics. This paper provides a general overview to the complexities and challenges of using xAPI in a general systemic analytics solution - called the Connected Learning Analytics (CLA) toolkit. The importance of design is emphasised, as is the notion of common vocabularies and xAPI Recipes. Early decisions about vocabularies and structural relationships between statements can serve to either facilitate or handicap later analytics solutions. The CLA toolkit case study provides us with a way of examining both the strengths and the weaknesses of the current xAPI specification, and we conclude with a proposal for how xAPI might be improved by using JSON-LD to formalise Recipes in a machine readable form.

The International Workshop of Smart Environments and Analytics on Video-Based Learning ([email protected]) aims to connect research efforts on Video-Based Learning with Smart Environments and Analytics to create synergies between these fields. The main objective is to build a research community around the intersection of these topical areas. In particular, [email protected] aims to develop a critical discussion about the next generation of video-based learning environments and their analytics, the form of these analytics and the way they can be analyzed in order to help us to better understand and improve the value of educational videos to support teaching and learning. [email protected] is based on the rationale that combining and analyzing learners' interactions with other available data obtained from learners, new avenues for research on video-based learning have emerged. This can have a significant impact in current educational trends such as Massive Open Online Courses (MOOCs) and Flipped Classroom.

The majority of applications and products that use learning analytics to understand and improve learning experiences assume the creation of actionable items that will affect students through an intermediary. Much less focus is devoted to exploring how to provide insight directly to students. Furthermore, student engagement has always been a relevant aspect to increase the quality of a learning experience. Learning analytics techniques can be used to provide real-time insight tightly integrated with the learning outcomes directly to the students. This paper describes a case study deployed in a first year engineering course using a flipped learning strategy to explore the behavior of students interacting with a dashboard updated in real time providing indicators of their engagement with the course activities. The results show different patterns of use and their evolution throughout the experience and shed some light on how students perceived this resource.

This demonstration introduces the Connected Learning Analytics (CLA) Toolkit. The CLA toolkit harvests data about student participation in specified learning activities across standard social media environments, and presents information about the nature and quality of the learning interactions.

It is of high relevance to the LAK community to explore blended learning scenarios where students can interact at diverse digital and physical learning spaces. This workshop aims to gather the sub-community of LAK researchers, learning scientists and researchers from other communities, interested in ubiquitous, mobile and/or face-to-face learning analytics. An overarching concern is how to integrate and coordinate learning analytics to provide continued support to learning across digital and physical spaces. The goals of the workshop are to share approaches and identify a set of guidelines to design and connect Learning Analytics solutions according to the pedagogical needs and contextual constraints to provide support across digital and physical learning spaces.

The areas of educational data mining and learning analytics focus on the extraction of knowledge and actionable items from data sets containing detailed information about students. However, the potential impact from these techniques is increased when properly contextualized within a learning environment. More studies are needed to explore the connection between student interactions, approaches to learning, and academic performance. Self-regulated learning (SRL) is defined as the extent to which a student is able to motivationally, metacognitively, and cognitively engage in a learning experience. SRL has been the focus of research in traditional classroom learning and is also argued to play a vital role in the online or blended learning contexts. In this paper, we study how SRL affects students' online interactions with various learning activities and its influence in academic performance. The results derived from a naturalistic experiment among a cohort of first year engineering students showed that positive self-regulated strategies (PSRS) and negative self-regulated strategies (NSRS) affected both the interaction with online activities and academic performance. NSRS directly predicted academic outcomes, whereas PSRS only contributed indirectly to academic performance via the interactions with online activities. These results point to concrete avenues to promote self-regulation among students in this type of learning contexts.

The pervasive collection of data has opened the possibility for educational institutions to use analytics methods to improve the quality of the student experience. However, the adoption of these methods faces multiple challenges particularly at the course level where instructors and students would derive the most benefit from the use of analytics and predictive models. The challenge lies in the knowledge gap between how the data is captured, processed and used to derive models of student behavior, and the subsequent interpretation and the decision to deploy pedagogical actions and interventions by instructors. Simply put, the provision of learning analytics alone has not necessarily led to changing teaching practices. In order to support pedagogical change and aid interpretation, this paper proposes a model that can enable instructors to readily identify subpopulations of students to provide specific support actions. The approach was applied to a first year course with a large number of students. The resulting model classifies students according to their predicted exam scores, based on indicators directly derived from the learning design.

In this paper, we briefly describe the goal and activities of the LAK15 workshop on Visual Aspects of Learning analytics.

Designing, deploying and validating learning analytics tools for instructors or students is a challenge requiring techniques and methods from different disciplines, such as software engineering, human-computer interaction, educational design and psychology. Whilst each of these disciplines has consolidated design methodologies, there is a need for more specific methodological frameworks within the cross-disciplinary space defined by learning analytics. In particular there is no systematic workflow for producing learning analytics tools that are both technologically feasible and truly underpin the learning experience. In this paper, we present the LATUX workflow, a five-stage workflow to design, deploy and validate awareness tools in technology-enabled learning environments. LATUX is grounded on a well-established design process for creating, testing and re-designing user interfaces. We extend this process by integrating the pedagogical requirements to generate visual analytics to inform instructors' pedagogical decisions or intervention strategies. The workflow is illustrated with a case study in which collaborative activities were deployed in a real classroom.

A main goal for learning analytics is to inform the design of a learning experience to improve its quality. The increasing presence of solutions based on big data has even questioned the validity of current scientific methods. Is this going to happen in the area of learning analytics? In this paper we postulate that if changes are driven solely by a digital footprint, there is a risk of focusing only on factors that are directly connected to numeric methods. However, if the changes are complemented with an understanding about how students approach their learning, the quality of the evidence used in the redesign is significantly increased. This reasoning is illustrated with a case study in which an initial set of activities for a first year engineering course were shaped based only on the student's digital footprint. These activities were significantly modified after collecting qualitative data about the students approach to learning. We conclude the paper arguing that the interpretation of the meaning of learning analytics is improved when combined with qualitative data which reveals how and why students engaged with the learning tasks in qualitatively different ways, which together provide a more informed basis for designing learning activities.

The higher education sector has seen a shift in teaching approaches over the past decade with an increase in the use of video for delivering lecture content as part of a flipped classroom or blended learning model. Advances in video technologies have provided opportunities for students to now annotate videos as a strategy to support their achievement of the intended learning outcomes. However, there are few studies exploring the relationship between video annotations, student approaches to learning, and academic performance. This study seeks to narrow this gap by investigating the impact of students' use of video annotation software coupled with their approaches to learning and academic performance in the context of a flipped learning environment. Preliminary findings reveal a significant positive relationship between annotating videos and exam results. However, negative effects of surface approaches to learning, cognitive strategy use and test anxiety on midterm grades were also noted. This indicates a need to better promote and scaffold higher order cognitive strategies and deeper learning with the use of video annotation software.

Feedback is important for any activity, and learning is no exception. Whereas assessment can give summative feedback about the proficiency of the learning, learning analytics can give a much finer level of feedback about the learning process. Learning analytics can help in identifying the effectiveness of learning elements, can help in engaging students, can guide teachers in the preparation and deployment of the teaching activity. In this paper, we present a number of different initiatives carried out at UC3M that include elements of learning analytics for different purposes.

Learning analytic techniques are allowing the observation of complex learning activities that were hidden until now. Writing is a task in which behavioral patterns can be observed to measure the level of engagement. Previous studies relied mostly on data collected by observers. In this paper Tracer, a novel learning analytic system to visualize behavioral patterns of students while writing and measuring engagement is described. The tool combines and analyzes the information obtained from document revisions and Website logs while students work in a writing assignment and provides visualizations and measurements for the level of engagement. A user study was conducted in a software engineering course where students wrote and submitted a project proposal using Google Docs. Tracer generated a graphical view of the gauged engagement, and an engagement time for each student. The obtained results show that the engagement time gauged by Tracer was moderately correlated to those reported by the students.

In this paper, an overview is given about the research carried out in the area of mobile teaching and learning by Universidad Carlos III de Madrid, a member of the eMadrid Excellence Network. Mobile learning is raising growing expectations and is considered by some authors the next disruptive revolution in education. Cognitive and pedagogical theories supporting this prospect are reviewed. How these theories can be translated into meaningful educational practices is analysed by exploring ways of usage of mobile devices for supporting learning and teaching. Finally, a portfolio of experiments and case studies carried out by the Gradient group of the Universidad Carlos III de Madrid testing the application and effects of mobile learning are reported. These experiments show that the use of these devices is changing educational practices in a fundamental way.

The collection of learner events within a server-client architecture occurs either at server, client or both complementarily. Such collection may be incomplete due to various factors, particularly for client-based monitoring, where learners can disable, delete or even modify their event logs due to privacy policies. The quality and accuracy of any analysis based on such data collections depends critically on the quality of the subjacent dataset. We propose three initial metrics to evaluate the completeness of a learning dataset: client-to-server ratio, event-to-activity ratio and subjective ratio. These metrics provide a glimpse on the coverage rate of the monitoring and can be applied to distinguish subsets of data with a minimum level of reliability to be used in a learning analytics study.

The main objective of the 1st International Workshop on Learning Analytics and Linked Data (#LALD2012) is to connect the research efforts on Linked Data and Learning Analytics in order to create visionary ideas and foster synergies between the two young research fields. Therefore, the workshop will collect, explore, and present datasets, technologies and applications for Technology Enhanced Learning (TEL) to discuss Learning Analytics approaches that make use of educational data or Linked Data sources. During the workshop, an overview of available educational datasets and related initiatives will be given. The participants will have the opportunity to present their own research with respect to educational datasets, technologies and applications and discuss major challenges to collect, reuse, and share these datasets.

The use of technology in every day tasks enables the possibility to collect large amounts of observations of events taking place in different environments. Most tools are capable of storing a detailed account of the operations executed by users in certain files commonly known as logs. These files can be further analyzed to infer information that is not directly visible such as the most popular applications, times of the day with highest activity, calories burnt after a running session, etc. Graphic visualizations of this data can be used to support this type of analysis as shown in [1]. Visualization can also be applied in the domain of learning experiences to track and analyse the data obtained from both learners and instructors. There are several tools that have been proposed in specific environments such as, for example, in personal learning environments [5], to foster self-reflection and awareness [2], and to support instructors in web-based distance learning [3]. These visualizations need to take into account aspects such as how to access and protect personal data, filter management, multi-user support and availability. In this paper, the web-based visualization platform GLASS (Gradient's Learning Analytics System) is presented. The architecture of the tool has been conceived to support a large number of modular visualizations derived from a common dataset containing a large number of recorded events. The tool was developed following a bottom-up methodology to provide a set of basic operations required by any visualization. The design goal is to provide a highly versatile, modular platform that simplifies the implementation of new visualizations.

IMS Learning Design is considered by many authors the "de facto" standard in educational modeling languages. The versatility of the framework enables its use in very different situations. However, such versatile framework is usually hidden by its complex management. One handicap identified in practical experiences is the lack of flexibility of scripted courses during the enactment phase. The activity sequence and learning resources are rigidly defined during authoring. This fact makes difficult to react to unexpected events that may happen in live courses. Also, this rigidness does not allow instructors to give "their personal touch" to courses. This paper presents the improvements made on GRAIL - an IMS LD compliant player-aimed at the support of a flexible enactment phase. Two types of modifications are considered: the modification of the learning flow and the management of course content with a wiki engine. Finally, this paper discusses how the integration of third party services in the activity sequence relaxes the rigidness of scripted learning flows. Experiences deployed in real scenarios allowed analyzing how such integration offered flexibility in practical situations.

Most of the current learning analytic techniques have as starting point the data recorded by Learning Management Systems (LMS) about the interactions of the students with the platform and among themselves. But there is a tendency on students to rely less on the functionality offered by the LMS and use more applications that are freely available on the net. This situation is magnified in studies in which students need to interact with a set of tools that are easily installed on their personal computers. This paper shows an approach using Virtual Machines by which a set of events occurring outside of the LMS are recorded and sent to a central server in a scalable and unobtrusive manner.

Abstract Computer‐supported collaborative learning is a paradigm that uses technology to support collaborative methods of instruction. When combining collaborative learning with the need to exchange documents between students and the teaching staff in a blended learning scenario, version control systems (VCSs) greatly simplify this collaboration. Furthermore, these tools need to be adopted in regular classes as they are used in industrial environments. But deploying a collaborative environment in which version control is used does not scale for large classes. This paper presents SubCollaboration, a platform that uses the VCS Subversion to manage a large number of work spaces in a collaborative learning environment. The tool maintains a reference workspace where teaching staff introduces new material that is then synchronized with the team repositories. Two case studies are presented showing that students easily learn the use of version control and its deployment in large classes is feasible. Copyright © 2010 John Wiley & Sons, Ltd.

Flexibility is an intrinsic property present in any learning environment. When capturing the structure of a learning experience with conventional languages such as learning design, this flexibility can be seriously compromised. In this document two mechanisms are proposed to avoid this pitfall. On one hand, flexibility needs to be considered from the early stages of the design phase. Editors need to include default constructions to easily accommodate changes. On the other hand, run-time environments need to offer the possibility of changing most of the execution parameters with an intuitive interface available to the teaching staff.

Collaborative learning models are widely used in educational institutions. These models require a high interaction level among students and are mainly oriented towards in-class scenarios. But when collaborative models are deployed in a distant scenario, user expressiveness is significantly reduced thus creating a gap that hinders the effectiveness of this collaboration. A computer-supported model provides a set of tools to compensate for the distant scenario and reduce this gap. This paper presents the issues and solutions derived from the design and deployment of a complex collaborative model in a distant scenario. The course structure was captured using the Learning Design specification, and an architecture based on virtual network computing was used to provide the required collaborative tools. The course was included as part of a regular undergraduate program in three higher educational institutions.

Automatic assessment has become an important technique to reduce the grading load on teaching staff while providing an exhaustive evaluation environment for students. Several systems have evolved over the years providing sophisticated evaluation capabilities. However, fully automated assessment covers only a portion of the overall evaluation requirements in a typical Computer Science course. In this paper we present a tool for automatic assignment management that aims at satisfying several objectives. First, to support the large variety of assignment types and grading policies under the same paradigm by means of a phgeneric architecture. Second, to provide a phmulti-agent, scalable platform to cope with large enrollment classes. And third, to provide phfull connectivity with other administrative tools already present in educational institutions.

An automatic abstraction/refinement algorithm for symbolic CTL model checking is presented. Conservative model checking is thus done for the full CTL language-no restriction is made to the universal or existen tial fragments. The algorithm begins with conserv ativ everification of an initial abstraction. If the conclusion is negativ e,it deriv es a “goal set” of states which require further resolution. It then successiv ely refines, with respect to this goal set, the appro ximations made in the sub-formulas, until the giv en form ula is v erified or computational resources are exhausted. This method applies uniformly to the abstractions based in over-appro ximation as well as under-approximations of the model. Both the refinement and the abstraction procedures are based in BDD-subsetting. Note that refinement procedures which are based on error traces, are limited to over-appro ximation on the universal fragment (or for language con tainment), whereas the goal set method is applicable to all consisten t appro ximations, and for all CTL formulas.

Regarding finite state machines as Markov chains facilitates the application of probabilistic methods to very large logic synthesis and formal verification problems. In this paper we present symbolic algorithms to compute the steady-state probabilities for very large finite state machines (up to 10/sup 27/ states). These algorithms, based on Algebraic Decision Diagrams (ADD's)-an extension of BDD's that allows arbitrary values to be associated with the terminal nodes of the diagrams-determine the steady-state probabilities by regarding finite state machines as homogeneous, discrete-parameter Markov chains with finite state spaces, and by solving the corresponding Chapman-Kolmogorov equations. We first consider finite state machines with state graphs composed of a single terminal strongly connected component; for this type of system we have implemented two solution techniques: One is based on the Gauss-Jacobi iteration, the other one is based on simple matrix multiplication. Then we extend our treatment to the most general case of systems which can be modelled as finite state machines with arbitrary transition structures; here our approach exploits structural information to decompose and simplify the state graph of the machine. We report experimental results obtained for problems on which traditional methods fail.

This paper studies the problem of estimating worst case power dissipation in a sequential circuit.We approach this problem by nding the maximum average weight cycles in a weighted directed g r aph.In order to handle practical sized examples, we use symbolic methods, based o n A lgebraic Decision Diagrams (ADDs), for computing the maximum average length cycles as well as the number of gate transitions in the circuit, which is necessary to construct the weighted directed g r aph.

Regarding finite state machines as Markov chains facilitates the application of probabilistic methods to very large logic synthesis and formal verification problems. Recently, we have shown how symbolic algorithms based on Algebraic Decision Diagrams may be used to calculate the steadystate probabilities of finite state machines with more than 10 8 states. These algorithms treated machines with state graphs composed of a single terminal strongly connected component. In this paper we consider the most general case of systems which can be modeled as state machines with arbitrary transition structures. The proposed approach exploits structural information to decompose and simplify the state graph of the machine. 1 Introduction Finite state machines (FSMs), or their extensions, are often employed to model real digital systems for formal verification. As the complexity of those systems increases, probabilistic approaches to design and implementation verification become of interest; for...

Symbolic computation techniques play a fundamental role in logic synthesis and formal hardware verification algorithms. Recently, Algebraic Decision Diagrams, i.e., BDDs with a set of constant values different to the set /spl lcub/0,1/spl rcub/, have been used to solve general purpose problems, such as matrix multiplication, shortest path calculation, and solution of linear systems, as well as logic synthesis and formal verification problems, such as timing analysis, probabilistic analysis of finite state machines, and state space decomposition for approximate finite state machine traversal. ADD-based procedures for single-source and all-pairs shortest path weight calculation have appeared to be very effective for the manipulation of large graphs (over 10/sup 27/ vertices and 10/sup 36/ edges). However, for those procedures to be applicable to real problems, for example flow network problems, computing only shortest path weights is not enough; what it is needed is an algorithm that, given the weight of a shortest path between two vertices of a graph, actually determines the sequence of vertices belonging to the shortest path. This paper proposes a symbolic algorithm to execute shortest path back-tracing which exploits the compactness of the ADD data structure to handle large graphs.>

In this paper we present theory and experiments on the algebraic decision diagrams (ADDs). These diagrams extend BDD's by allowing values from an arbitrary finite domain to be associated with the terminal nodes. We present a treatment founded in Boolean algebras and discuss algorithms and results in applications like matrix multiplication and shortest path algorithms. Furthermore, we outline possible applications of ADD's to logic synthesis, formal verification, and testing of digital systems.