Recent technological innovations, especially in extended reality (XR) and artificial intelligence (AI), redefine storytelling approaches. These innovations are expanding creative possibilities for filmmakers while transforming how audiences engage with and experience films and other entertainment products. Despite technological advancements, the pre-production phase depends primarily on traditional planning and visualization methods. This research proposes a novel paradigm to create storyboards in XR, leveraging the capabilities of object-detection and pose-estimation systems to benefit the storyboarding phase. An application has been designed and developed to create 3D storyboards on a real scale within a physical environment and all its furniture. Wearing a head-mounted display for XR, users can move in the physical space, enrich it with virtual elements and characters, and frame the environment to obtain storyboard panels that mix real and virtual elements. The proposed system has been tested to assess its usability, and initial findings indicate that users have appreciated this application.

This article presents a brain–computer interface (BCI) coupled with an augmented reality (AR) system to support human–robot interaction in controlling a robotic arm for pick-and-place tasks. BCIs can process steady-state visual evoked potentials (SSVEPs), which are signals generated through visual stimuli. The visual stimuli may be conveyed to the user with AR systems, expanding the range of possible applications. The proposed approach leverages the capabilities of the NextMind BCI to enable users to select objects in the range of the robotic arm. By displaying a visual anchor associated with each object in the scene with projected AR, the NextMind device can detect when users focus their eyesight on one of them, thus triggering the pick-up action of the robotic arm. The proposed system has been designed considering the needs and limitations of mobility-impaired people to support them when controlling a robotic arm for pick-and-place tasks. Two different approaches for positioning the visual anchors are proposed and analyzed. Experimental tests involving users show that both approaches are highly appreciated. The system performances are extremely robust, thus allowing the users to select objects in an easy, fast, and reliable way.

Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for robotic teleoperation is evaluated to assess if it can be efficiently used in complex tasks that require accurate control of the robotic end-effector. The environment and the employed robot are captured using RGB-D cameras, while the remote user controls the motion of the robot with VR controllers. The captured data are transmitted and reconstructed in 3D so as to allow the remote user to monitor the task execution progress in real time, using a VR headset. The EVR system is compared with two other interface alternatives: i) teleoperation in pure VR (the model of the robot is rendered with respect to its real joint states), and ii) teleoperation in EVRR (the model of the robot is superimposed on the real robot). The results show that pure point cloud interfaces suffer from visualization issues, reducing the effectiveness of the robot teleoperation. However, the accuracy and user experience can be greatly improved by including the robot model.

In this work, a Mixed Reality (MR) system is evaluated to assess whether it can be efficiently used in teleoperation tasks that require an accurate control of the robot end-effector. The robot and its local environment are captured using multiple RGB-D cameras, and a remote user controls the robot arm motion through Virtual Reality (VR) controllers. The captured data is streamed through the network and reconstructed in 3D, allowing the remote user to monitor the state of execution in real time through a VR headset. We compared our method with two other interfaces: i) teleoperation in pure VR, with the robot model rendered with the real joint states, and ii) teleoperation in MR, with the rendered model of the robot superimposed on the actual point cloud data. Preliminary results indicate that the virtual robot visualization is better than the pure point cloud for accurate teleoperation of a robot arm.

Virtual character animation is receiving an ever-growing attention by researchers, who proposed already many tools with the aim to improve the effectiveness of the production process. In particular, significant efforts are devoted to create animation systems suited also to non-skilled users, in order to let them benefit from a powerful communication instrument that can improve information sharing in many contexts like product design, education, marketing, etc. Apart from methods based on the traditional Windows-Icons-Menus-Pointer (WIMP) paradigms, solutions devised so far leverage approaches based on motion capture/retargeting (the so-called performance-based approaches), on non-conventional interfaces (voice inputs, sketches, tangible props, etc.), or on natural language processing (NLP) over text descriptions (e.g., to automatically trigger actions from a library). Each approach has its drawbacks, though. Performance-based methods are difficult to use for creating non-ordinary movements (flips, handstands, etc.); natural interfaces are often used for rough posing, but results need to be later refined; automatic techniques still produce poorly realistic animations. To deal with the above limitations, we propose a multimodal animation system that combines performance- and NLP-based methods. The system recognizes natural commands (gestures, voice inputs) issued by the performer, extracts scene data from a text description and creates live animations in which pre-recorded character actions can be blended with performer’s motion to increase naturalness.

This paper presents a framework for automatically generating speech-based interfaces for controlling virtual and augmented reality (AR) applications on wearable devices. Starting from a set of natural language descriptions of application functionalities and a catalog of general-purpose icons, annotated with possible implied meanings, the framework creates both vocabulary and grammar for the speech recognizer, as well as a graphic interface for the target application, where icons are expected to be capable of evoking available commands. To minimize user's cognitive load during interaction, a semantics-based optimization mechanism was used to find the best mapping between icons and functionalities and to expand the set of valid commands. The framework was evaluated by using it with see-through glasses for AR-based maintenance and repair operations. A set of experimental tests were designed to objectively and subjectively assess first-time user experience of the automatically generated interface in relation to that of a fully personalized interface. Moreover, intuitiveness of the automatically generated interface was studied by analyzing the results obtained through trained users on the same interface. Objective measurements (in terms of false positives, false negatives, task completion rate, and average number of attempts for activating functionalities) and subjective measurements (about system response accuracy, likeability, cognitive demand, annoyance, habitability, and speed) reveal that the results obtained by the first-time users and experienced users with the proposed framework's interface are very similar, and their performances are comparable with those of both the considered references.

Drone-based delivery of goods could become a reality in the near future, as witnessed by the increasing successful experiences in both research and commercial fields. In this paper, a prototype system exploiting a do it yourself quad copter drone for delivering products is proposed. On the one hand, the hardware choices made in order to limit risks arising from autonomous delivery are presented. On the other hand, a framework for orders placement and shipment is shown. The advantages of a system like the one described in this paper are mainly related to an increased delivery speed, especially in urban contexts with traffic, to the possibility to make deliveries in areas usually difficult to be reached, and to the drone's ability to autonomously carry out consignments. A practical use case, in which the proposed system is used for delivering drugs (an application in which the need to quickly receive the good might be particularly important) is shown. Nevertheless, the proposed prototype could be employed in other contexts, such as take-away deliveries, product shipments, registered mail consignments, etc.

Existing tracking methods designed for interacting with projection-based displays generally require visible artifacts to be introduced in the environment in order to guarantee effective stability and accuracy. For instance, in optical-oriented approaches, either the camera sensor or the reference pattern used for tracking are often located within the user's sight (or interfere with it), thus occluding portions of the scene or altering the perception of the virtual environment. Several ways to tackle these issues have been recently explored. Proposed approaches basically aim at making the presence of tracking references in the virtual space transparent to the user. However, such solutions introduce possibly critical constraints on required hardware or environment configuration. In this work, a novel tracking approach based on imperceptible fiducial markers is proposed. The approach relies on a hiding technique that allows digital images to be embedded in (and retrieved from) a projected scene by exploiting the properties of light polarization and additive color mixing. In particular, the virtual scene is obtained by overlapping the light beams of two projectors and by dealing with markers’ hiding via color compensation. A prototype setup has been deployed, where interaction with a flat surface projection environment has been evaluated in terms of tracking accuracy and artifacts avoidance performance by using a consumer camera equipped with a polarizing filter. Although the performed tests presented in this article represent only a preliminary and a partial evaluation of the proposed approach, they provided encouraging results indicating that the proposed technique could be possibly applied in more complex interaction scenarios still with limited hardware requirements.

A wide variety of remotely controlled mobile robots have been developed in the past. Despite the evolution in robotics functionalities, not much effort has been made to customize control interfaces to meet user preferences and needs. General users can find difficulties in controlling mobile robots by standard interfaces. On the other hand, recent advances in multi-touch devices now allow researchers to design and implement intuitive and user friendly interfaces. This paper presents recent efforts to control mobile platforms by using customizable multi-touch gestures drawn on a commonly available hand-held device. In particular, a live stream video coming from an on board camera allows both to automatically and manually control the platform to accomplish target tracking and following tasks.

Abstract Shared visualization environments represent an effective means to enhance collaborative work in engineering and scientific design tasks. The availability of high‐speed networks allows researchers to work together from geographically distributed locations, and mobile devices are able to carry out ubiquitous 3D visualization tasks through wireless network connections. This paper presents a scalable architecture for the delivery of shared 3D visualization services to heterogeneous terminals ranging from powerful workstations to mobile devices such as PDAs and smart‐phones. The framework design allows both desktop and mobile clients to simultaneously visualize the same model by sharing a common view. Remote‐rendering servers support effective visualization on thin clients, and a load balancing mechanism provides efficient resource usage. Copyright © 2011 John Wiley & Sons, Ltd.

With the tremendous growth of information available to end users through the Web, search engines come to play ever a more critical role. Nevertheless, because of their general purpose approach, it is always less uncommon that obtained result sets provide a burden of useless pages. Next generation Web architecture, represented by Semantic Web, provides the layered architecture possibly allowing to overcome this limitation. Several search engines have been proposed, which allow to increase information retrieval accuracy by exploiting a key content of Semantic Web resources, that is relations. However, in order to rank results, most of the existing solutions need to work on the whole annotated knowledge base. In this paper we propose a relation-based page rank algorithm to be used in conjunction with Semantic Web search engines that simply relies on information which could be extracted from user query and annotated resource. Relevance is measured as the probability that retrieved resource actually contains those relations whose existence was assumed by the user at the time of query definition.

Abstract Scientific research and practical applications of solar physics require data and computational services to be integrated seamlessly and efficiently. The European Grid for Solar Observations (EGSO) leverages Grid‐oriented concepts and technology to provide a high‐performance infrastructure for solar applications. In this paper, an architecture for a data brokerage service is proposed. Brokers interact with providers and consumers in order to build a profile of both parties. In particular, a broker interacts with providers in order to gather information on the data potentially available to consumers, and with the consumers in order to identify the set of providers that are most likely to satisfy specific data needs. The brokerage technique is based on a multi‐tier management of metadata. Copyright © 2006 John Wiley & Sons, Ltd.

Mobile devices such as Personal Digital Assistants, Tablet PCs, and cellular phones have greatly enhanced user capability to connect to remote resources. Although a large set of applications are now available bridging the gap between desktop and mobile devices, visualization of complex 3D models is still a task hard to accomplish without specialized hardware. This paper proposes a system where a cluster of PCs, equipped with accelerated graphics cards managed by the Chromium software, is able to handle remote visualization sessions based on MPEG video streaming involving complex 3D models. The proposed framework allows mobile devices such as smart phones, Personal Digital Assistants (PDAs), and Tablet PCs to visualize objects consisting of millions of textured polygons and voxels at a frame rate of 30 fps or more depending on hardware resources at the server side and on multimedia capabilities at the client side. The server is able to concurrently manage multiple clients computing a video stream for each one; resolution and quality of each stream is tailored according to screen resolution and bandwidth of the client. The paper investigates in depth issues related to latency time, bit rate and quality of the generated stream, screen resolutions, as well as frames per second displayed.

Ray tracing programs are widely used to generate photo-realistic images, but the high computation time may discourage their implementation on single-processor machines; moreover, cost reduction of multi-processor general purpose architectures makes parallel rendering an attractive field of research. We propose a new algorithm which addresses the main issues of a parallel implementation of ray tracing on a message-passing-based machine. We adopt efficient strategies for dynamic workload distribution among processors, task synchronization and communication delay reduction. The resulting implementation is highly flexible, since any number of processors can be employed without introducing synchronization problems. We show an implementation of our algorithm on an nCUBE 2 supercomputer which is a general purpose parallel architecture with distributed memory. A theoretical evaluation of our algorithm allows us to identify a decreasing function for rendering times; the considered examples confirm the theoretical expectations showing that the efficiency of our system may reach up to the 91% of the value achievable by dividing the sequential rendering time by the number of processors employed.

This paper presents a new algorithm to generate ray-cast CSG animation frames. We consider sequences of frames where only the objects can move; in this way, we take advantage of the high screen area coherence of this kind of animation. A new definition of bounding box allows us to reduce the number of pixels to be computed for the frames after the first. We associate a CSG subtree and two new flags, denoting if the box has changed in the current frame and if it will change in the next frame, with each box. We show with three examples the advantages of our technique when compared with an algorithm which entirely renders each frame of an animation. Intersections with CSG objects may be reduced to about one-fifth, while the rendering may be computed up to four times faster for the test sequences. © 1998 John Wiley & Sons, Ltd.

The Q-Coder algorithm is a very efficient compression technique for bi-level images based on the arithmetic coding. The paper presents a new and fast version of the Q-Coder algorithm in which the carry-propagated adders have been replaced by carry-save adders. In this way, all the additions can be performed with a delay time of a single fill adder, independently of the length of the operands. The compression method is faster than the traditional Q-Coder algorithm with an almost unnoticeable increasing of the hardware requirements.

The generation of 3-D solid objects, and more generally solid geometric modelling, is very important in Computer Aided Design (CAD). An important role is played by the Constructive Solid Geometry (CSG) representation scheme. IN CSG, objects are described by trees of Boolean operations on half-spaces or boundaries of primitive solids. The study of techniques to speed up the rendering of scenes modelled with the CSG scheme is an attractive field of research; in this paper we propose a new algorithm which reduces the computational complexity for ray casting approaches. Our strategy identifies a set of areas on the plane of view where the rays starting from the observer have to be traced; for each zone, only a portion of the entire CSG tree has to be considered for intersection tests, instead of the whole database of the primitive objects. A comparison of our algorithm with a ray caster that adopts bounding volume hierarchies and with a freeware ray tracer called POV-Ray shows that, for the examples considered, we may reduce the intersection tests to one third of those performed when standard optimizations are adopted.