The progression of AI technology has spurred a growing number of artists to engage in AI Art production. This trend has sparked a multifaceted societal discourse. Given the tight integration of AI Art with society, it is essential to explore the potential art innovations and impacts AI Art might introduce. This article explores the relationship between AI visual art and conventional art, tracing the development of AI visual art and its societal reception. Through historical case studies, the article forecasts the future influence of AI visual art in the following aspects: Regarding artistic creation, AI visual art challenges established criteria for artistic evaluation. It presents artists with enhanced learning and creative capacities, paving the way for a new artist archetype characterized by human-computer symbiosis. From a societal standpoint, AI visual art is to advance the democratization of art, developing alongside traditional art forms.

Drawing its name from the Uyghur expression for’dance on silk,’’DanceYipékda’ epitomizes the elegance and fluidity of cultural expression that this installation seeks to embody. This project unites the realms of AI-generated calligraphy, projection mapping, and 3D printing to weave a narrative as intricate as silk itself—reflecting the complexity and beauty of cross-cultural communication. Leveraging cutting-edge AI technologies, the artwork initiates the creative process by generating 2D calligraphic art, which is subsequently transformed into a 3D format, further accentuated with dynamic projection mapping that breathes life and color into the static forms. As a multimedia art installation,’DanceYipékda’ brings to life the ancient tradition of calligraphy, employing technology as a bridge between historical art forms and contemporary artistic expression. It is a dialogue across time, a dance of light and shadow, sparking a transformative conversation on the convergence of art, technology, and society, encouraging viewers to construct a collective understanding that surpasses linguistic and geographical barriers.

BLE beacon networks are widely adopted for IoT and smart city applications, but they are susceptible to security threats, including piggybacking and spoofing attacks. These attacks can infringe on or even jeopardize the profitability of network owners. To address this issue, BLE beacon packets are often encrypted or updated periodically, which requires frequent synchronization between individual beacons and a centralized server, consuming significant resources. We propose a novel crowd-assisted, secure BLE identifier (ID) updating framework that significantly reduces network overhead. The proposed framework quantifies mobile crowdsourcing participant presence to dynamically adjust the pool size of the required beacon IDs. Our experiments, which factor in real-life user presence information, prove the practicality of our framework, which reduced the network resource consumption by up to 90%.

Abstract Food plays an important role in our lives that goes beyond mere sustenance. Food affects behavior, mood, and social life. It has recently become an important focus of multimedia and social media applications. The rapid increase of available image data and the fast evolution of artificial intelligence, paired with a raised awareness of people’s nutritional habits, have recently led to an emerging field attracting significant attention, called food computing, aimed at performing automatic food analysis. Food computing benefits from technologies based on modern machine learning techniques, including deep learning, deep convolutional neural networks, and transfer learning. These technologies are broadly used to address emerging problems and challenges in food-related topics, such as food recognition, classification, detection, estimation of calories and food quality, dietary assessment, food recommendation, etc. However, the specific characteristics of food image data, like visual heterogeneity, make the food classification task particularly challenging. To give an overview of the state of the art in the field, we surveyed the most recent machine learning and deep learning technologies used for food classification with a particular focus on data aspects. We collected and reviewed more than 100 papers related to the usage of machine learning and deep learning for food computing tasks. We analyze their performance on publicly available state-of-art food data sets and their potential for usage in multimedia food-related applications for various needs (communication, leisure, tourism, blogging, reverse engineering, etc.). In this paper, we perform an extensive review and categorization of available data sets: to this end, we developed and released an open web resource in which the most recent existing food data sets are collected and mapped to the corresponding geographical regions. Although artificial intelligence methods can be considered mature enough to be used in basic food classification tasks, our analysis of the state-of-the-art reveals that challenges related to the application of this technology need to be addressed. These challenges include, among others: poor representation of regional gastronomy, incorporation of adaptive learning schemes, and reverse engineering for automatic food creation and replication.

Bluetooth Low Energy (BLE) beacon network is one of the essential infrastructures for many IoT and smart city applications that involve a plethora of sensing tasks. However, the BLE beacon network usually suffers from poor reliability and high maintenance costs due to the short-lived battery lifetime. Multiple works have attempted to extend the lifetime via energy harvesting hardware, adaptive advertising interval by user existence-aware operation, and energy-efficient routing schemes. However, few attempts were made to reduce the energy consumption related to sensing tasks. In light of this shortcoming, a sensor information-aware framework is proposed to adjust the sensing task interval adaptively based on the predicted portion of changes of the sensor measurements. Furthermore, to estimate the impact of varying sensing task intervals on the amount of sensed information, a model that correlates energy and amount of information is proposed. The sensor portion of changes is predicted with a novel neural network, coined oracle-interpreter network, that significantly reduces the energy consumption while upkeeping a good prediction accuracy by leveraging two independent neural networks tailored for feature extraction and prediction tasks. The effectiveness of the proposed framework is verified by comprehensive simulations based on real-life data. The results demonstrate that the proposed framework can effectively reduce the energy consumption involved in sensing tasks up to 30%, machine learning tasks up to 60%, and finally, extend the lifetime up to 75%.

Bluetooth Low Energy (BLE) beacon network is one of the essential infrastructures for many IoT and smart city applications. However, the BLE beacon network usually suffers from poor reliability and high maintenance costs due to the short-lived battery lifetime. A few recent works tackled the challenge by adjusting the operating configuration subject to the nearby user occupancy in a reactive manner. However, previous works failed to adapt to dynamically changing user occupancy behaviors due to the lack of a prediction mechanism. Such shortcomings lead to a shorter lifetime and longer packet arrival delays. To overcome this limitation, a novel neural network architecture that makes accurate and timely user occupancy prediction is introduced. The proposed network learns partial correlation of the time-series data and attention score for robust prediction. The predictions are then utilized to adaptively change the operation configurations to maximize the lifetime and minimize the packet arrival delay. To the best of our knowledge, this is the first work to leverage time-series prediction to optimize the performance of a BLE beacon. The effectiveness of the proposed learning methods is verified by comprehensive simulations with real-life data. The results demonstrate that the proposed method can extend a beacon lifetime by 50% more than the existing reactive approach. Moreover, the packet arrival delays are also reduced by up to 40%.

To address the short-lived battery lifetime of Bluetooth low energy (BLE) beacons, researchers proposed solar-powered designs, equipped with rechargeable energy storage such as a supercapacitor. However, accurately monitoring the energy status - an essential step for device maintenance - has shown to be a major concern. Existing energy status monitoring methods, which are either crowd-assisted or require on-site data collection, suffer from severe losses of energy status information. This paper presents an energy status recovery framework with support vector regression (SVR) to address this issue. The proposed framework leverages recurrence training of SVR with lost energy status information to capture features from discharge behavior, achieving high accuracy while minimizing training and prediction time. Multiple real-life BLE beacon energy level records are evaluated to demonstrate that our proposed framework can recover the energy information with at least 98% accuracy under a data loss rate of up to 99%.

With the current and continuous advancements in the field of text-to-image modeling, it has become critical to design prompts that make the best of these model capabilities and guides them to generate the most desirable images, and thus the field of prompt engineering has emerged. Here, we study a method to use prompt engineering to enhance text-to-image model representation of the Arabic culture. This work proposes a simple, novel approach for prompt engineering that uses the domain knowledge of a state-of-the-art language model, GPT, to perform the task of prompt augmentation, where a simple, initial prompt is used to generate multiple, more detailed prompts related to the Arabic culture from multiple categories through a GPT model through a process known as in-context learning. The augmented prompts are then used to generate images enhanced for the Arabic culture. We perform multiple experiments with a number of participants to evaluate the performance of the proposed method, which shows promising results, specially for generating prompts that are more inclusive of the different Arabic countries and with a wider variety in terms of image subjects, where we find that our proposed method generates image with more variety 85 % of the time and are more inclusive of the Arabic countries more than 72.66 % of the time, compared to the direct approach.

A sensor network is an underpinning infrastructure that enables various future IoT applications, such as precision agriculture, smart farm, and greenhouse monitoring. However, these sensor devices often suffer from short-lived battery lifetime that incurs frequent maintenance operation. Although there have been a few attempts to smartly reduce the power consumption associated with communication tasks of the sensors, very few have addressed the power consumption of sensing tasks. In light of this shortcoming, we propose an information-aware sensing framework that adaptively adjusts the sensing interval for energy-saving operations based on the learned behavior of the sensor data. To prove the effectiveness of the proposed framework, we have deployed four BLE beacons equipped with luminosity and temperature sensors to collect real-life data from a desert greenhouse, which is then used to train and evaluate our proposed framework. Additionally, we have implemented the proposed framework on a commodity BLE beacon device to validate the energy-saving performance of the proposed framework. The results demonstrate that the proposed framework can effectively reduce the energy consumption involved in sensing tasks by 30% and extend the battery lifetime by up to 75%.

With growing demands for Internet of Things (IoT) applications, BLE beacon networks are rapidly being adopted. Periodic battery replacement operations and onsite maintenance are required to ensure continuous and reliable service. These operations are labor intensive and resource exhaustive. Therefore, Bluetooth gateways/mobile devices are often employed to monitor/collect the energy status. However, the gateways consume a considerable amount of network requests, and the user existence influences the data collection, thus the data accuracy, based on mobile devices. Reducing the number of energy status reports and maintaining the high accuracy of the energy status monitoring service is essential to catalyze a generic adoption of beacon networks and IoT infrastructure of similar nature in more businesses and real-life applications. In this article, we proposed a novel energy status monitoring framework that will dynamically change the energy status report interval based on the discharging rate of the battery, thereby reducing the total number of network requests and maintaining the required accuracy of energy status. The proposed framework identifies the BLE beacons with similar battery discharging rates, suggests a dynamic report interval, and leverages this information to reduce the number of energy status reports. We have experimented with real-life BLE beacon energy status data for 50 days to demonstrate that we could reduce the total number of network requests up to 70% while retaining 99% estimation accuracy.

Bluetooth Low Energy (BLE) beacons are designed to operate for years on a coin-cell battery. However, the formation of a mesh network, overlaying on the existing Bluetooth Low Energy (BLE) beacons infrastructure, can severely degrade the lifetime of underlying beacons owing to the excessive current drawn by the scanning event. Even though we can sustain the lifetime of the underlying beacon with duty-cycle scanning, such duty-cycle scanning imposes another challenge to the overlay mesh in disseminating the packet. To this end, this paper proposes a novel overlay protocol that: 1) employs duty-cycle scanning to guarantee the lifetime of the underlying beacon, while 2) defining a set of scanning policies to increase the packet dissemination rate through the overlay mesh network. The duty-cycle scanning defines the scanning time slot based on the lowest feasible duty cycle unveiled through a comprehensive analysis of energy consumed by advertising and scanning events. The scanning policies, on the other hand, allow each node to explore all possible time slots before locking their scanning event to a particular time slot that is most likely to hear the incoming packet. Extensive experiments with practical implementation demonstrate the feasibility of our proposed overlay mesh for real-world use cases.

This paper presents a nonlinear location estimation to infer the position of a user holding a smartphone. We consider a large location with$M$number of grid points, each grid point is labeled with a unique fingerprint consisting of the received signal strength (RSS) values measured from$N$number of Bluetooth Low Energy (BLE) beacons. Given the fingerprint observed by the smartphone, the user’s current location can be estimated by finding the top-k similar fingerprints from the list of fingerprints registered in the database. Besides the environmental factors, the dynamicity in holding the smartphone is another source to the variation in fingerprint measurements, yet there are not many studies addressing the fingerprint variability due to dynamic smartphone positions held by human hands during online detection. To this end, we propose a nonlinear location estimation using the kernel method. Specifically, our proposed method comprises of two steps: 1) a beacon selection strategy to select a subset of beacons that is insensitive to the subtle change of holding positions, and 2) a kernel method to compute the similarity between this subset of observed signals and all the fingerprints registered in the database. The experimental results based on large-scale data collected in a complex building indicate a substantial performance gain of our proposed approach in comparison to state-of-the-art methods. The dataset consisting of the signal information collected from the beacons is available online.

Selling counterfeit online has become a serious problem, especially with the advancement of social media and mobile technology. Instead of investigating the products directly, one can only check the images, tags annotated by the sellers on the images, or the price to decide if a seller sells counterfeits. One of the ways to detect counterfeit sellers is to investigate their social graphs, in which counterfeit sellers show different behaviour in network measurements, such as those in centrality and EgoNet. However, social graphs are not easily accessible. They may be kept private by the operators, or there are no connections at all. This article proposes a framework to detect counterfeit sellers using their connection graphs discovered from their shared images. Based on 153 K shared images from Taobao, it is proven that counterfeit sellers have different network behaviours. It is observed that the network measurements follow Beta function well. Those distributions are formulated to detect counterfeit sellers by the proposed framework, which is 60% better than approaches using classification.

To address the short-lived battery lifetime of Bluetooth low energy (BLE) beacons, solar-powered designs were proposed, equipped with rechargeable energy storage such as a supercapacitor. However, energy status monitoring, which is essential for device maintenance, proved to be a major concern as the energy status of energy harvesting devices can change quickly due to charging and discharging behaviours. Existing energy status monitoring methods performed in a crowd-assisted manner or by demanding on-site data collections are accompanied by severe loss of energy status information. This paper presents an accurate energy status recovery framework with SVR to address this issue. The proposed framework leverages recurrent training of SVR with lost energy status information to capture features from discharge behaviour to achieve high accuracy while minimizing the training and prediction time. Multiple real-life BLE beacon energy level records are evaluated to demonstrate that our proposed framework can recover the energy information with at least 90% accuracy under a data loss rate of up to 99%.

One key feature of Bluetooth low energy (BLE) beacons is the received signal strength, which can be used to estimate the distance between any Bluetooth-compatible receiver (e.g., smartphone, tablet, etc.) and fixed deployed beacon. Although received signal strength (RSS) can be measured easily with commonly available smart devices, the measurements are unreliable, in which general estimation models are not robust to different hardware and settings for real deployed beacon networks. Furthermore, the lack of consideration for object mobility in these models undermines its practicality. Motivated by the above limitations, this article proposes a novel distance classifier, d-Classifier, to classify the distance with a feature vector constructed with features such as hardware type and deployment environment to improve the robustness. Moreover, comprehensive experiments related to mobility are conducted to study the relationship between packet receiving rate and estimation accuracy. Improved performance can be achieved by providing extra mobility information with a list of RSS values during estimation. The proposed classifier is validated with an extensive data set that includes over 200 k data collected from real beacon networks. Overall, our proposed d-Classifier achieves a significant performance gain,$>25\%$accuracy improvement, over its prior arts.

Bluetooth low energy (BLE) beacon networks are one common infrastructure for IoT and smart city applications because of their scalability and affordability, as well as the proliferation of Bluetooth-enabled devices. However, BLE beacon networks suffer from short battery lifetime, which induces additional maintenance costs. In this paper, we propose a novel user existence-aware BLE beacon firmware, User-B, that extends BLE beacon lifetime by changing its operating configuration. Leveraging scan response and request features of Bluetooth Core Specifications, a mechanism for the detection of nearby user smartphones is proposed. Furthermore, we present an energy consumption model of the proposed firmware, along with an optimization problem for finding the optimal configuration that minimizes the overall energy consumption and overhead induced by switching delay. Last but not least, we introduce a prototype of the User-B firmware and demonstrate experiments. Through the experiments, we prove that the User-B firmware can extend a beacon's lifetime up to 250 percent under low user-existence frequency and high energy demand application conditions.

This paper presents a novel bluetooth low energy (BLE) protocol enabling a BLE node to perform RF fingerprint acquisition by measuring the received signal strength (RSS) from its neighboring nodes and simultaneously broadcast the acquired fingerprint via its advertising packet. However, the fingerprint acquisition and broadcast process in a dense BLE network is very challenging owing to: 1) the likelihood of packet collision; and 2) the length-constrained packet. To this end, we exploit a compressive sensing (CS) framework allowing each node to acquire no more than$M$measurements from a very dense network, in which the number of nodes$N$is far greater than$M$. By aggregating the$M$-dimensional compressed fingerprint vector from$s

Technologies related to artificial intelligence (AI) have a strong impact on the changes of research and creative practices in visual arts. The growing number of research initiatives and creative applications that emerge in the intersection of AI and art motivates us to examine and discuss the creative and explorative potentials of AI technologies in the context of art. This article provides an integrated review of two facets of AI and art: (1) AI is used for art analysis and employed on digitized artwork collections, or (2) AI is used for creative purposes and generating novel artworks. In the context of AI-related research for art understanding, we present a comprehensive overview of artwork datasets and recent works that address a variety of tasks such as classification, object detection, similarity retrieval, multimodal representations, and computational aesthetics, among others. In relation to the role of AI in creating art, we address various practical and theoretical aspects of AI Art and consolidate related works that deal with those topics in detail. Finally, we provide a concise outlook on the future progression and potential impact of AI technologies on our understanding and creation of art.

This paper presents a novel Q-Learning method in forwarding the proximity sensing information to the remote server through low-power mesh network overlays on Bluetooth Low Energy (BLE) technology. Even though proximity sensing information can be easily monitored with pervasive smartphones, it is almost impossible to remotely monitor this information in a harsh location where it is not easy to access the Internet. With our overlay mesh network, each node should decide to either forward the packet or continue with their own activity when receiving the packet forwarding request, so as to minimize the end-to-end packet delivery latency but maximize the utilization of underlying infrastructures. Reinforcement learning (RL) is employed to train each node to make the above decision. Despite extensive upfront training, there is a high possibility that each node might still encounter an unseen state owing to the network dynamics. However, our novel Q-learning is able to deal with above challenges by constructing a Q-table during online learning, and then use the Q-table as input data for offline training. The experimental results indicate the substantial performance gain of our proposed approach in comparison to the existing Q-learning methods.

As Internet of things (IoT) infrastructures such as BLE beacon networks are gaining more attention, excessive battery consumption and subsequent maintenance operations have proven to be crucial drawbacks. As part of the green IoT trend, light energy harvesting BLE beacons have been proposed in the literature to reduce energy consumption from batteries and extend their lifetime. However, these devices do not consider adjusting the angle of the solar panel, which prevents them from maximizing their energy harvesting capability by adapting to various indoor lighting conditions. Furthermore, an algorithm to compute the angle to optimize the energy harvesting capability has not yet been investigated for small energy harvesting IoT devices. To address such issues, our paper first proposes a model of lighting conditions in an environment with multiple light sources, and based on the proposed model, we present an algorithm to compute the optimal angle of the solar panel that would maximize the harvested energy. Finally, we prototype an energy harvesting BLE beacon with an adjustable solar panel angle and conduct real-life experiments in three different locations with varying lighting conditions. The experiments prove that the proposed design can accelerate the energy storage charging rate by up to 570%.

Bluetooth Low Energy (BLE) beacon network is one of the essential infrastructures for many IoT and smart city applications due to the proliferation of Bluetooth-enabled devices. However, the BLE beacon network usually suffers from high maintenance costs due to the short battery lifetime. A recent work proposed duty-cycling BLE advertising interval subject to the detected existence of a user; if a user is detected, the beacon operates in a shorter advertising interval, otherwise operates in a longer advertising interval to reduce its energy consumption. However, since such reactive approach operates based on hardcoded advertising intervals for the two scenarios, the energy-efficiency is bound to be sub-optimal. To overcome this limitation, this paper proposes to predict the user traffic condition thereby adapting the optimal advertising interval based on the predicted user traffic condition. To this end, we introduce a novel neural network architecture that learns partial correlation and leverages attention mechanism to make accurate predictions with minimum prediction lag. The effectiveness of the proposed learning methods is verified by comprehensive simulations. It is proved that the proposed method can extend a beacon lifetime by at least 200% more than the state-of-the-art techniques.

In recent years, online social networks (OSNs) have become extremely popular and been one of the most common ways for storing and distributing images. Naturally, such widespread availability of OSN makes it a viable channel for transmitting additional data along with the image sharing. However, various lossy operations, e.g., resizing and compression, conducted by OSN platforms impose great challenges for designing a robust watermarking scheme over OSN shared images. In this paper, we tackle this challenge and propose a robust high-capacity watermarking technique, by using Facebook as a representative OSN. To achieve the satisfactory robustness, we first probe into Facebook and recover the image manipulation mechanism via a deep convolutional neural network (DCNN) approach. Assisted with the precise knowledge on the lossy channel offered by Facebook, we then suggest a DCT-domain image watermarking method that is highly robust against the lossy operations on Facebook, even without any error correcting codes (ECC). The proposed technique is also extended to other popular OSNs, e.g., Wechat and Twitter. Extensive experimental results are provided to show the superior performance of our method in terms of the embedding capacity, data extraction accuracy, and quality of the reconstructed images.

While many technologies (e.g., RFID, QR code, etc.) have been developed for items identification, they fail to provide continuous monitoring for items in transit. This paper introduces a Bluetooth Low Energy (BLE) beacon-based system, which can be deployed easily with any off-the-shelf smartphone without modification on the existing infrastructures. However, it is an elusive challenge to achieve a fast item identification and counting involving massive items stacked up inside a confined space (e.g., a container), resulting in an ultra-dense beacon network (UDBN). To this end, we propose a novel beaconing solution capable of informing the receiver about their own presence as well as the presence of their neighboring beacons for identification purpose. Specifically, our proposed solution provides a well-designed yet innovative protocol data unit (PDU) which allows the beacon to encapsulate its neighboring information into its own advertising packet. A prototype consisting of 300 beacons is implemented to demonstrate the feasibility of our proposed solution for real-world applications. The extensive experiment confirm the superiority of our proposed solution in delivering a fast item identification and counting in UDBN.

"Keep Running" is a collection of human and machine generated paintings using a generative adversarial network technology. The horse artworks are produced during the lockdown period in the Middle East due to the Covid-19. Many recent AI artworks are either generated in photo-realistic style, or abstract style with distorted faces, fragmented figures and a combination of unknown objects. Besides the cultural and historic symbols that horses represent in this region, what's unique with our work is showing the possibility of using AI to create horse paintings with distinguishable features and forms, while still rendering different aesthetic and even sentimental expressions in the horse paintings. Our first artwork is a series of storytelling-like paintings of an evolving horse figure in motion with changing backgrounds. Another one is a set of different horse portrait paintings that are presented in a grid with each of them evolved and generated stylishly from the same yet repeated machine processes. Our AI artworks are not just artistic and meaningful, but also paying a salute to the early works of machine-assisted art by Eadweard Muybridge and Andy Warhol, for their influences to the art world today.

Bluetooth Low Energy (BLE) beacon networks are a popular infrastructure for IoT and smart city applications due to their scalability and affordability, as well as the proliferation of Bluetooth-enabled devices. However, BLE beacon networks suffer from short battery lifetime, inducing additional maintenance costs. Previous works have tackled this problem by proposing a more energy-efficient BLE beacon firmware that will change its operating configuration based on user existence information. However, previous efforts could not adapt to varying user traffic conditions and therefore was impractical. To address this issue, this paper proposes a novel neural network-driven framework, User-P, that extends beacon lifetime by changing its operating configuration by predicting user traffic conditions. Furthermore, the paper also presents a novel machine learning method tailored for user traffic prediction. Last but not least, the effectiveness of the proposed framework and methods are proven through a set of simulations. The simulation results show that the proposed framework can extend the beacon lifetime by 180% in comparison to that of the state-of-the-art techniques.

Bluetooth low-energy (BLE) beacon is a key enabler for smart interaction between the user device and the physical thing, in which the physical thing can actively engage users for interaction via its advertising packet. However, reliability is always an issue for the beacon-based interaction since the beacon employs an unreliable broadcasting approach which provides no way to check if the user device has received the correct packet. We define the sparse observation to describe the phenomenon where the number of packets received by the user device within an arbitrarily small time duration is less than the number of deployed beacons. This article studies the sparse observation causing by the following two factors: 1) the unpredictable environmental variations and 2) the uncontrollable operating conditions of a beacon. An analysis is provided to investigate the interaction reliability in connection with the above two factors. Motivated by the above challenges, a novel solution, which exploits the ambient RF fingerprinting to address the sparse-observation issues, is proposed to enhance the interaction reliability. Our proposed solution is validated with extensive experiments consisting of real data collected from both indoor and outdoor environments. Finally, the feasibility of our proposed solution is demonstrated with a proof-of-concept prototype implemented over multiple physical things.

With the advance of mobile technology and social media, image sharing has become part of our daily lives. For many applications, such as follower/followee recommendation, shared images are an excellent source to discover connections among users who shared them. Shared images on social media are like invitations for user interactions, such as comment, like, and more. Social signals are in those images, and those signals can be objects that interest related users such that they will start to interact. Conventionally, connections among users are discovered through recognizing objects among those shared images, such as a using convolutional neural network (CNN) to extract features that are sensitive to object recognition. However, social signals are not limited to object, they can be colour, textual, or even a concept that may not be captured effectively by conventional CNN. This paper proposes a CNN-based analytic framework to detect social signals among users. The CNN is optimized using a triplet network with user-shared images, and the relationships among users who upload them. It is observed that images from 2 users with a connection have a shorter distance after encoding, than 2 users without a connection. A framework is implemented, which is verified with over 1.7 million images by over 2000 users from two image-oriented social networks, Skyrock and Flickr. It is proven that the proposed analytic framework shows an up to 89% improvement on approaches using object recognition for follower/followee recommendation. To the best of our knowledge, this paper is the first to propose an analytic framework to detect social signals from visual features for connection discovery.

Lately, there are many works exploited the radio frequency (RF) fingerprint for occupancy detection. However, most works suffer severe performance variations owing to the unreliable received signal strength (RSS). In this paper, we propose a deep learning approach to occupancy detection: 1) an unsupervised denoising-contractive autoencoder (DCAE) is built to learn a robust fingerprint representation from the raw RSS measurements, and 2) a supervised softmax function is added at the last layer for classification. A real testbed with Bluetooth Low Energy (BLE) beacons was built such that we can collect real-world RSS data for experiments. The data were collected via different devices at different times to better reflect environmental variations. The experimental results show that our proposed approach achieves a substantial performance gain in comparison to the conventional machine learning approaches. Specifically, our proposed DCAE is able to reconstruct the noisy and always changing data with less than 0.047 mean square error. Overall, our occupancy detection combining DCAE and softmax classifier achieves sub-room level accuracy for at least 99.3% of the time.

Bluetooth Low Energy (BLE) beacon networks are one common infrastructure for IoT and smart city applications because of their scalability and affordability, as well as the proliferation of Bluetooth-enabled devices. However, BLE beacon networks suffer from short battery lifetime, which induces additional maintenance costs. In this paper, we propose a novel user existence-aware BLE beacon firmware, USTEA, that extends BLE beacon lifetime by changing its operating configuration. Leveraging scan response and request features of Bluetooth Core Specifications, a mechanism for the detection of nearby user smartphones is proposed. Furthermore, we present an energy consumption model of the proposed firmware, along with an optimization problem for finding the optimal configuration that minimizes the overall energy consumption and overhead induced by switching delay. Last but not least, we introduce a prototype of the USTEA firmware and demonstrate experiments. Through the experiments, we prove that the USTEA firmware can extend a beacon's lifetime up to 250% under low user-existence frequency and high energy demand application conditions.

While Bluetooth Low Energy (BLE) beacons have been massively deployed to broadcast their advertising packets to any receivers in their vicinity, it is relatively difficult, if not impossible, for a beacon to report the packet back to the server in the absence of a receiver. This paper proposes a novel BLE-based overlay mesh (BOM) that enables the mesh functionality to existing beacon networks without introducing new infrastructure. However, it is an elusive challenge to jointly manage the beaconing and flooding events. To this end, BOM employs 1) best-effort scheduling (BES) to minimize the packet collision rate (PCR) while scheduling the time slots for beaconing events, and 2) received signal strength (RSS)-based bounded flooding (RBF) to maximize the packet delivery ratio (PDR) for the advertising packet while forwarding the relaying packet across the BOM network. Extensive simulations indicate the substantial performance gain of our proposed approach in comparison to the legacy approaches. Specifically, BES reduces the PCR to 66.67%, whereas RBF improves the PDR for the advertising packet to 52% while maintaining approximately the same PDR for the relaying packet. The practical experiment with a real network testbed further demonstrates the feasibility of BOM.

The development of Internet of Things technology has connected the smart things to the Internet, enabling users to interact for different applications such as indoor positioning or location-based notification service. To improve the user experience, an accurate distance estimation is required to ensure the interaction can be delivered precisely. For general beacon-based application, the objects keep moving while they are interacting with the beacons. Therefore, their mobility needs to be considered for distance estimation. In this paper, comprehensive experiments are conducted to study the relationship between distance estimation accuracy and packet received rate from two angels, the beacon advertising interval and the object moving speed. Moreover, an improved distance estimation method using Kalman filter and support vector regression is proposed, which has archived at least 40% improvement comparing to current approaches. The proposed idea is also implemented in real-world application which archive less than 100μs computation time.

Bluetooth low energy (BLE) beacon network is one of the most favored IoT infrastructures due to its flexibility and scalability. Monitoring and updating the battery statuses of the on-site BLE beacons is an essential task for reliable operation and timely maintenance of the infrastructure. However, unregulated frequent updates of the battery statuses result in stressing the beacon network management platform, possibly threatening the reliable operation of the infrastructure. Whereas too infrequent updates degrade the freshness and reliability of the updated information. Without a reliable estimation on battery status, management and timely battery replacement operation would be difficult. To address this issue, this paper presents an efficient update method of battery status for BLE beacon network that minimizes the stress on the management platform server. The proposed approach leverages the correlation in battery status information between certain beacons to reduce the number of necessary updates while retaining high accuracy. Necessary reference data estimation, reference data reliability checking, and error correction on the estimation are the three major components in the solution. An estimation model allows accurate estimation in the cold-start stage. Moreover, an error-correction model allows to check the reliability of reference data and make a correction on the estimated value.

The maturing deployment of the Internet of Things is gradually realizing new smart applications that strongly leverage recent advances in proximity detection methods. To this end, Bluetooth low energy (BLE) beacons are one of the preferred candidates because of the widespread use of Bluetooth-enabled devices. However, traditional battery-powered BLE beacons suffer from a limited operation lifetime, inducing additional maintenance operations and costs. This paper addresses this issue by proposing design principles for an ambient light energy harvesting BLE beacon capable of perpetual operation in the indoor environment. The contributions made in this paper include: 1) investigation and modeling of related hardware components, namely the BLE beacon, photovoltaic panel, and capacitor; 2) design principles for selecting hardware components subject to varying environmental conditions and application requirements; and 3) prototyping and field-tests to prove its practicality. Through multiple experiments, this paper proves that the design can operate perpetually under 40 lux light intensity, and can last over 17 h once fully charged.

Device-free occupancy detection is very important for certain Internet of Things applications that do not require the user to carry a receiver. This paper achieves the device-free occupancy detection with RF fingerprinting, which labels each zone with a 2M-dimensional fingerprint vector. Specifically, the fingerprint vector consists of received signal strength (RSS) values measured from M Bluetooth low energy (BLE) beacons and also their corresponding temporal RSS variations. However, the unreliable RSS values caused two common issues with the fingerprint vector: 1) noise and 2) sparsity. To this end, we propose denoising-contractive autoencoder (DCAE) to jointly deal with these two issues, by learning a robust fingerprint prior to device-free occupancy detection. We validate the performance of our proposed DCAE with large-scale real-world datasets. The experimental results indicate the substantial performance gain of our proposed DCAE in comparison with state-of-the-art autoencoders. In particular, the classifier trained using the fingerprints learned by our proposed DCAE is able to maintain at least 90% accuracy when the noise factor or sparsity ratio increases to 0.6 and 0.5, respectively.

Bluetooth low energy (BLE) beacons have been widely deployed to deliver proximity-based services (PBSs) to user's smartphones when users are in the proximity of a beacon. Conventional proximity detection simply uses the received signal strength (RSS) to infer the proximity, and then retrieves the PBS by mapping the beacon ID with the corresponding service in the cloud database. Such an approach suffers two major issues: 1) the severe RSS fluctuation might confuse the smartphone during the detection and 2) a malicious PBS can be delivered by manipulating the same beacon ID. This paper proposes RF fingerprinting to label a beacon with an N-dimensional fingerprint vector, which consists of N RSS values from N deployed beacons. The contribution of our proposed method is twofold: 1) we infer the proximity based on the fingerprint vector instead of relying solely on the single RSS value and 2) we retrieve the PBS by mapping the fingerprint vector instead of the hard-coded beacon ID. The challenge with our proposed approach is the incomplete fingerprint observation during real-time detection, resulting in an underdetermined proximity detection problem. To this end, we exploit the compressive sensing (CS) approach based on the differential evolutional algorithm to address such an underdetermined problem. Extensive simulations with realworld datasets show that our proposed approach outperforms the legacy machine learning techniques with substantial performance gains.

With the advancement of social media and mobile technology, any smartphone user can easily become a seller on social media and e-commerce platforms, such as Instagram and Carousell in Hong Kong or Taobao in China. A seller shows images of their products and annotates their images with suitable tags that can be searched easily by others. Those images could be taken by the seller, or the seller could use images shared by other sellers. Among sellers, some sell counterfeit goods, and these sellers may use disguising tags and language, which make detecting them a difficult task. This article proposes a framework to detect counterfeit sellers by using deep learning to discover connections among sellers from their shared images. Based on 473K shared images from Taobao, Instagram, and Carousell, it is proven that the proposed framework can detect counterfeit sellers. The framework is 30% better than approaches using object recognition in detecting counterfeit sellers. To the best of our knowledge, this is the first work to detect online counterfeit sellers from their shared images.

Visual arts, especially paintings, appear everywhere in our daily lives. They are not only liked by art lovers but also by ordinary people, both of whom are curious about the stories behind these artworks and also interested in exploring related artworks. Among various methods, the mobile visual search has its merits in providing an alternative solution to text and voice searches, which are not always applicable. Mobile visual search for visual arts is far more challenging than the general image visual search. Conventionally, visual search, such as searching products and plant, focuses on locating images containing similar objects. Hence, approaches are designed to locate objects and extract scale-invariant features from distorted photos that are captured by the mobile camera. However, the objects are only part of the visual art piece; the background and the painting style are both important factors that are not considered in the conventional approaches. In this article, an empirical investigation is conducted to study issues in photos taken by mobile cameras, such as orientation variance and motion blur, and how they influence the results of the mobile visual arts search. Based on the empirical investigation results, a photo-rectification pipeline is designed to rectify the photos into perfect images for feature extraction. A new method is proposed to learn high discriminative features for visual arts, which considers both the content information and style information in visual arts. Apart from conducting solid experiments, a real-world system is built to prove the effectiveness of the proposed methods. To the best of our knowledge, this is the first article to solve problems for visual arts search on mobile devices.

editorial Free Access Share on Introduction to the Special Issue on Big Data, Machine Learning, and AI Technologies for Art and Design Editor: James She Social Media Lab., Hong Kong Uni. of Science 8 Technology, H.K. Social Media Lab., Hong Kong Uni. of Science 8 Technology, H.K.View Profile Authors Info & Claims ACM Transactions on Multimedia Computing, Communications, and ApplicationsVolume 15Issue 2sApril 2019 Article No.: 57pp 1–3https://doi.org/10.1145/3338002Published:19 July 2019Publication History 1citation653DownloadsMetricsTotal Citations1Total Downloads653Last 12 Months169Last 6 weeks25 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF

Lately, Bluetooth Low Energy (BLE) beacon has attracted a lot of interests for its capabilities in enhancing the interaction between smart things in the Internet of Things (IoT) ecosystem via proximity approach. Even though Proximity sensing is capable of delivering a correct interaction, it might have a problem for explicit interaction when exact distance estimation is required. Considering those interactive applications which are distance-dependent, this paper proposed an optimized support vector machine (O-SVM) on the cloud for distance estimation and a Kalman filter (KF) on the edge to obtain a near true RSS value from a list of RSS measurements. Four benchmark functions (i.e., two from Industries and two Machine Learning Techniques) have been used for performance evaluation. Simulation with real signal samples was conducted to verify the performance of our proposed algorithm. Besides examining the performance gain of our proposed solution over the four benchmark functions, we also implemented the proposed solution on a smartphone for practical testing to demonstrate its feasibility. The proposed solution not only outperforms the rest with significant performance gain, i.e., > 50% error reduction compared to the benchmark functions. Furthermore, practical implementation verified that our proposed approach is able to return the estimate distance in less than 1s, such real-time response is desirable for many delay- sensitive applications.

A BLE beacon is a small electronic device that has recently been proposed as a building block to construct an infrastructure supporting emerging smart applications. However, due to its simple communication protocol architecture, which broadcasts a static payload, a BLE beacon-based infrastructure is vulnerable to different types of abuses and attacks, in particular free-riding and device spoofing. Many beacon manufacturers propose dynamically randomizing beacon advertisement packets at the device firmware level as a solution. However, this approach is difficult to implement for already deployed beacon nodes as it requires a firmware update on each device. To alleviate these drawbacks, a crowd-assisted architecture for securing BLE beacons is proposed in this paper. A detailed architecture is presented along with experimental results and an implementation to demonstrate its feasibility. It is found that the beacon ID can be changed by user's mobile phone within a 20 m range with probability of almost 100% under both stationary and mobile conditions.

While the Internet of Things (IoT) is driving a transformation of current society toward a smarter one, new challenges and opportunities have arisen to accommodate the demands of IoT development. Low power wireless devices are, undoubtedly, the most viable solution for diverse IoT use cases. Among such devices, Bluetooth low energy (BLE) beacons have emerged as one of the most promising due to the ubiquity of Bluetooth-compatible devices, such as iPhones and Android smartphones. However, for BLE beacons to continue penetrating the IoT ecosystem in a holistic manner, interdisciplinary research is needed to ensure seamless integration. This paper consolidates the information on the state-of-the-art BLE beacon, from its application and deployment cases, hardware requirements, and casing design to its software and protocol design, and it delivers a timely review of the related research challenges. In particular, the BLE beacon's cutting-edge applications, the interoperability between packet profiles, the reliability of its signal detection and distance estimation methods, the sustainability of its low energy, and its deployment constraints are discussed to identify research opportunities and directions.

Billions of user images, which are shared on social media, can be widely accessible by others due to their sharing nature. Using machine-generated labels to annotate those images is a reliable for user connections discovery on social networks. The machine-generated labels are obtained from encoded vectors using up-to-date image processing and computer vision techniques, such as convolution neural network. By analyzing 2 million user-shared images from 8 online social networks, a phenomenon is observed that the distribution of user similarity based on their shared images follows exponential functions. Users who share visually similar images are likely having follower/followee relationships, regardless of the origins and the content sharing mechanisms of a social network. This phenomenon is nicely formulated for a multimedia big data recommendation engine as an alternative to social graphs for recommendation. By utilizing the formulation of the distribution, it is proven the proposed engine can be 46 percent better than previous approaches in Fl score and achieves a comparable performance of friends-of-friends approach. To the best of our knowledge, this is the first attempt in related fields to characterize such phenomenon by massive user-shared images collected from real-world SNs, and then formulate into practical analytics engine for connection discovery.

The emergence of Bluetooth low energy (BLE) beacons has promoted the development of proximity-based service (PBS), which is a context-aware application delivered subject to the Proximity of Interest (PoI). Most commercial applications use the sequential proximity detection with a fixed scanning mechanism to identify the target PoI. Such sequential execution, though is able to produce reliable detection, suffers severe performance degradation especially when the number of deployed beacons in the vicinity increases. To understand the effects of dense deployment, we conduct an empirical investigation and derive the statistical properties of both received signal strength (RSS) and signal inter-arrival time. In light of the statistical insights, this paper proposes a high resolution proximity detection using an adaptive scanning mechanism fusion with a spontaneous Differential Evolution (AS+sDE). This novel approach enables the receiver to adapt its scanning duration conditioned on the deployment density and make an almost spontaneous detection in parallel with the scanning. The feasibility of the proposed approach is verified by both simulations and real-world implementations. For a density of$\leq 5\ beacons/m^2$,AS+sDEachieves a superior performance with a high accuracy rate, i.e., on average$<1s$is spent to guarantee at least 90 percent accuracy.

Presents corrections to the paper, “High resolution beacon-based proximity detection for dense deployment", (Ng, P.C., et al), IEEE Trans. Mobile Comput., vol. 17, no. 6, pp. 1369–1382, Jun. 2018.

The stage background is one of the most important features for a dance performance, as it helps to create the scene and atmosphere. In conventional dance performances, the background images are usually selected or designed by professional stage designers according to the theme and the style of the dance. In new media dance performances, the stage effects are usually generated by media editing software. Selecting or producing a dance background is quite challenging and is generally carried out by skilled technicians. The goal of the research reported in this article is to ease this process. Instead of searching for background images from the sea of available resources, dancers are recommended images that they are more likely to use. This work proposes the idea of a novel system to recommend images based on content-based social computing. The core part of the system is a probabilistic prediction model to predict a dancer’s interests in candidate images through social platforms. Different from traditional collaborative filtering or content-based models, the model proposed here effectively combines a dancer’s social behaviors (rating action, click action, etc.) with the visual content of images shared by the dancer using deep matrix factorization (DMF). With the help of such a system, dancers can select from the recommended images and set them as the backgrounds of their dance performances through a media editor. According to the experiment results, the proposed DMF model outperforms the previous methods, and when the dataset is very sparse, the proposed DMF model shows more significant results.

Existing interactive systems suffer from low user engagement due to their passiveness and steep learning curve. To address these issues, this paper presents an interactive framework, Notify-and-Interact, which leverages the Bluetooth low energy (BLE) beacon infrastructure to notify and a smart-phone to interact, such that it transforms a passive interactive system into an active one. The proposed framework is demonstrated in the Ping Yuan and Kinmay W Tang Gallery, where a series of wildlife artworks are exhibited. Engagement conversion rate is measured, and users' quality of experience (QoE) is surveyed through likert assessment. Artworks with Notify-and-Interact outperforms the QR code with a high engagement conversion rate at the interaction stage, i.e., 86% over 53%, and an average engagement time of 55.67s over 28.69s, respectively. The mean opinion score (MOS) shows that around 80% of the users expressed high satisfaction with the installed Notify-and-Interact framework in the gallery.

Modern recommender systems usually employ collaborative filtering with rating information to recommend items to users due to its successful performance. However, because of the drawbacks of collaborative-based methods such as sparsity, cold start, etc., more attention has been drawn to hybrid methods that consider both the rating and content information. Most of the previous works in this area cannot learn a good representation from content for recommendation task or consider only text modality of the content, thus their methods are very limited in current multimedia scenario. This paper proposes a Bayesian generative model called collaborative variational autoencoder (CVAE) that considers both rating and content for recommendation in multimedia scenario. The model learns deep latent representations from content data in an unsupervised manner and also learns implicit relationships between items and users from both content and rating. Unlike previous works with denoising criteria, the proposed CVAE learns a latent distribution for content in latent space instead of observation space through an inference network and can be easily extended to other multimedia modalities other than text. Experiments show that CVAE is able to significantly outperform the state-of-the-art recommendation methods with more robust performance.

In this paper, user log data of mobile notificationsare collected from a real mobile news application, and notificationopening rate and reaction time are identified as key parametersthat characterize user behavior. The results indicate that notificationopening patterns depend on the content of notifications, the time at which they are sent, the activeness of users, andthe reaction time. This paper also demonstrates that there existsan inverse relationship between the notification opening rate ofusers and their notification reaction time, and between theirnotification opening rate and notification opening volume. Then, a novel method of quantifying notification usefulness for users isdeveloped, where 3 models are presented to describe usefulnessof notifications against the volume of notifications received, eachrelevant in different contexts. Utilizing this usefulness model, a notification optimization framework is developed for sendingnotifications from different news categories. The optimized notificationallocation improved the average notification opening rateby by 65.24% from 4.92% to 8.13%, and the average reactiontime improved by 13.13% from 20.4 hours to 17.68 hours.

This paper aims to generate a better representation of visual arts, which plays a key role in visual arts analysis works. Museums and galleries have a large number of artworks in the database, hiring art experts to do analysis works (e.g., classification, annotation) is time consuming and expensive and the analytic results are not stable because the results highly depend on the experiences of art experts. The problem of generating better representation of visual arts is of great interests to us because of its application potentials and interesting research challenges---both content information and each unique style information within one artwork should be summarized and learned when generating the representation. For example, by studying a vast number of artworks, art experts summary and enhance the knowledge of unique characteristics of each visual arts to do visual arts analytic works, it is non-trivial for computer. In this paper, we present a unified framework, called DeepArt, to learn joint representations that can simultaneously capture contents and style of visual arts. This framework learns unique characteristics of visual arts directly from a large-scale visual arts dataset, it is more flexible and accurate than traditional handcraft approaches. We also introduce Art500k, a large-scale visual arts dataset containing over 500,000 artworks, which are annotated with detailed labels of artist, art movement, genre, etc. Extensive empirical studies and evaluations are reported based on our framework and Art500k and all those reports demonstrate the superiority of our framework and usefulness of Art500k. A practical system for visual arts retrieval and annotation is implemented based on our framework and dataset. Code, data and system are publicly available at http://deepart.ece.ust.hk.

"Drag A Star 3.0" is a site-specific interactive art which setup in a café umbrella context. Audiences were meant to sit down and relax under the umbrella while interacting with this piece. With their smart phone, audience able to generate their own unique design star and send it to the star field. Audience could even embed their star with a wish, just like the old myth of wishing upon a shooting star. Stars that being generated are stored in the web server database, it is then being retrieved and visually display as a star in the night sky. Thus, every star tells a story. Audience could catch the shooting star by just performing a simple dragging gesture and then able to read at others wishes, or even reply to the wishes. This art piece is a combination of different technologies which involve projection mapping technique display, mobile application, web-based messaging system and web server database. Every action by the users is stored in the web server database and all these actions would determine the visual component of the star field. These participatory social interactions between the audiences enable the connection of audiences from the past, present and future.

A proximity-based service (PBS) leverages the estimated proximity to provide users the accessibility to object or location restricted service. This paper exploits the interaction between Bluetooth Low Energy (BLE) Beacon and smartphone to set forth the fundamental building block of a beacon-based PBS system. In real-world scenarios, a beacon-based PBS system might suffer from sparse conditions when some beacons malfunction or beacons can only be deployed in a few specific positions. Motivated by such limitations, a similarity filter extended with compressive sampling matching pursuit (SF-CoSaMP) is proposed to ensure the reliability of proximity detection under such sparse conditions before smartphone proceed to retrieve the corresponding PBS. An extensive simulation with large volume of collected data has been conducted and the results prove the reliability of the proposed algorithm with high detection accuracy in an environment with sparse deployment.

With the advent and popularity of the social network, social graphs become essential to improve services and information relevance to users for many social media applications to predict follower/followee relationship, community membership, and so on. However, the social graphs could be hidden by users due to privacy concerns or kept by social media. Recently, connections discovered from user-shared images using machine-generated labels are proved to be more accessible alternatives to social graphs. But real-time discovery is difficult due to high complexity, and many applications are not possible. This article proposes an efficient computation framework for connection discovery using user-shared images, which is suitable for any image processing and computer vision techniques for connection discovery on the fly. The framework includes the architecture of online computation to facilitate real-time processing, offline computation for a complete processing, and online/offline communication. The proposed framework is implemented to demonstrate its effectiveness by speeding up connection discovery through user-shared images. By studying 300K+ user-shared images from two popular social networks, it is proven that the proposed computation framework reduces 90% of runtime with a comparable accurate with existing frameworks.

Many social media applications, such as recommendation, virality prediction, and marketing, make use of user gender, which may not be explicitly specified or kept privately. Meanwhile, advanced mobile devices have become part of our lives and a huge amount of content is being generated by users every day, especially user shared images shared by individuals in social networks. This particular form of user generated content is widely accessible to others due to the sharing nature. When user gender is only accessible to exclusive parties, these user shared images are proved to be an easier way to identify user gender. This work investigated 3,152,344 images by 7,450 users from Fotolog and Flickr, two image-oriented social networks. It is observed that users who share visually similar images are more likely to have the same gender. A multimedia big data system that utilizes this phenomenon is proposed for user gender identification with 79% accuracy. These findings are useful for information or services in any social network with intensive image sharing.

With the advances in mobile devices and the popularity of social networks, users can share multimedia content anytime, anywhere. One of the most important types of emerging content is video, which is commonly shared on platforms such as Instagram and Facebook. User connections, which indicate whether two users are follower/followee or have the same interests, are essential to improve services and information relevant to users for many social media applications. But they are normally hidden due to users’ privacy concerns or are kept confidential by social media sites. Using user-shared content is an alternative way to discover user connections. This article proposes to use user-shared videos for connection discovery with the Bag of Feature Tagging method and proposes a distributed streaming computation framework to facilitate the analytics. Exploiting the uniqueness of shared videos, the proposed framework is divided into Streaming processing and Online and Offline Computation. With experiments using a dataset from Twitter, it has been proved that the proposed method using user-shared videos for connection discovery is feasible. And the proposed computation framework significantly accelerates the analytics, reducing the processing time to only 32% for follower/followee recommendation. It has also been proved that comparable performance can be achieved with only partial data for each video and leads to more efficient computation.

The meeting of pervasive screens and smart devices has witnessed the birth of screen-smart device interaction (SSI), a key enabler to many novel interactive use cases. Most current surveys focus on direct human-screen interaction, and to the best of our knowledge, none have studied state-of-the-art SSI. This survey identifies three core elements of SSI and delivers a timely discussion on SSI oriented around the screen, the smart device, and the interaction modality. Two evaluation metrics (i.e., interaction latency and accuracy) have been adopted and refined to match the evaluation criterion of SSI. The bottlenecks that hinder the further advancement of the current SSI in connection with this metrics are studied. Last, future research challenges and opportunities are highlighted in the hope of inspiring continuous research efforts to realize the next generation of SSI.

Social graphs, representing online friendships among users, are one of the fundamental types of data for many applications, such as recommendation, virality prediction and marketing in social media. However, this data may be unavailable due to the privacy concerns of users, or kept private by social network operators, which makes such applications difficult. Inferring users' interests and discovering users' connections through their shared multimedia content has attracted more and more attention in recent years. This paper proposes a Gaussian relational topic model for connection discovery using user shared images in social media. The proposed model not only models users' interests as latent variables through their shared images, but also considers the connections between users as a result of their shared images. It explicitly relates user shared images to user connections in a hierarchical, systematic and supervisory way and provides an end-to-end solution for the problem. This paper also derives efficient variational inference and learning algorithms for the posterior of the latent variables and model parameters. It is demonstrated through experiments with over 200k images from Flickr that the proposed method significantly outperforms the methods in previous works.

The paper studies the scenario of wireless multicast with a single transmitter and a relay that jointly deliver successively refined (or multi-resolution) sources to multiple receivers. By taking the end-to-end mean square error distortion (EED) as the performance metric, the problems of power allocation at the transmitter and relay are formulated. Due to nonlinearity of the formulations, a generalized programming algorithm is developed to obtain near optimal solutions. Case studies are conducted to verify the proposed formulations and solution approaches. The results show the advantages of using a relay assisted multiresolution approach.

The paper investigates the scenario of wireless multicast with a single transmitter and multiple relays that jointly deliver successively refined (or multi-resolution) sources to multiple receivers. An asynchronous cooperative joint source-channel coding (JSCC) protocol is proposed, which can mitigate the complexity and difficulty in signal synchronization under the wireless multicast with cooperative relays. By considering the end-to-end mean square error distortion (EED), the problems of JSCC multicast are formulated to minimize the total power consumption by jointly selecting proper relays and the power allocations at the base station as well as the selected relays. To solve the formulated problem that is nonetheless nonlinear in nature, a two-step iterative algorithm is investigated for power allocations at all transmitters. Case studies are conducted to demonstrate the advantages of using the proposed JSCC in a relay-assisted multi-resolution wireless multicast network.

Informative directories have always responded to a fundamental need of humanity: providing available information around people. However, the escalating amount of content to be visualized on directories makes relevant information search extremely time-consuming. Meanwhile, digital displays based on screen-smart device interaction become an emerging interface of smart services to deal with daily-life challenges like information seeking. Also, multimedia content, such as movies, can be understood by multimedia analytics for recommendation, but there is no effective way to visualize the content of a directory. This paper proposes a novel directory visualization framework, called analytics-driven dynamic visualization on digital directory (AVDD): understanding user preferences via smartphone-based interaction and optimizing visualization by visual analytics in terms of high content relevancy and screen utilization for advanced directories. With experiments in laboratory and real-world settings, AVDD is proven to be effective for visualizing directory with screen utilization over 98% and the score for Likert-scale surveys achieving 73% on average in a movie directory.

User-shared images are shared on social media about a user’s life and interests that are widely accessible to others due to their sharing nature. Unlike for online profiles and social graphs, most users are unaware of the privacy risks relating to shared images, as they do not directly disclose characteristics such as gender and origin. Recently, however, user-shared images have been proven to be an accessible alternative to social graphs for online friendship recommendation and gender identification. This article evaluates 1.6M user-shared images from an image-oriented social network, Fotolog, and concludes how they can create privacy risks by proposing a system for de-anonymization, as well as inferring information on online profiles with the user-shared images. It is concluded that given user-shared images, using social graphs is 2 and 2.5 times more effective in de-anonymization than using origins or genders. With two showcases, it is also proven that using user-shared images is effective in online friendship recommendation, gender identification, and origin inference. To the best of our knowledge, this is the first article to evaluate the privacy issue qualitatively with big multimedia data from a real social network.

Predicting content going viral in social networks is attractive for viral marketing, advertisement, entertainment, and other applications, but it remains a challenge in the big data era today. Previous works mainly focus on predicting the possible popularity of content rather than the timing of reaching such popularity. This work proposes a novel yet practical iterative algorithm to predict virality timing, in which the correlation between the timing and growth of content popularity is captured by using its own big data naturally generated from users’ sharing. Such data is not only able to correlate the dynamics and associated timings in social cascades of viral content but also can be useful to self-correct the predicted timing against the actual timing of the virality in each iterative prediction. The proposed prediction algorithm is verified by datasets from two popular social networks—Twitter and Digg—as well as two synthesized datasets with extreme network densities and infection rates. With about 50% of the required content virality data available (i.e., halfway before reaching its actual virality timing), the error of the predicted timing is proven to be bounded within a 40% deviation from the actual timing. To the best of our knowledge, this is the first work that predicts content virality timing iteratively by capturing social cascades dynamics.

This paper investigates the scenario of wireless multicast with a single transmitter and multiple relays that jointly deliver successively refined (or multiresolution) sources to multiple receivers. An asynchronous cooperative joint source-channel coding (JSCC) protocol is proposed, aimed at mitigating the complexity and difficulty in signal synchronization in multicast. Taking end-to-end mean square error distortion (EED) as the quality of service (QoS) measure, the problems of JSCC transmission are formulated to minimize the total power consumption where both relay selection and power allocation at the base station (BS) and all the relays are jointly determined. To solve the formulated problem that is nonetheless nonlinear in nature, a two-step iterative algorithm is investigated for power allocations at all transmitters, where a sequential quadratic programming method is developed to find a strict local minimum. To reduce the computation complexity, a heuristic algorithm for relay selection is presented. Case studies are conducted to verify the proposed formulations and solution methods. We will demonstrate the advantages of using the proposed relay-assisted multiresolution approach.

"Drag A Star" is an interactive installation artwork that gives audiences an immersive and stunning interactive experience to remember the myth of making wishes upon a shooting star. Through the interactions with the display, audiences can learn about meteorites from outer space based on scientific and artistic perspectives by catching a shooting star with their smartphones. Audiences can send their wishes to a shooting star through their smartphones, while being able to read and reply the wishes from others at the same time. The piece was created based on the latest technologies of digital display, screen smart-device interactions, mobile applications, and web-based messaging systems. Extensive scientific, artistic and design efforts were integrated to create these cyber-physical interactive experiences between shooting stars and audiences. The artistic statement of this installation, akin to many ancient myths about wishing upon shooting stars, is about the possibility of catching a shooting star physically through technologies, and realizing someone's wish after reading them. Hence the existence of shooting stars could likely be the social media in outer space - a world where a connection is made between other beings in the Universe.

Billions of user-shared images are generated by individuals in many social networks today, and this particular form of user data is widely accessible to others due to the nature of online social sharing. When user social graphs are only accessible to exclusive parties, these user-shared images are proved to be an easier and effective alternative to discover user connections. This work investigated over 360 000 user shared images from two social networks, Skyrock and 163 Weibo, in which 3 million follower/ followee relationships are involved. It is observed that the shared images from users with a follower / followee relationship show relatively higher similarities . A multimedia big data system that utilizes this observed phenomenon is proposed as an alternative to user- generated tags and social graphs for follower/followee recommendation and gender identification. To the best of our knowledge, this is the first attempt in this field to prove and formulate such a phenomenon for mass user-shared images along with more practical prediction methods. These findings are useful for information or services recommendations in any social network with intensive image sharing, as well as for other interesting personalization applications, particularly when there is no access to those exclusive user social graphs.

editorial Free Access Share on Introduction to: Special Issue on Smartphone-Based Interactive Technologies, Systems, and Applications Editors: James She Hong Kong University of Science & Technology, Hong Kong Hong Kong University of Science & Technology, Hong KongView Profile , Alvin Chin BMW Group, United States BMW Group, United StatesView Profile , Feng Xia Dalian University of Technology, China Dalian University of Technology, ChinaView Profile , Jon Crowcroft University of Cambridge, UK University of Cambridge, UKView Profile Authors Info & Claims ACM Transactions on Multimedia Computing, Communications, and ApplicationsVolume 12Issue 1sArticle No.: 11pp 1–4https://doi.org/10.1145/2820398Published:21 October 2015Publication History 4citation201DownloadsMetricsTotal Citations4Total Downloads201Last 12 Months18Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

This paper conducts a summarized reality check of a P2P-based IPTV system from the operator's perspective (NextTV) through the analysis of large-scale operational data. NextTV is an IPTV system by a leading Asia media company based in Taiwan — NextMedia, in which P2P delivery mechanism is adopted to deal with system and network scalability issues for popular videos. This paper contributes by revealing 1) the relationship between the user properties and video popularity and 2) how effective the P2P mechanism is in the IPTV system for popular videos with different user properties. The results demonstrate a successful showcase of using the P2P delivery mechanism for IPTV services by analyzing the real operational data of all users. The approach is different from many previous works which has used simulations, test-beds and traffic measurements of an incomplete collection of users. The results potentially helps the design of the next generation of the P2P delivery mechanism for IPTV services.

The cyber-physical directory is proposed previously to enhance the effectiveness of current digital directories, which used the tag-cloud representation to make relevant information obviously bigger to users due to their interests. Such customized visualizations can speed up the information search for user social activities at a location. Unfortunately, such visualization poorly utilizes the total area of a directory display by leaving a lot of unused areas. This paper introduces a 3-step optimized visualization framework that solves this problem while preserving the advantages of tag-cloud representations. The proposed framework is successfully implemented for its feasibility, and is proved its high utilization and practicality to users in a real scenario.

The trend of replacing public static signages with digital displays creates opportunities for interactive display systems, which can be used in collaborative workspaces, social gaming platforms and advertising. Based on marketing communication concepts and existing models for consumer behavior, three stages, namely attraction, interaction and conation, are defined in this article to analyze the effectiveness of interactive display advertising. By reviewing various methods and strategies employed by existing systems with attraction, interaction and conation stages, this article concludes that smart mobile devices should be integrated as a component to increase the effectiveness of interactive displays as advertising tools. Future research challenges related to this topic are also discussed.

Digital displays have been extensively used at various indoor and outdoor venues to provide updated information to the audiences. With the pervasiveness of the smartphones nowadays, it is possible to utilize smartphones to design new interactive display systems. Although many such systems have been proposed, none of them address the issue of supporting interaction with multiple displays. A new solution is proposed in this paper to allow users to use smartphone sensors to intuitively interact with multiple displays at one venue. This approach is based on azimuth angle measured by the smartphone's orientation sensor, and situation abstraction which eliminates the use of location information. Each display at the venue is optimally assigned with an azimuth angle range. A smartphone constantly measures its azimuth angle. If it is within the azimuth angle ranges of a particular display, the smartphones presumes the user is interested to interact with this display. As a result, a user can interact with the desired display intuitively by pointing his/her smartphone at it, and switch smoothly from one display to another by changing smartphone's orientation.

This paper demonstrates effective strategies for reducing the end-to-end distortion in a decode-and-forward relay network, where a coded wireless multicast/broadcast system utilizes a successively refinable source along with superposition coding (SPC) at the channel to provision large-scale multimedia services. Exploiting the nature of the successive refinement of source information, where enhancement layer data refines the coarse-resolution base layer into full resolution, the proposed framework relays successive refinable information with reduced distortions through a few proposed strategies. Promising results are justified with a practical measure of perceived receiver quality known as end-to-end distortion (EED). Formulations are derived to investigate the impact to distortion when subject to varying fading channel conditions when receivers utilize successive-interference cancellation (SIC) in decoding the SPC broadcast channel. It is concluded that the relaying of solely the enhancement layer information is a more effective way to reduce distortion among other strategies, which has neither been apparent nor discovered from any previous literature.

Multi-level superposition coded modulation (SCM) is a scalable technique for wireless video broadcast/ multicast, in which iterative turbo structures provide receivers with multi-resolution demodulations subject to a high complexity. Forward error correction using low-density parity-check (LDPC) code is helpful for better received video quality but further increasing the receiver complexity. In this paper, a method is proposed to reduce the receiver complexity by using a sequential structure with a faster convergence for demodulation. In addition, the iterative demodulator and the LDPC decoder are jointly designed as a multi-loop iterative structure to reduce the decoding complexity. Experimental results show that up to 67% decoding complexity is reduced and better video quality is achieved at receivers under low signal-to-noise ratios.

Emerging 4G standards, such as WiMAX, LTE, and TD-SCDMA, have adopted the proven technique of Adaptive Modulation and Coding (AMC) to dynamically react to channel fluctuations while maintaining bit- error rate targets of the transmission. To mitigate the vicious effects due to stale channel state indication (CSI) problem, this paper introduces a novel framework by incorporating AMC with layered transmission through Superposition Coding (SPC). A Markov chain model is adopted under this framework, to effectively assist the system in selecting the optimal modulation and coding scheme for each layer in every multi-resolution unicast transmission. Extensive simulation is conducted to verify the proposed framework and compare it with a number of counterparts. The results demonstrate that the proposed framework can achieve a much better spectrum efficiency due to improved robustness by addressing the stale CSI problem at each multi-resolution modulated transmission.

This paper introduces a cross-layer design framework for robust and efficient video multicasting over IEEE 802.16 (also known as WiMAX) networks in metropolitan areas. In the framework, multiple description coding (MDC) on scalable video bitstreams at the source for achieving multiresolution robustness is jointly designed with superposition coding (SCM) on multicast signals at the channel to overcome multiuser channel diversity in wireless multicast. The coded multicast signals under the proposed framework can cope with multiuser channel diversity and mitigate the impact due to short-term channel fluctuations, which are the two most challenging issues in achieving robust and efficient video multicasting in metropolitan areas. We formulate the proposed framework and analyze its video quality performance in terms of the total receivable/ recoverable bitstreams by a receiver. A heuristic methodology is developed for system parameter selection and performance optimization that can be applied to practical scenarios of video multicasting for IPTV services in WiMAX. Simulation is conducted based on actual standard video sequences to verify the proposed methodology on parameter selection and performance optimization. Performance gains of the proposed cross-layer design framework in the presence of fading channel diversity are demonstrated.

This paper proposes an efficient yet simple design framework for achieving flexible resource allocation and packet scheduling for non-real-time polling service (nrtPS) traffic in IEEE 802.16 networks. By jointly considering the selective automatic repeat request mechanism at the media access control layer as well as the adaptive modulation and coding technique at the physical layer, the proposed framework enables a graceful tradeoff between resource utilization and packet delivery delay while maintaining the minimum throughput requirements of nrtPS applications. An analytical model is developed for parameter manipulation in the proposed framework, where some important performance metrics, such as inter-service time, delivery delay, goodput, and resource utilization, are investigated for performance evaluation. Simulation results are given to demonstrate the efficiency of the proposed framework and verify the accuracy of the analytical model.

Exploiting the broadcast nature of wireless communications, multicast transmission is an efficient way to improve the network throughput by transmitting the same contents to multiple receivers simultaneously. It has been considered as a key technology for supporting emerging services in next-generation IEEE 802.16 based wireless metropolitan area networks (WMANs), such as Internet Protocol TV (IPTV) and mobile TV. Therefore, it is critical to devise efficient multicast scheduling schemes to support these multimedia services. In this paper, we propose a novel multicast scheduling scheme, using downlink cooperative transmission for achieving high throughput not only for all multicast groups but also for each group member. Extensive simulations are conducted to demonstrate the effectiveness and efficiency of the proposed scheme.

In IEEE 802.16 networks, a subscriber station (SS) could be a single mobile user, a residence house, or an office building providing Internet service for multiple customers. Considering the heterogeneity among SSs which have different traffic load/demands, in the paper, we introduce the weighted proportional fair (WPF) scheduling scheme for best effort (BE) service in IEEE 802.16 networks to achieve the flexible and efficient resource allocation. Furthermore, an analytical model is developed to investigate the performance of WPF in terms of spectral efficiency, throughput, resource utilization, and fairness. Extensive simulations are conducted to illustrate the efficiency of the proposed scheme and verify the accuracy of the analytical model.

A cross-layer design (CLD) framework for robust and efficient video multicasting over IEEE 802.16 (or WiMAX) is introduced. In the framework, multiple description coding on scalable video bitstreams at the source for achieving multi-resolution robustness is jointly designed with superposition coding (i.e., multi-resolution modulation) on multicast signals at the channel to overcome the channel diversity problem in wireless multicast. The resulting cross-layer coded multicast signals enable us to recover some lost bitstreams in high quality layers, which is not possible if multi-resolution modulation is used alone for multicasting as in previous works. Simulation results show that indeed our joint design outperforms the scheme using only superposition coded multicast by achieving better video quality for users under multi-user channel diversity.

In this paper, we propose an application-driven MAC-layer buffer management framework based on a novel active dropping (AD) mechanism for real-time video streaming in IEEE 802.16 Point-to-Multi-Point (PMP) networks. The basic idea of the proposed approach is that the MAC-layer protocol data units (MPDUs) of a video stream could be actively dropped at the Base Station (BS) if the corresponding frame is not with a sufficient confidence to be successfully delivered to the recipient within its application-layer delay bound. In contrast to the conventional cross-layer techniques that manipulate transmission and/or retransmission priorities for sending MPDUs of a single stream, the proposed AD mechanism can be more effectively bound the delay of each video frame and release precious transmission resources for the subsequent frames or the frames of the other competing streams. This is considered as an intelligent approach for minimizing delay propagation due to bad channels or any other possible reason. A comprehensive analytical model is formulated on deriving how confident a frame can be effectively delivered within its application-layer delay bound by jointly considering the effect of playback buffering. Extensive simulation is performed to demonstrate the effectiveness of the proposed scheme. We expect that the proposed application-driven MAC-layer buffer management can incorporate with the emerging cross-layer design paradigm for real-time video streaming in TDMA-based wireless broadband access networks such as IEEE 802.16.

As a promising broadband wireless access standard, IEEE 802.16 specified some advance physical layer techniques and media access control layer protocols, which pose many fundamental differences in terms of automatic repeat request (ARQ) mechanism, scheduling scheme, and resource allocation, compared with those done in many previous works. In this paper, we analyze the performance of ARQ in IEEE 802.16 networks by jointly considering the opportunistic scheduling scheme, where the delivery delay and goodput are investigated as two performance metrics. Simulation results are given to verify the proposed analysis model.