Communication is defined as "*Who* says *what* to *whom* with *what* effect." A message from a communicator generates downstream receiver effects, also known as behavior. Receiver behavior, being a downstream effect of the message, carries rich signals about it. Even after carrying signals about the message, the behavior signal is often ignored while training vision-language models. We show that training VLMs on receiver behavior can actually help improve their content-understanding abilities. We demonstrate that training VLMs to predict receiver behaviors, such as likes, comments, and replay graphs, which are available at scale, enhances the VLM's performance across a broad range of downstream content understanding tasks. We show this performance increase over 6 types of behavior, 46 different tasks covering image, video, text, and audio over 26 benchmark datasets across both zero-shot and fine-tuning settings, outperforming many supervised baselines on diverse tasks ranging from emotion recognition to captioning by up to 150%. We note that since receiver behavior, such as likes, comments, and replay graphs, is collected by default on the internet and does not need any human annotations to be useful, the performance improvement we get after training on this data is essentially free lunch. We also release **BLIFT**, our **Behaviour-LLaVA IFT** dataset comprising 730k images and videos with their receiver behavior collected from multiple platforms on which we train our models to achieve this. The dataset and code are available at [behavior-in-the-wild.github.io/behavior-llava](https://behavior-in-the-wild.github.io/behavior-llava).

Despite the importance of long-term memory in marketing and brand building, until now, there has been no large-scale study on the memorability of ads. All previous memorability studies have been conducted on short-term recall on specific content types like action videos. On the other hand, long-term memorability is crucial for advertising industry, and ads are almost always highly multimodal. Therefore, we release the first memorability dataset, LAMBDA, consisting of 1749 participants and 2205 ads covering 276 brands. Running statistical tests over different participant subpopulations and ad types, we find many interesting insights into what makes an ad memorable, e.g., fast-moving ads are more memorable than those with slower scenes; people who use ad-blockers remember a lower number of ads than those who don't. Next, we present a model, Henry, to predict the memorability of a content. Henry achieves state-of-the-art performance across all prominent literature memorability datasets. It shows strong generalization performance with better results in 0-shot on unseen datasets. Finally, with the intent of memorable ad generation, we present a scalable method to build a high-quality memorable ad generation model by leveraging automatically annotated data. Our approach, SEED (Self rEwarding mEmorability Modeling), starts with a language model trained on LAMBDA as seed data and progressively trains an LLM to generate more memorable ads. We show that the generated advertisements have 44% higher memorability scores than the original ads. We release this large-scale ad dataset, UltraLAMBDA, consisting of 5 million ads. Our code and the datasets, LAMBDA and UltraLAMBDA, are open-sourced at https://behavior-in-the-wild.github.io/memorability.

Invasive Brain Computer Interface (BCI) systems through Electrocorticographic (ECoG) signals require efficient recognition of spatiotemporal patterns from a multi-electrodes sensor array. Such signals are excellent candidates for automated pattern recognition through machine learning algorithms. The importance of these patterns can be highlighted through feature extraction techniques. However, the signal variability due to non-stationarity is ignored while extracting features, and which features to use can be challenging to figure out by visual inspection. In this study, we introduce the signal split parameter to account for the variability of the signal and increase the accuracy of the machine learning classifier. We use genetic selection, which allows the selection of the optimal combination of features from a pool of 8 different feature sets. Genetic selection of features increases accuracy and reduces the BCI prediction time. Along with Genetic selection, we also use a reduced signal length, which leads to a higher Information Transfer Rate. Thus this approach enables the design of a fast and accurate motor-related EcoG BCI system.

This work proposes a hybrid Brain-Computer Interface system for the automation of a wheelchair for the disabled. Herein a working prototype of a BCI-based wheelchair is detailed that can navigate inside a typical home environment with minimum structural modification and without any visual obstruction and discomfort to the user. The prototype is based on a combined mechanism of steady-state visually evoked potential and eye blinks. To elicit SSVEP, LEDs flickering at 13Hz and 15Hz were used to select the left and right direction, respectively, and EEG data was recorded. In addition, the occurrence of three continuous blinks was used as an indicator for stopping an ongoing action. The wavelet packet denoising method was applied, followed by feature extraction methods such as Wavelet Packet Decomposition and Canonical Correlation Analysis over narrowband reconstructed EEG signals. Bayesian optimization was used to obtain 5-fold cross-validations to optimize the hyperparameters of the Support Vector Machine. The resulting new model was tested and the average cross-validation accuracy 89.65% ± 6.6% (SD) and testing accuracy 83.53% ± 8.59% (SD) were obtained. The wheelchair was controlled by RaspberryPi through WiFi. The developed prototype demonstrated an average of 86.97% success rate for all trials with 4.015s for each command execution. The prototype can be used efficiently in a home environment without causing any discomfort to the user.

Brain Computer Interfaces (BCIs) are regarded as the potential method that bridges the gap between the human brain and the external world. Non-invasive electroencephalographic (EEG) signals are highly individualistic and hence show potential for effective biometric systems. The presented study analyzed the EEG signals for characteristic features elicited by movement and imagination of 4 different upper limb movements. The same limb movement imagery tasks were compared for performance and its validity in developing an effective multimodal biometric system for individuals with motor disabilities. The study involved 10 subjects executing imagined lifting of left and right hands and clenching left and right-hand fists. Along with imagined movement (Motor Imagery), data for actual limb movement was collected, and the performance was compared for both imaginary and actual movement. The proposed pipeline achieved less than 2% False acceptance rate for all the imaginary and actual actions. A novel multimodal approach combining different Motor Imagery (MI) actions was successfully implemented with 98.28% accuracy. Moreover, both imaginary and actual movements showed equally good capability for biometrics purposes suggesting the usability of the proposed biometrics system for people who lost motor abilities or people with poor motor imagery skills.

Drones have now found their way into diverse fields of application. Getting accustomed to the presence of these drones requires seamless integration into our lives and depends a lot on Human-Drone Interaction (HDI). This paper presents a novel approach to develop a non-invasive Brain-Computer Interface (BCI) for control of the quadcopters as an assistive device (AD) for those people suffering from neurodegenerative diseases or impaired mobility and have lost the ability to explore the world around them freely. Electroencephalography (EEG) signals of individuals are captured, and actions corresponding to the wearer's thoughts were classified and used to control a quadcopter. Our main contribution lies in modeling this problem as a Markovian process thus enabling us to maximize the accuracy of data post-classification especially since the data may still contain outliers due to the inability of the user to maintain a constant thought stream, sensor noise, or classification errors. We propose an algorithm that aims at achieving robust control by breaking the problem into sub-parts, which are- classification, outlier removal, maximum likelihood estimation, and autonomy, where each step is optimized individually. We also present a shared control algorithm incorporating visual feedback by three-dimensional reconstruction of the environment to augment user's decisions with autonomous obstacle avoidance. The entire algorithm of this BCI is built using the Robot Operating System (ROS) framework. Our results suggest that post-processing classified data improves accuracy and system reaction time with minimal detriment to computational efficiency.

Evaluation of descriptive type questions automatically is an important problem. In this paper, we have concentrated on programing language course which is a core subject in CS discipline. In this paper, we have given a tool prototype which evaluates the descriptive type of questions in computer programing course using the notion of program equivalence.

In this paper, we present a novel brain computer interface based home automation system using two responses - Steady State Visually Evoked Potential (SSVEP) and the eyeblink artifact, augmented by a Bluetooth based indoor localization system, to greatly increase the number of controllable devices. The hardware implementation of this system to control a table lamp and table fan using brain signals has also been discussed and state-of-the-art results have been achieved.