Detecting and segmenting cell nuclei in Hematoxylin and Eosin (H&E) stained tissue images is a critical clinical task with broad applications. However, it is a challenging problem due to variations in staining and size, overlapping boundaries, cell clustering, and the high morphological and size variability of lesion regions in medical images. Accurate segmentation in medical imaging requires precise global contour localization and careful handling of local boundaries. Existing CNN-based and Transformer-based models are often limited by high parameter counts and computational complexity, making it difficult to effectively integrate these features. To address this challenge, we propose a spatially feature-aware segmentation and classification model for cell nuclei, named SpaceSeg. The Partial Gated Feed-forward Network module in SpaceSeg enhances feature representations, enabling the model to focus on key features while reducing attention to redundant information. The Spatial Attention Block enhances the spatial information of features passed to the decoder, allowing the network to automatically focus on the most critical regions in the image, especially those related to nuclear boundaries. The Partial Gated CNN module more accurately fuses low-level and high-level features, helping the model learn finer semantic information. Experimental results on the PanNuke and MoNuSeg datasets demonstrate that SpaceSeg outperforms all existing state-of-the-art models.

Lung cancer remains a leading cause of cancer mortality globally. Recently, targeted therapies have significantly improved clinical outcomes in lung cancer patients, making accurate identification of driver gene mutations crucial for precision treatment. Artificial intelligence approaches leveraging routinely acquired whole-slide histopathology images (WSIs) offer a promising and cost-effective means for molecular biomarker prediction, potentially enhancing clinical decision-making. However, mutation prediction from WSIs faces substantial technical challenges, including high data dimensionality, weakly supervised labels, and severe class imbalance inherent in clinical datasets. To address these issues, we propose GEMIL, a novel multipleinstance learning (MIL) model. GEMIL features a hierarchical attention-based encoder that employs deep non-linear projections and structured regularization to learn discriminative patch-level representations. These features are then aggregated by a querydriven decoder, which efficiently consolidates instance information into a robust slide-level prediction. We validated GEMIL through extensive experiments on the large-scale PathGene dataset, with further evaluation on an independent TCGA cohort. The results demonstrate that GEMIL consistently outperforms state-of-the-art MIL methods across multiple gene mutation prediction tasks (TP53, EGFR, etc.), improving the average accuracy by 2.8% and the F1-score by 3.1%. Consequently, GEMIL provides a robust and generalizable computational tool for WSI-based biomarker prediction, holding significant potential to advance precision oncology.

We present a self-supervised framework that learns population-level codes for arbitrary ensembles of neural recordings at scale. We address key challenges in scaling models with neural time-series data, namely, sparse and variable electrode distribution across subjects and datasets. The Population Transformer (PopT) stacks on top of pretrained temporal embeddings and enhances downstream decoding by enabling learned aggregation of multiple spatially-sparse data channels. The pretrained PopT lowers the amount of data required for downstream decoding experiments, while increasing accuracy, even on held-out subjects and tasks. Compared to end-to-end methods, this approach is computationally lightweight, while achieving similar or better decoding performance. We further show how our framework is generalizable to multiple time-series embeddings and neural data modalities. Beyond decoding, we interpret the pretrained and fine-tuned PopT models to show how they can be used to extract neuroscience insights from large amounts of data. We release our code as well as a pretrained PopT to enable off-the-shelf improvements in multi-channel intracranial data decoding and interpretability. Code is available at https://github.com/czlwang/PopulationTransformer.

We construct the first rigorously justified probabilistic algorithm for recovering the solution operator of a hyperbolic partial differential equation (PDE) in two variables from input-output training pairs. The primary challenge of recovering the solution operator of hyperbolic PDEs is the presence of characteristics, along which the associated Green's function is discontinuous. Therefore, a central component of our algorithm is a rank detection scheme that identifies the approximate location of the characteristics. By combining the randomized singular value decomposition with an adaptive hierarchical partition of the domain, we construct an approximant to the solution operator using $O(\Psi_\epsilon^{-1}\epsilon^{-7}\log(\Xi_\epsilon^{-1}\epsilon^{-1}))$ input-output pairs with relative error $O(\Xi_\epsilon^{-1}\epsilon)$ in the operator norm as $\epsilon\to0$, with high probability. Here, $\Psi_\epsilon$ represents the existence of degenerate singular values of the solution operator, and $\Xi_\epsilon$ measures the quality of the training data. Our assumptions on the regularity of the coefficients of the hyperbolic PDE are relatively weak given that hyperbolic PDEs do not have the "instantaneous smoothing effect" of elliptic and parabolic PDEs, and our recovery rate improves as the regularity of the coefficients increases. We also include numerical experiments which corroborate our theoretical findings.

We use (multi)modal deep neural networks (DNNs) to probe for sites of multimodal integration in the human brain by predicting stereoencephalography (SEEG) recordings taken while human subjects watched movies. We operationalize sites of multimodal integration as regions where a multimodal vision-language model predicts recordings better than unimodal language, unimodal vision, or linearly-integrated language-vision models. Our target DNN models span different architectures (e.g., convolutional networks and transformers) and multimodal training techniques (e.g., cross-attention and contrastive learning). As a key enabling step, we first demonstrate that trained vision and language models systematically outperform their randomly initialized counterparts in their ability to predict SEEG signals. We then compare unimodal and multimodal models against one another. Because our target DNN models often have different architectures, number of parameters, and training sets (possibly obscuring those differences attributable to integration), we carry out a controlled comparison of two models (SLIP and SimCLR), which keep all of these attributes the same aside from input modality. Using this approach, we identify a sizable number of neural sites (on average 141 out of 1090 total sites or 12.94%) and brain regions where multimodal integration seems to occur. Additionally, we find that among the variants of multimodal training techniques we assess, CLIP-style training is the best suited for downstream prediction of the neural activity in these sites.

When evaluating stimuli reconstruction results it is tempting to assume that higher fidelity text and image generation is due to an improved understanding of the brain or more powerful signal extraction from neural recordings. However, in practice, new reconstruction methods could improve performance for at least three other reasons: learning more about the distribution of stimuli, becoming better at reconstructing text or images in general, or exploiting weaknesses in current image and/or text evaluation metrics. Here we disentangle how much of the reconstruction is due to these other factors vs. productively using the neural recordings. We introduce BrainBits, a method that uses a bottleneck to quantify the amount of signal extracted from neural recordings that is actually necessary to reproduce a method's reconstruction fidelity. We find that it takes surprisingly little information from the brain to produce reconstructions with high fidelity. In these cases, it is clear that the priors of the methods' generative models are so powerful that the outputs they produce extrapolate far beyond the neural signal they decode. Given that reconstructing stimuli can be improved independently by either improving signal extraction from the brain or by building more powerful generative models, improving the latter may fool us into thinking we are improving the former. We propose that methods should report a method-specific random baseline, a reconstruction ceiling, and a curve of performance as a function of bottleneck size, with the ultimate goal of using more of the neural recordings.

We present the Brain Treebank, a large-scale dataset of electrophysiological neural responses, recorded from intracranial probes while 10 subjects watched one or more Hollywood movies. Subjects watched on average 2.6 Hollywood movies, for an average viewing time of 4.3 hours, and a total of 43 hours. The audio track for each movie was transcribed with manual corrections. Word onsets were manually annotated on spectrograms of the audio track for each movie. Each transcript was automatically parsed and manually corrected into the universal dependencies (UD) formalism, assigning a part of speech to every word and a dependency parse to every sentence. In total, subjects heard over 38,000 sentences (223,000 words), while they had on average 168 electrodes implanted. This is the largest dataset of intracranial recordings featuring grounded naturalistic language, one of the largest English UD treebanks in general, and one of only a few UD treebanks aligned to multimodal features. We hope that this dataset serves as a bridge between linguistic concepts, perception, and their neural representations. To that end, we present an analysis of which electrodes are sensitive to language features while also mapping out a rough time course of language processing across these electrodes. The Brain Treebank is available at https://BrainTreebank.dev/

The COVID-19 pandemic has contributed to un-precedented rates of unemployment and greater uncertainty in the job market. There is a growing need for data-driven tools and analyses to better inform the public on trends within the job market. In particular, obtaining a “snapshot” of available employment opportunities mid-pandemic promises insights to inform policy and support retraining programs. In this work, we combine data scraped from the Canadian Job Bank and Numbeo globally crowd-sourced repository to explore the relationship between job postings during a global pandemic and Key Performance Indicators (e.g. quality of life [QOL] index, cost of living) for major cities across Canada. This analysis aims to help Canadians make informed career decisions, collect a “snapshot” of the Canadian employment opportunities amid a pandemic, and inform job seekers in identifying the correct fit between the desired lifestyle of a city and their career. We collected a new high-quality dataset of job postings from jobbank.gc.ca obtained with the use of ethical web scraping and performed exploratory data analysis on this dataset to identify job opportunity trends. When optimizing for average salary of job openings with QOL, affordability, cost of living, and traffic indices, it was found that Edmonton, AB consistently scores higher than the mean, and is therefore an attractive place to move. Furthermore, we identified optimal provinces to relocate to with respect to individual skill levels. It was determined that Ajax, Marathon, and Chapleau, ON are each attractive cities for IT professionals, construction workers, and healthcare workers respectively when maximizing average salary. Finally, we publicly release our scraped dataset as a mid-pandemic snapshot of Canadian employment opportunities and present a public web application that provides an interactive visual interface that summarizes our findings for the general public and the broader research community.

We collect a large real-world test set, ObjectNet, for object recognition with controls where object backgrounds, rotations, and imaging viewpoints are random. Most scientific experiments have controls, confounds which are removed from the data, to ensure that subjects cannot perform a task by exploiting trivial correlations in the data. Historically, large machine learning and computer vision datasets have lacked such controls. This has resulted in models that must be fine-tuned for new datasets and perform better on datasets than in real-world applications. When tested on ObjectNet, object detectors show a 40-45% drop in performance, with respect to their performance on other benchmarks, due to the controls for biases. Controls make ObjectNet robust to fine-tuning showing only small performance increases. We develop a highly automated platform that enables gathering datasets with controls by crowdsourcing image capturing and annotation. ObjectNet is the same size as the ImageNet test set (50,000 images), and by design does not come paired with a training set in order to encourage generalization. The dataset is both easier than ImageNet (objects are largely centered and unoccluded) and harder (due to the controls). Although we focus on object recognition here, data with controls can be gathered at scale using automated tools throughout machine learning to generate datasets that exercise models in new ways thus providing valuable feedback to researchers. This work opens up new avenues for research in generalizable, robust, and more human-like computer vision and in creating datasets where results are predictive of real-world performance.