As a bio-inspired vision sensor, spike camera records light intensity by accumulating photons and firing a spike once a preset threshold is reached. For high-light regions, the accumulated photons may reach the threshold multiple times within a readout interval, while only one spike can be stored and read out, resulting in incorrect intensity representation and a limited dynamic range. Multi-level (ML) spike camera enhances the dynamic range by introducing a spike-firing counter (SFC) to count spikes within each readout interval for each pixel, and uses different spike symbols to represent the arrival of different amounts of photons. However, when the light intensity becomes even higher, each pixel requires an SFC with a higher bit depth, causing great cost to the manufacturing process. To address these issues, we propose time-encoding (TE) spike camera, which transforms the counting of spikes to recording of the time at which a specific number of spikes (i.e., an overflow) is reached. To encode time information with as few bits as possible, instead of directly utilising a timer, we leverage a periodic timing signal with a higher frequency than the readout signal. Then the recording of overflow moment can be transformed into recording the number of accumulated timing signal cycles until the overflow occurs. Additionally, we propose an image reconstruction scheme for TE spike camera, which leverages the multi-scale gradient features of spike data. This scheme includes a similarity-based pyramid alignment module to align spike streams across the temporal domain and a light intensity-based refinement module, which utilises the guidance of light intensity to fuse spatial features of the spike data. Experimental results demonstrate that TE spike camera effectively improves the dynamic range of spike camera.

The bitrate of a video bitstream directly determines whether it can be transmitted successfully under current network circumstances. Under extreme circumstances, environments may require the video to be transmitted at a bitrate lower than normal application scenarios. Encoding videos with a rescaling strategy is a possible way to solve this problem. In this paper, we developed a content-adaptive variable resolution framework for intra coding to solve the bitrate adaptation problem. Specifically, we proposed a double-line hyperbolic R-D model to characterize the relationship between bit per pixel (bpp) and mean square error (MSE) in wide bitrate range at different resolutions. Then, we employed convolutional neural network (CNN) to predict model parameters of different coding resolution from video content. Finally, we determined the coding resolution under current bandwidth condition. Without modifying the encoder or decoder, experiments on multiple videos with different contents show that our proposed framework can predict the encoding resolution with high accuracy. In addition, experimental results show that our proposed framework for variable resolution coding outperforms fixed down-sample ratio method in x265 intra coding, leading to on average 4.67% BD-rate reduction.

Recently, a novel retina-inspired camera, namely spike camera, has shown great potential for recording high-speed dynamic scenes. Unlike conventional digital cameras that compact the visual information within an exposure interval into a single snapshot, the spike camera continuously outputs binary spike streams to record the dynamic scenes, yielding a very high temporal resolution. Most of the existing reconstruction methods for spike camera focus on reconstructing images with the same resolution as spike camera. However, as a trade-off of high temporal resolution, the spatial resolution of spike camera is limited, resulting in inferior details of the reconstruction. To address this issue, we develop a spike camera super-resolution framework, aiming to super resolve high-resolution intensity images from the low-resolution binary spike streams. Due to the relative motion between the camera and the objects to capture, the spikes fired by the same sensor pixel no longer describes the same points in the external scene. In this paper, we exploit the relative motion and derive the relationship between light intensity and each spike, so as to recover the external scene with both high temporal and high spatial resolution. Experimental results demonstrate that the proposed method can reconstruct pleasant high-resolution images from low- resolution spike streams.

Recently, the SoftCast scheme has shown great potential for robust image/video communication in wireless scenarios, where the channel quality may fluctuate drastically and unpredictably. However, the analog-like transmission in Soft-Cast is not always efficient in terms of power usage, compared with digital approaches. In this paper, we propose a layered image/video SoftCast scheme, in which a coarse version of the image is transmitted by a base layer in digital way, while the residual details are delivered by an enhancement layer in pseudo-analog way. In order to achieve optimal overall transmission performance, this paper studies the problem of optimal bit rate selection for the base layer, by building a rate-residual model based on the relationship between the base layer bit rate and the residual signal spectrum. Experimental results show that the proposed scheme can improve the performance of original scheme remarkably, while still preserving the smooth quality degradation characteristic of SoftCast.