Distributed video coding (DVC) is a coding paradigm that entails low complexity encoding by exploiting the source statistics at the decoder. To improve the DVC coding efficiency, this paper presents a novel adaptive technique for mode decision to control and take advantage of skip mode and intra mode in DVC initially proposed by Luong et al. in 2013. The adaptive mode decision (AMD) is not only based on quality of key frames but also the rate of Wyner–Ziv (WZ) frames. To improve noise distribution estimation for a more accurate mode decision, a residual motion compensation is proposed to estimate a current noise residue based on a previously decoded frame. The experimental results, integrating AMD in two efficient DVC codecs, show that the proposed AMD DVC significantly improves the rate distortion performance without increasing the encoding complexity. For a GOP size of 2 on the set of six test sequences, the average (Bjøntegaard) bitrate saving of the proposed codec is 35.5% on WZ frames compared with the DISCOVER codec. This saving is mainly achieved by AMD.

Standard techniques in matrix factorization (MF) – a popular method for latent factor model-based design – result in dense matrices for both users and items. Users are likely to have some affinity toward all the latent factors – making a dense matrix plausible, but it is not possible for the items to possess all the latent factors simultaneously; hence it is more likely to be sparse. Therefore, we propose to factor the rating matrix into a dense user matrix and a sparse item matrix, leading to the blind compressed sensing (BCS) framework. To further enhance the prediction quality of our design, we aim to incorporate user and item metadata into the BCS framework. The additional information helps in reducing the underdetermined nature of the problem of rating prediction caused by extreme sparsity of the rating dataset. Our design is based on the belief that users sharing similar demographic profile have similar preferences and thus can be described by the similar latent factor vectors. We also use item metadata (genre information) to group together the similar items. We modify our BCS formulation to include item metadata under the assumption that items belonging to common genre share similar sparsity pattern. We also design an efficient algorithm to solve our formulation. Extensive experimentation conducted on the movielens dataset validates our claim that our modified MF framework utilizing auxiliary information improves upon the existing state-of-the-art techniques.

The paper addresses a robust wavelet-based speech enhancement for automatic speech recognition in reverberant and noisy conditions. We propose a novel scheme in improving the speech, late reflection, and noise power estimates from the observed contaminated signal. The improved estimates are used to calculate the Wiener gain in filtering the late reflections and additive noise. In the proposed scheme, optimization of the wavelet family and its parameters is conducted using an acoustic model (AM). In the offline mode, the optimal wavelet family is selected separately for the speech, late reflections, and background noise based on the AM likelihood. Then, the parameters of the selected wavelet family are optimized specifically for each signal subspace. As a result we can use a wavelet sensitive to the speech, late reflection, and the additive noise, which can independently and accurately estimate these signals directly from an observed contaminated signal. For speech recognition, the most suitable wavelet is identified from the pre-stored wavelets, and wavelet-domain filtering is conducted to the noisy and reverberant speech signal. Experimental evaluations using real reverberant data demonstrate the effectiveness and robustness of the proposed method.

We propose a new paradigm for no-reference image quality assessment (IQA) exploiting neurological and psychophysical properties of the human visual system (HVS)1. Physiological and psychological evidences exist that HVS has different behavioral patterns under low and high noise/artifact levels. In this paper, we propose a dual-model approach for blind IQA under near-threshold and suprathreshold noise conditions. The underlying assumption for the proposed dual-model approach is that for images with low-level near-threshold noise, HVS tries to gauge the strength of the noise, so image quality can be well approximated via measuring strength of the noise. On the other hand, for images with structures overwhelmed by high-level suprathreshold noise, perceptual quality assessment relies on a cognitive model and the HVS tries to recover meaningful contents from the noisy pixels using past experiences and prior knowledge encoded into an internal generative model of the brain. And image quality is therefore closely related to the agreement between the noisy observation and the internal generative model explainable part of the image. Specifically, under near-threshold noise condition, a noise level estimation algorithm using natural image statistics is used, while under suprathreshold condition, an active inference model based on the free-energy principle is adopted. The near- and suprathreshold models can be seamlessly integrated through a mathematical transformation between estimates from both models. The proposed dual-model algorithm has been tested on additive Gaussian noise contaminated images. Experimental results and comparative studies suggest that although being a no-reference approach, the proposed algorithm has prediction accuracy comparable with some of the best full-reference IQA methods. The dual-model perspective of IQA presented in this paper is expected to facilitate future research in the field of visual perceptual modeling.

Concerns with global warming prompted many governments to mandate increased proportion of electricity generation from renewable sources. This, together with the desire to have more efficient and secure power generation and distribution, has driven research in the next-generation power grid, namely, the smart grid. Through integrating advanced information and communication technologies with power electronic and electric power technologies, smart grid will be highly reliable, efficient, and environmental-friendly. A key component of smart grid is the communication system. This paper explores the design goals and functions of the smart grid communication system, followed by an in-depth investigation on the communication requirements. Discussions on some of the recent developments related to smart grid communication systems are also introduced.

In some security applications, it is important to transmit just enough information to take the right decisions. Traditional video codecs try to maximize the global quality, irrespective of the video content pertinence for certain tasks. To better maintain the semantics of the scene, some approaches allocate more bitrate to the salient information. In this paper, a semantic video compression scheme based on seam carving is proposed. The idea is to suppress non-salient parts of the video by seam carving. The reduced sequence is encoded with H.264/AVC while the seams are encoded with our approach. The main contributions of this paper are (1) an algorithm that segments the sequence into group of pictures, depending on the content, (2) a spatio-temporal seam clustering method, (3) an isolated seam discarding technique, improving the seam encoding, (4) a new seam modeling, avoiding geometric distortion and resulting in a better control of the seam shapes, and (5) a new encoder which reduces the overall bit-rate. A full reference object-oriented quality metric is used to assess the performance of the approach. Our approach outperforms traditional H.264/AVC intra encoding with a Bjontegaard's rate improvement between 7.02 and 21.77% while maintaining the quality of the salient objects.

The enormous economic loss caused by power quality problems (more than $ 150 billion per year in USA) makes power quality monitoring an important component in power grid. With highly connected fragile digital equipment and appliances, Smart Grid has more stringent timeliness and reliability requirements on power quality monitoring. In this work, we propose a change-point detection theory-based power quality monitoring scheme to detect the most detrimental power quality events, such as voltage sags, transients and swells in a quick and reliable manner. We first present a method for single-sensor detection scenario. Based on that, we extend the scheme to multi-sensor joint detection scheme which further enhances the detection performance. A group of conventional power quality monitoring schemes (i.e. Root-mean-square, Short-time Fourier transform, MUSIC, and MBQCUSUM) are compared with the proposed scheme. Experimental results assert the superior of the proposed scheme in terms of detection latency and robustness.

We are living through an historic era in computing. As the price of data storage and processing continues to plummet, we are moving closer to a world where exhaustive search makes sense for certain types of intelligent systems. Signal and image processing are two related domains that benefit from this ubiquity of data storage and computing power. In this paper, a new, more collaborative, approach to solving signal and image processing tasks is built from the ground up to take into account the reality of this new age of data and computing superfluity. Starting with the mature field of ensemble methods and moving to the more-recently introduced field of meta-algorithmics, systems can be designed which are by nature to specifically incorporate new machine-learning technologies. These are more robust, more accurate, more adaptive, and ultimately less costly to build and maintain than the traditional machine-learning approaches. Applications to image and signal processing will then be discussed. Combined, these examples illustrate a new meta-architectural approach to the creation of machine intelligence systems.

The recent successes of over-the-top (OTT) video services have intensified the competition between the traditional broadcasting video and OTT video. Such competition has pushed the traditional video service providers to accelerate the transition of their video services from the broadcasting video to the carrier-grade IP video streaming. However, there are significant challenges in providing large-scale carrier-grade IP video streaming services. For a compressed video sequence, central to the guaranteed real-time delivery are the issues of video rate, buffering, and timing as compressed video pictures are transmitted over an IP network from the encoder output to the decoder input. Toward the understanding and eventual resolution of these issues, a mathematical theory of compressed video buffering is developed to address IP video traffic regulation for the end-to-end video network quality of service. In particular, a comprehensive set of theoretical relationships is established for decoder buffer size, network transmission rate, network delay and jitter, and video source characteristics. As an example, the theory is applied to measure and compare the burstiness and delay of video streams coded with MPEG-2, advanced video coding, and high-efficiency video coding standards. The applicability of the theory to IP networks that consist of a specific class of routers is also demonstrated.

Technologies for coding non-camera-captured video contents have received great interests lately due to the rapid growth of application areas such as wireless display and screen sharing, etc. In response to the market demands, the ITU-T Video Coding Expert Group and ISO/IEC Motion Picture Expert Group have jointly launched a new standardization project, i.e. the High-Efficiency Video Coding (HEVC) extensions on screen content coding (HEVC SCC). Several new video coding tools, including intra block copy, palette coding, adaptive color transform, and adaptive motion resolution, have been developed and adopted into HEVC SCC draft standard. This paper reviews the main features and coding technologies in the current HEVC SCC draft standard, with discussions about the performance and complexity aspects compared with prior arts.

In this paper, we propose a tracking algorithm to detect power lines from millimeter-wave radar video. We propose a general framework of cascaded particle filters which can naturally capture the temporal correlation of the power line objects, and the power-line-specific feature is embedded into the conditional likelihood measurement process of the particle filter. Because of the fusion of multiple information sources, power line detection is more effective than the previous approach. Both the accuracy and the recall of power line detection are improved from around 68% to over 92%.

Fifth generation mobile communications (5G) are expected to accommodate rapidly increasing mobile traffic aiming at the realization of a “Hyper Connected World” in which all people and surrounding things are connected and information is exchanged between them, and to play the role of a basis of the Internet of everything. A lot of research projects have started focusing on the research of 5G since around 2013, and some agreements are being formed about the vision and the candidate technologies of 5G. The 5G wireless network is expected to become a “Heterogeneous Network” where new wireless access technologies incompatible with 4G and the wireless access technologies for unlicensed band are incorporated with the enhanced technology of 4G (e.g. IMT-advanced). This paper introduces the vision and technology trends of 5G, shows key directions of the research and development of 5G in the future, and introduces some of our studies on 5G.

For the transmission of aerial surveillance videos taken from unmanned aerial vehicles (UAVs), region of interest (ROI)-based coding systems are of growing interest in order to cope with the limited channel capacities available. We present a fully automatic detection and coding system which is capable of transmitting high-resolution aerial surveillance videos at very low bit rates. Our coding system is based on the transmission of ROI areas only. We assume two different kinds of ROIs: in order to limit the transmission bit rate while simultaneously retaining a high-quality view of the ground, we only transmit new emerging areas (ROI-NA) for each frame instead of the entire frame. At the decoder side, the surface of the earth is reconstructed from transmitted ROI-NA by means of global motion compensation (GMC). In order to retain the movement of moving objects not conforming with the motion of the ground (like moving cars and their previously occluded ground), we additionally consider regions containing such objects as interesting (ROI-MO). Finally, both ROIs are used as input to an externally controlled video encoder. While we use GMC for the reconstruction of the ground from ROI-NA, we use meshed-based motion compensation in order to generate the pelwise difference in the luminance channel (difference image) between the mesh-based motion compensated and the current input image to detect the ROI-MO. High spots of energy within this difference image are used as seeds to select corresponding superpixels from an independent (temporally consistent) superpixel segmentation of the input image in order to obtain accurate shape information of ROI-MO. For a false positive detection rate (regions falsely classified as containing local motion) of less than 2% we detect more than 97% true positives (correctly detected ROI-MOs) in challenging scenarios. Furthermore, we propose to use a modified high-efficiency video coding (HEVC) video encoder. Retaining full HDTV video resolution at 30 fps and subjectively high quality we achieve bit rates of about 0.6–0.9 Mbit/s, which is a bit rate saving of about 90% compared to an unmodified HEVC encoder.

Mobile terminals have become the most familiar communication tool we use, and various types of people have come to use mobile terminals in various environments. Accordingly, situations in which we talk over the telephone in noisy environments or with someone who speaks fast have increased. However, it is sometimes difficult to hear a person's voice in these cases. To make the voice received through mobile terminals easy to hear, authors have developed two technologies. One is a voice enhancement technology that emphasizes a caller's voice according to the noise surrounding the recipient, and the other is a speech rate conversion technology that slows speech while maintaining voice quality. In this paper, we explain the trends and the features of these technologies and discuss ways to implement their algorithms on mobile terminals.

Free viewpoint video (FVV) is one of the new trends in the development of advanced visual media type that aims to provide a new immersive user experience and interactivity that goes beyond higher image quality (HD/4K TV) and higher realism (3D TV). Potential applications include interactive personal visualization and free viewpoint navigation. The goal of this paper is to provide an overview of the FVV system and some target application scenarios. Associated standardization activities and technological barriers to overcome are also described. This paper is organized as follows: a general description of the FVV system and functionalities is given in Section I. Since an FVV system is composed of a chain of processing modules, an in-depth functional description of each module is provided in Section II. Examples of emerging FVV applications and use cases are given in Section III. A summary of technical challenges to overcome for wider usage and market penetration of FVV is given in Section IV.

An infinite mixture model is applied to model-based speaker clustering with sampling-based optimization to make it possible to estimate the number of speakers. For this purpose, a framework of non-parametric Bayesian modeling is implemented with the Markov chain Monte Carlo and incorporated in the utterance-oriented speaker model. The proposed model is called the utterance-oriented Dirichlet process mixture model (UO-DPMM). The present paper demonstrates that UO-DPMM is successfully applied on large-scale data and outperforms the conventional hierarchical agglomerative clustering, especially for large amounts of utterances.

Low-cost, compact, and accurate systems for in-home rehabilitation are needed in aging, aged, and hyper-aged groups. In this study, we developed an in-home rehabilitation system for patients with balance disorders by providing visual feedback of postural information in real-time. Our system measures the user's whole body motion and the center of pressure (COP) using a Kinect and Wii Balance Board (WBB). The accuracy of body motion for estimating the anterior folding and lateral bending angles was validated experimentally by comparing the estimates with the angles given by an optical motion capture system. Additional experiments showed that the COP has a small correlation coefficient with the angles, suggesting that WBB is necessary for measuring the COP.

Being connected “anywhere anytime” has become a way of life for much of the world's population. Thanks to major technological advances in internet, wireless communication, video technology, silicon manufacturing, etc., our mobile devices have become not only faster and more powerful, but also smaller and sleeker. With the popularity of rich media on the rise, the no. 1 data traffic over the mobile network is attributed to video. That is the reason why we depict the Freeman Dyson's book title “From Eros to Gaia.” Equipped with rich media capabilities, our mobile devices enable a completely different storytelling experience unlike anything the human race has experienced before. In this paper, we review the latest technological evolutions in the wireless space and in the video space, namely long-term evolution and high-efficiency video coding, respectively. We then discuss how these advanced technologies impact our way of life at present and in years to come.

We propose a method to measure the capture-to-display delay (CDD) of a visual communication application. The method does not require modifications to the existing system, nor require the encoder and decoder clocks be synchronized. Furthermore, we propose a solution to solve the multiple-overlapped-timestamp problem due to the exposure time of the camera. We analyze the measurement error, and implement the method in software to measure the CDD of a cellphone video chat application over various types of networks. Experiments confirm the effectiveness of our proposed method.