Search Results (80)

In order to address the issues of image quality degradation and distortion that arise in the context of video transmission coding and decoding, a method based on an enhanced version of CycleGAN is put forth. The lightweight attention module is integrated into the residual block of the generator module structure, thereby facilitating the extraction of image details and motion compensation. Furthermore, the perceptual function LPIPS loss is increased to align the image restoration effect more closely with human perception. Additionally, the network training method is modified, and the original image is divided into 128 × 128 small blocks for training, thus enhancing the network’s accuracy in restoring details. The experimental results demonstrate that the algorithm attains an average PSNR value of 30.1147 on the publicly accessible YUV sequence dataset, YUV Trace Dataset, which is a 9.02% enhancement compared to the original network. Additionally, the LPIPS value reaches 0.2639, representing a 10.42% reduction, and effectively addresses the issue of image quality deterioration during transmission. Full article

As urban environments become increasingly complex and the costs and challenges of infrastructure upgrades continue to rise, wayfinding signage has become an effective solution to cope with urban dynamics due to its low cost and high flexibility. Although the functionality of wayfinding signage has been extensively studied, the perceptual differences between designers and non-designers have not been adequately explored. Ignoring these differences may lead to the overlooking of users’ real and diverse needs, resulting in suboptimal signage performance in practical applications and ultimately a reduction in the overall functionality and user experience of urban spaces. This study aims to bridge this perceptual gap. For this study, we conducted a questionnaire survey in China to compare the visual preferences of designers and non-designers regarding text, shape, color coding, and patterns. The results indicate that designers prioritize functionality and clarity to ensure the effective use of signage in complex urban environments, whereas non-designers prefer wayfinding signages that reflect local cultural symbols and characteristics. Our conclusions suggest that the public’s expectations for wayfinding signage extend beyond basic navigational functions, with an emphasis on cultural expression and visual appeal. Understanding these perceptual differences is crucial in developing design strategies that balance functionality, esthetics, and sustainability, thereby facilitating the sustainable integration of signage into urban landscapes. Full article

As most of the videos are destined for human perception, many techniques have been designed to improve video coding based on how the human visual system perceives video quality. In this paper, we propose the use of two perceptual coding techniques, namely contrast masking and texture masking, jointly operating under the High Efficiency Video Coding (HEVC) standard. These techniques aim to improve the subjective quality of the reconstructed video at the same bit rate. For contrast masking, we propose the use of a dedicated weighting matrix for each block size (from $4\times 4$ up to $32\times 32$), unlike the HEVC standard, which only defines an $8\times 8$ weighting matrix which it is upscaled to build the $16\times 16$ and $32\times 32$ weighting matrices (a $4\times 4$ weighting matrix is not supported). Our approach achieves average Bjøntegaard Delta-Rate (BD-rate) gains of between $2.5\%$ and $4.48\%$, depending on the perceptual metric and coding mode used. On the other hand, we propose a novel texture masking scheme based on the classification of each coding unit to provide an over-quantization depending on the coding unit texture level. Thus, for each coding unit, its mean directional variance features are computed to feed a support vector machine model that properly predicts the texture type (plane, edge, or texture). According to this classification, the block’s energy, the type of coding unit, and its size, an over-quantization value is computed as a QP offset (DQP) to be applied to this coding unit. By applying both techniques in the HEVC reference software, an overall average of $5.79\%$ BD-rate gain is achieved proving their complementarity. Full article

Ensemble coding allows observers to form an average to represent a set of elements. However, it is unclear whether observers can extract an average from a cross-category set. Previous investigations on this issue using low-level stimuli yielded contradictory results. The current study addressed this issue by presenting high-level stimuli (i.e., a crowd of facial expressions) simultaneously (Experiment 1) or sequentially (Experiment 2), and asked participants to complete a member judgment task. The results showed that participants could extract average information from a group of cross-category facial expressions with a short perceptual distance. These findings demonstrate cross-category ensemble coding of high-level stimuli, contributing to the understanding of ensemble coding and providing inspiration for future research. Full article

This study focuses on accent shift or stylization to American English features in Anglophone pop-rock music and examines linguistic constraints alongside music-related considerations, as well as the effect of changes in musical genre on variable accent shift. The case study is the British band Mumford and Sons and their variable production of non-prevocalic rhotics as either present or absent. Mumford and Sons is of interest because they have displayed a change in their musical style throughout their career from Americana to alt-rock. The band’s four studio albums were auditorily analyzed and coded for rhotic vs. non-rhotic with aid from spectrograms. The linguistic factors considered were word class, preceding vowel according to the word’s lexical set, complexity of the preceding vowel, syllable complexity, stress, and location within the word and phrase. In addition, the effect of singing-related factors of syllable elongation and rhyming, and of the specific album, were also explored. Results show that rhoticity is favored in content words, stressed contexts, complex syllables, and NURSE words. This pattern is explained as stemming from the perceptual prominence of those contexts based on their acoustic and phonological characteristics. Results further show that syllable elongation leads to more rhoticity and that rhyming words tend to agree in their (non-)rhoticity. Finally, the degree of rhoticity decreases as the band departs from Americana in their later albums, highlighting the relevance of music genre for accent stylization. Full article

Auditory temporal processing is a vital component of auditory stream segregation, or the process in which complex sounds are separated and organized into perceptually meaningful objects. Temporal processing can degrade prior to hearing loss, and is suggested to be a contributing factor to difficulties with speech-in-noise perception in normal-hearing listeners. The current study tested this hypothesis in middle-aged adults—an under-investigated cohort, despite being the age group where speech-in-noise difficulties are first reported. In 76 participants, three mechanisms of temporal processing were measured: peripheral auditory nerve function using electrocochleography, subcortical encoding of periodic speech cues (i.e., fundamental frequency; F0) using the frequency following response, and binaural sensitivity to temporal fine structure (TFS) using a dichotic frequency modulation detection task. Two measures of speech-in-noise perception were administered to explore how contributions of temporal processing may be mediated by different sensory demands present in the speech perception task. This study supported the hypothesis that temporal coding deficits contribute to speech-in-noise difficulties in middle-aged listeners. Poorer speech-in-noise perception was associated with weaker subcortical F0 encoding and binaural TFS sensitivity, but in different contexts, highlighting that diverse aspects of temporal processing are differentially utilized based on speech-in-noise task characteristics. Full article

In this paper, a novel data transmission scheme, interference perceptual multi-modulation (IP-MM), is proposed for full-duplex (FD) systems. In order to unlink the conventional uplink (UL) data transmission using a single modulation and coding scheme (MCS) over the entire assigned UL bandwidth, IP-MM enables the transmission of UL data channels based on multiple MCS levels, where a different MCS level is applied to each subband of UL transmission. In IP-MM, a deep convolutional neural network is used for MCS-level prediction for each UL subband by estimating the potential residual self-interference (SI) according to the downlink (DL) resource allocation pattern. In addition, a subband-based UL transmission procedure is introduced from a specification point of view to enable IP-MM-based UL transmission. The benefits of IP-MM are verified using simulations, and it is observed that IP-MM achieves approximately $20\%$ throughput gain compared to the conventional UL transmission scheme. Full article

Self-supervised learning (SSL) has emerged as a promising paradigm for learning flexible speech representations from unlabeled data. By designing pretext tasks that exploit statistical regularities, SSL models can capture useful representations that are transferable to downstream tasks. Barlow Twins (BTs) is an SSL technique inspired by theories of redundancy reduction in human perception. In downstream tasks, BTs representations accelerate learning and transfer this learning across applications. This study applies BTs to speech data and evaluates the obtained representations on several downstream tasks, showing the applicability of the approach. However, limitations exist in disentangling key explanatory factors, with redundancy reduction and invariance alone being insufficient for factorization of learned latents into modular, compact, and informative codes. Our ablation study isolated gains from invariance constraints, but the gains were context-dependent. Overall, this work substantiates the potential of Barlow Twins for sample-efficient speech encoding. However, challenges remain in achieving fully hierarchical representations. The analysis methodology and insights presented in this paper pave a path for extensions incorporating further inductive priors and perceptual principles to further enhance the BTs self-supervision framework. Full article

Pedestrian trajectory prediction is one of the most important topics to be researched for unmanned driving and intelligent mobile robots to perform perceptual interaction with the environment. To solve the problem of the SGAN (social generative adversarial networks) model lacking an understanding of pedestrian interaction and scene constraints, this paper proposes a trajectory prediction method based on a scenario-constrained generative adversarial network. Firstly, a self-attention mechanism is added, which can integrate information at every moment. Secondly, mutual information is introduced to enhance the influence of latent code on the predicted trajectory. Finally, a new social pool is introduced into the original trajectory prediction model, and a scene edge extraction module is added to ensure the final output path of the model is within the passable area in line with the physical scene, which greatly improves the accuracy of trajectory prediction. Based on the CARLA (CAR Learning to Act) simulation platform, the improved model was tested on the public dataset and the self-built dataset. The experimental results showed that the average moving deviation was reduced by $26.4\%$ and the final offset was reduced by $23.8\%$, which proved that the improved model could better solve the uncertainty of pedestrian turning decisions. The accuracy and stability of pedestrian trajectory prediction are improved while maintaining multiple modes. Full article

The predictive coding theory is currently widely accepted as the theoretical basis of perception and chronic perceptual disorders are explained as the maladaptive compensation of the brain to a prediction error. Although this gives us a general framework to work with, it is still not clear who may be more susceptible and/or vulnerable to aberrations in this system. In this paper, we study changes in predictive coding through the lens of tinnitus and pain. We take a step back to understand how the predictive coding system develops from infancy, what are the different neural and bio markers that characterise this system in the acute, transition and chronic phases and what may be the factors that pose a risk to the aberration of this system. Through this paper, we aim to identify people who may be at a higher risk of developing chronic perceptual disorders as a reflection of aberrant predictive coding, thereby giving future studies more facets to incorporate in their investigation of early markers of tinnitus, pain and other disorders of predictive coding. We therefore view this paper to encourage the thinking behind the development of preclinical biomarkers to maladaptive predictive coding. Full article

In the industry 4.0 era, video applications such as surveillance visual systems, video conferencing, or video broadcasting have been playing a vital role. In these applications, for manipulating and tracking objects in decoded video, the quality of decoded video should be consistent because it largely affects the performance of the machine analysis. To cope with this problem, we propose a novel perceptual video coding (PVC) solution in which a full reference quality metric named video multimethod assessment fusion (VMAF) is employed together with a deep convolutional neural network (CNN) to obtain consistent quality while still achieving high compression performance. First of all, in order to achieve the consistent quality requirement, we propose a CNN model with an expected VMAF as input to adaptively adjust the quantization parameters (QP) for each coding block. Afterwards, to increase the compression performance, a Lagrange coefficient of rate-distortion optimization (RDO) mechanism is adaptively computed according to rate-QP and quality-QP models. The experimental results show that the proposed PVC solution has achieved two targets simultaneously: the quality of video sequence is kept consistently with an expected quality level and the bit rate saving of the proposed method is higher than traditional video coding standards and the relevant benchmark, notably with around 10% bitrate saving on average. Full article

In recent years, underwater image processing has played an essential role in ocean exploration. The complexity of seawater leads to the phenomena of light absorption and scattering, which in turn cause serious image degradation problems, making it difficult to capture high-quality underwater images. A novel underwater image enhancement model based on Hybrid Enhanced Generative Adversarial Network (HEGAN) is proposed in this paper. By designing a Hybrid Underwater Image Synthesis Model (HUISM) based on a physical model and a deep learning method, many richly varied paired underwater images are acquired to compensate for the missing problem of underwater image enhancement dataset training. Meanwhile, the Detection Perception Enhancement Model (DPEM) with Perceptual Loss is designed to transfer the coding knowledge in the form of the gradient to the enhancement model through the perceptual loss, which leads to the generation of visually better and detection-friendly underwater images. Then, the synthesized and enhanced models are integrated into an adversarial network to generate high-quality underwater clear images through game learning. Experiments show that the proposed method significantly outperforms several state-of-the-art methods both qualitatively and quantitatively. Furthermore, it is also demonstrated that the method can improve target detection performance in underwater environments, which has specific application value for subsequent image processing. Full article

A dermatologist-like automatic classification system is developed in this paper to recognize nine different classes of pigmented skin lesions (PSLs), using a separable vision transformer (SVT) technique to assist clinical experts in early skin cancer detection. In the past, researchers have developed a few systems to recognize nine classes of PSLs. However, they often require enormous computations to achieve high performance, which is burdensome to deploy on resource-constrained devices. In this paper, a new approach to designing SVT architecture is developed based on SqueezeNet and depthwise separable CNN models. The primary goal is to find a deep learning architecture with few parameters that has comparable accuracy to state-of-the-art (SOTA) architectures. This paper modifies the SqueezeNet design for improved runtime performance by utilizing depthwise separable convolutions rather than simple conventional units. To develop this Assist-Dermo system, a data augmentation technique is applied to control the PSL imbalance problem. Next, a pre-processing step is integrated to select the most dominant region and then enhance the lesion patterns in a perceptual-oriented color space. Afterwards, the Assist-Dermo system is designed to improve efficacy and performance with several layers and multiple filter sizes but fewer filters and parameters. For the training and evaluation of Assist-Dermo models, a set of PSL images is collected from different online data sources such as Ph2, ISBI-2017, HAM10000, and ISIC to recognize nine classes of PSLs. On the chosen dataset, it achieves an accuracy (ACC) of 95.6%, a sensitivity (SE) of 96.7%, a specificity (SP) of 95%, and an area under the curve (AUC) of 0.95. The experimental results show that the suggested Assist-Dermo technique outperformed SOTA algorithms when recognizing nine classes of PSLs. The Assist-Dermo system performed better than other competitive systems and can support dermatologists in the diagnosis of a wide variety of PSLs through dermoscopy. The Assist-Dermo model code is freely available on GitHub for the scientific community. Full article

In this study, the factors influencing the cognitive load of computer programmers during the perception of different code tasks were investigated. The eye movement features of computer programmers were used to provide a significant relationship between the perceptual processes of the sample codes and cognitive load. Thanks to the relationship, the influence of various personal characteristics of programmers on cognitive load was examined. Various personal parameters such as programming experience, age, native language, and programming frequency were used in the study. The study was performed on the Eye Movements in Programming (EMIP) dataset containing 216 programmers with different characteristics. Eye movement information recorded during two different code comprehension tasks was decomposed into sub-information, such as pupil movement speed and diameter change. Rapid changes in eye movement signals were adaptively detected using the z-score peak detection algorithm. Regarding the cognitive load calculations, canonical correlation analysis was used to build a statistically significant and efficient mathematical model connecting the extracted eye movement features and the different parameters of the programmers, and the results were statistically significant. As a result of the analysis, the factors affecting the cognitive load of computer programmers for the related database were converted into percentages, and it was seen that linguistic distance is an essential factor in the cognitive load of programmers and the effect of gender on cognitive load is quite limited. Full article

Visual quality assessment is often used as a key performance indicator (KPI) to evaluate the performance of electronic devices. There exists a significant association between visual quality assessment and electronic devices. In this paper, we bring attention to alternative choices of perceptual loss function for end-to-end deep video coding (E2E-DVC), which can be used to reduce the amount of data generated by electronic sensors and other sources. Thus, we analyze the effects of different full-reference quality assessment (FR-QA) metrics on E2E-DVC. First, we select five optimization-suitable FR-QA metrics as perceptual objectives, which are differentiable and thus support back propagation, and use them to optimize an E2E-DVC model. Second, we analyze the rate–distortion (R-D) behaviors of an E2E-DVC model under different loss function optimizations. Third, we carry out subjective human perceptual tests on the reconstructed videos to show the performance of different FR-QA optimizations on subjective visual quality. This study reveals the effects of the competing FR-QA metrics on E2E-DVC and provides a guide for further future study on E2E-DVC in terms of perceptual loss function design. Full article