摘要 :
Diminishing the appearance of a fence in an image is a challenging research area due to the characteristics of fences (thinness, lack of texture, etc.) and the need for occluded background restoration. In this paper, we describe a...
展开
Diminishing the appearance of a fence in an image is a challenging research area due to the characteristics of fences (thinness, lack of texture, etc.) and the need for occluded background restoration. In this paper, we describe a fence removal method for an image sequence captured by a user making a sweep motion, in which occluded background is potentially observed. To make use of geometric and appearance information such as consecutive images, we use two well-known approaches: structure from motion and light field rendering. Results using real image sequences show that our method can stably segment fences and preserve background details for various fence and background combinations. A new video without the fence, with frame coherence, can be successfully provided.
收起
摘要 :
Over the last several decades, developments in underwater laser line scan (LLS) serial imaging sensors have resulted in significant improvements in turbid water imaging performance. In the last few years, there has been renewed in...
展开
Over the last several decades, developments in underwater laser line scan (LLS) serial imaging sensors have resulted in significant improvements in turbid water imaging performance. In the last few years, there has been renewed interest in distributed, truly multistatic time-varying intensity (TVI) (i.e., multiple transmitter nonsynchronous LLS) sensor configurations. In addition to being capable of high-quality image acquisition through tens of beam attenuation lengths, while simultaneously establishing a non-line-of-sight free-space communications link, these system architectures also have the potential to provide a more synoptic image coverage of larger regions of seabed and the flexibility to simultaneously examine a target from different perspectives. A related issue worth investigation is how to utilize these capabilities to improve rendering of the underwater scenes. In this regard, light field rendering (LFR)—a type of image-based rendering (IBR) technique—offers several advantages. Compared to other IBR techniques, LFR can provide signal-to-noise ratio (SNR) improvements and the ability to image through obscuring objects in front of the target. On the other hand, multistatic nonsynchronous LLS can be readily configured to acquire image sequences needed to generate LFR. This paper investigates the application of LFR to images taken from a distributed bistatic nonsynchronous LLS imager using both line-of-sight and non-line-of-sight imaging geometries to create multiperspective rendering of an unknown underwater scene. The issues related to effectively applying this technique to underwater LLS imagery are analyzed and an image postprocessing flow to address these issues is proposed. The results from a series of experiments at the Harbor Branch Oceanographic Institute at the Florida Atlantic University (HBOI–FAU, Fort Pierce, FL, USA) optical imaging test tank demonstrated the capability of using bistatic/multistatic nonsynch- onous LLS system to generated LFR and, therefore, verify the proposed image processing flow. The benefits of LFR to underwater imaging in challenging environments were further demonstrated via imaging against a variety of obstacles such as mesh screens, bubbles, and water at different turbidity. Image quality metrics based on mutual information and texture features were used in the analysis of the experimental results.
收起
摘要 :
Occlusions are common phenomena in light field rendering (LFR) technology applications. The 3-D spatial structures of some features may be missing or incorrect when capturing some samples due to occlusion discontinuities. Most pri...
展开
Occlusions are common phenomena in light field rendering (LFR) technology applications. The 3-D spatial structures of some features may be missing or incorrect when capturing some samples due to occlusion discontinuities. Most prior works on LFR, however, have neglected occlusions from other objects in 3-D scenes that do not participate in the capturing and rendering of the light field. To improve rendering quality, this report proposes an occlusion probability learning framework (OPLF) based on a deep Boltzmann machine (DBM) to compensate for the occluded information. In the OPLF, an occlusion probability density model is applied to calculate the visibility scores, which are modeled as hidden variables. Additionally, the probability of occlusion is related to the visibility, the camera configuration (i.e., position and direction), and the relationship between the occlusion object and occluded object. Furthermore, a deep probability model based on the OPLF is used for learning the occlusion relationship between the camera and object in multiple layers. The proposed OPLF can optimize the LFR quality. Finally, to verify the claimed performance, we also compare the OPLF with the most advanced occlusion theory and light field reconstruction algorithms. The experimental results show that the proposed OPLF outperforms other known occlusion quantization schemes.
收起
摘要 :
Light fields capture both the spatial and angular rays, thus enabling free-viewpoint rendering and custom selection of the focal plane. Scientists can interactively explore pre-recorded microscopic light fields of organs, microbes...
展开
Light fields capture both the spatial and angular rays, thus enabling free-viewpoint rendering and custom selection of the focal plane. Scientists can interactively explore pre-recorded microscopic light fields of organs, microbes, and neurons using virtual reality headsets. However, rendering high-resolution light fields at interactive frame rates requires a very high rate of texture sampling, which is challenging as the resolutions of light fields and displays continue to increase. In this article, we present an efficient algorithm to visualize 4D light fields with 3D-kernel foveated rendering (3D-KFR). The 3D-KFR scheme coupled with eye-tracking has the potential to accelerate the rendering of 4D depth-cued light fields dramatically. We have developed a perceptual model for foveated light fields by extending the KFR for the rendering of 3D meshes. On datasets of high-resolution microscopic light fields, we observe 3.47 x -7.28x speedup in light field rendering with minimal perceptual loss of detail. We envision that 3D-KFR will reconcile the mutually conflicting goals of visual fidelity and rendering speed for interactive visualization of light fields.
收起
摘要 :
Light fields capture both the spatial and angular rays, thus enabling free-viewpoint rendering and custom selection of the focal plane. Scientists can interactively explore pre-recorded microscopic light fields of organs, microbes...
展开
Light fields capture both the spatial and angular rays, thus enabling free-viewpoint rendering and custom selection of the focal plane. Scientists can interactively explore pre-recorded microscopic light fields of organs, microbes, and neurons using virtual reality headsets. However, rendering high-resolution light fields at interactive frame rates requires a very high rate of texture sampling, which is challenging as the resolutions of light fields and displays continue to increase. In this article, we present an efficient algorithm to visualize 4D light fields with 3D-kernel foveated rendering (3D-KFR). The 3D-KFR scheme coupled with eye-tracking has the potential to accelerate the rendering of 4D depth-cued light fields dramatically. We have developed a perceptual model for foveated light fields by extending the KFR for the rendering of 3D meshes. On datasets of high-resolution microscopic light fields, we observe 3.47 x -7.28x speedup in light field rendering with minimal perceptual loss of detail. We envision that 3D-KFR will reconcile the mutually conflicting goals of visual fidelity and rendering speed for interactive visualization of light fields.
收起
摘要 :
In this paper, we present a new Light Field representation for efficient Light Field processing and rendering called Fourier Disparity Layers (FDL). The proposed FDL representation samples the Light Field in the depth (or equivale...
展开
In this paper, we present a new Light Field representation for efficient Light Field processing and rendering called Fourier Disparity Layers (FDL). The proposed FDL representation samples the Light Field in the depth (or equivalently the disparity) dimen
收起
摘要 :
This survey gives an overview of the use of importance, an adjoint of light, in speeding up rendering. The importance of a light distribution indicates its contribution to the region of most interest-typically the directly visible...
展开
This survey gives an overview of the use of importance, an adjoint of light, in speeding up rendering. The importance of a light distribution indicates its contribution to the region of most interest-typically the directly visible parts of a scene. Importance can therefore be used to concentrate global illumination and ray tracing calculations where they matter most for image accuracy, while reducing computations in areas of the scene that do not significantly influence the image. In this paper, we attempt to clarify the various uses of adjoints and importance in rendering by unifying them into a single framework. While doing so, we also generalize some theoretical results-known from discrete representations-to a continuous domain.
收起
摘要 :
Light field image (LFI) records the radiation intensity and direction information of light in scene, offering powerful capabilities for computer vision. It is found that the distortion caused by LFI processing is obviously reflect...
展开
Light field image (LFI) records the radiation intensity and direction information of light in scene, offering powerful capabilities for computer vision. It is found that the distortion caused by LFI processing is obviously reflected in the two representations of LFI, that is, subaperture images (SAIs) and microlens images (MLI). Specifically, on the one hand, pseudoreference SAI (PSAI) of a distorted SAI, which is rendered with the adjacent distorted SAIs, can be used to measure the distortion of the involved SAIs. On the other hand, macropixel blocks (MPBs) of MLI can be adopted to measure LFI global angular distortion. Therefore, a PSAIs- and MLI-based blind LFI quality measurement (PM-BLFIQM) method is proposed in this article. First, the local light field (LLF) is defined to generate PSAI of the central distorted SAI in each LLF of LFI, and then a pseudo-full-reference quality measurement metric based on PSAIs is presented. Second, to measure the angular consistency of the distorted LFI, an MLI-based feature extraction scheme is proposed for MPBs and their mean removal version by using discrete cosine transform and singular value decomposition. Finally, the quality of the distorted LFI is predicted with the features extracted from LLFs and MLI. Experimental results demonstrate that both PSAI and MPB are effective for measuring the quality of distorted LFI. In addition, the results on three LFI databases show that the proposed method achieves higher Spearman rank-order correlation coefficient (SROCC) and Pearson linear correlation coefficient (PLCC) values, so it outperforms the state-of-the-art methods in terms of consistency with human visual perception.
收起
摘要 :
Today, invisibility does not seem out of reach to the realm of science. At least, it's more probable than it was a century ago. There are yet many technology gaps to bridge to reach true invisibility. What does being invisible mea...
展开
Today, invisibility does not seem out of reach to the realm of science. At least, it's more probable than it was a century ago. There are yet many technology gaps to bridge to reach true invisibility. What does being invisible mean then? In cognitive science, this invisibility phenomenon is called cognitive blindness. A typical example is the case of somebody leaving a meeting room without being noticed by an audience that is deeply engrossed in a conversation. Such a cognitive invisibility could be individually selective compared with real-world, physics-based absolute invisibility. Yet another potential investigation topic linking physics-based and human-eye invisibility is vibration. The approaches to invisibility presented here are within the probable reaches of today's science. Yet the door is widely open to our imagination, and we can find other ingenious schemes. As the pace of technology keeps ineluctably increasing, recent achievements and progress suggest that the active camouflage paradigm of the digital chameleon box is not just a Utopian dream, but is rather becoming closer to reality
收起
摘要 :
Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighbor...
展开
Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we propose
content-aware warping
, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network. Based on this learnable warping module, we propose a new end-to-end learning-based framework for novel view synthesis from a set of input source views, in which two additional modules, namely confidence-based blending and feature-assistant spatial refinement, are naturally proposed to handle the occlusion issue and capture the spatial correlation among pixels of the synthesized view, respectively. Besides, we also propose a weight-smoothness loss term to regularize the network. Experimental results on light field datasets with wide baselines and multi-view datasets show that the proposed method significantly outperforms state-of-the-art methods both quantitatively and visually. The source code is publicly available at
https://github.com/MantangGuo/CW4VS
.
收起
原文获取