LOW-COST RECONFIGURABLE HARDWARE IMPLEMENTATION OF REAL-TIME VIDEO STITCHING
Wang, Xuefeng (2018-10-02)
LOW-COST RECONFIGURABLE HARDWARE IMPLEMENTATION OF REAL-TIME VIDEO STITCHING
Wang, Xuefeng
(02.10.2018)
Tätä artikkelia/julkaisua ei ole tallennettu UTUPubiin. Julkaisun tiedoissa voi kuitenkin olla linkki toisaalle tallennettuun artikkeliin / julkaisuun.
Turun yliopisto
Tiivistelmä
With the development of video technology, the application of video image is used more and more widely, and the need for high quality and real-time video image increases considerably. At the same time, personalized video image gradually comes into people’s life, especially panoramic video technology. Through panoramic video, it greatly expands the people’s vision, so that obtaining the scene from all the angles in real-time becomes a reality.
Panoramic video is widely used in many situations, such as panoramic video conference, virtual reality and so on. The realization that panoramic video requires video stitching technology leads to active researches on improving the quality of panoramic video stitching and real-time stitching techniques. Most of existing video stitching technologies are based on software algorithm, because it is difficult to guarantee the real-time video stitching and low power consumption at the same time. According to the existing techniques, a reconfigurable low power hardware implementation for real-time video stitching based on the FPGA platform is proposed.
This paper proposes a flexible and low-cost hardware implementation for related applications. It can be easily reconfigured or redesigned to support cameras arrayed in different ways. The processing unit is a line instead of a frame. The software simulation is done by Matlab, and the hardware is based on FPGA chip.
Because the visual angle of each camera is limited, in order to get 360 degree panoramic, ten cameras are adopted. Based on the assumption that the position and angle of ten cameras are fixed, the whole system can be divided into the following modules: Fetching module, Color correction module, Seam line searching module, Blending module, Controlling module and Adjusting module. Adjusting module performs color re-correction and removing of drift problem from the final panorama. Fixed position of cameras ensures the fixed matching relationship between each source video image. It reduces memory resource to a great degree for all the processing unit is a single line instead of a frame in the traditional work. Meanwhile, the processing speed greatly improved. In order to have enough pixels for blending, the seam lines should not be far from the middle of the overlapped area. Thus, seam line searching with boundary is proposed in this design. It is worth noting, for the panoramic video stitching, it should not only consider in the space domain, but also the time domain. Seam line with memory is adopted here to reduce the blur.
In the hardware implementation, ten cameras with FIFOs are placed to form a circle with a certain degree to obtain 360degrees video image. All the video image data are firstly stored in the FIFO, which ensures the synchronization of the ten video sources when fetched by Fetching module. After the processing by FPGA, the final panoramic is output to the display through VGA module in real-time. It can be easily reconfigured or redesigned to support cameras arrayed in different ways in 2 dimensions, so the design can support 2 dimensional video stitching, vertical stitching and horizontal stitching.
After algorithm design, software simulation, hardware simulation and timing analysis, the whole system is debugged on the FPGA hardware platform. Through debugging and optimization, line-based video stitching algorithm is well realized.
Panoramic video is widely used in many situations, such as panoramic video conference, virtual reality and so on. The realization that panoramic video requires video stitching technology leads to active researches on improving the quality of panoramic video stitching and real-time stitching techniques. Most of existing video stitching technologies are based on software algorithm, because it is difficult to guarantee the real-time video stitching and low power consumption at the same time. According to the existing techniques, a reconfigurable low power hardware implementation for real-time video stitching based on the FPGA platform is proposed.
This paper proposes a flexible and low-cost hardware implementation for related applications. It can be easily reconfigured or redesigned to support cameras arrayed in different ways. The processing unit is a line instead of a frame. The software simulation is done by Matlab, and the hardware is based on FPGA chip.
Because the visual angle of each camera is limited, in order to get 360 degree panoramic, ten cameras are adopted. Based on the assumption that the position and angle of ten cameras are fixed, the whole system can be divided into the following modules: Fetching module, Color correction module, Seam line searching module, Blending module, Controlling module and Adjusting module. Adjusting module performs color re-correction and removing of drift problem from the final panorama. Fixed position of cameras ensures the fixed matching relationship between each source video image. It reduces memory resource to a great degree for all the processing unit is a single line instead of a frame in the traditional work. Meanwhile, the processing speed greatly improved. In order to have enough pixels for blending, the seam lines should not be far from the middle of the overlapped area. Thus, seam line searching with boundary is proposed in this design. It is worth noting, for the panoramic video stitching, it should not only consider in the space domain, but also the time domain. Seam line with memory is adopted here to reduce the blur.
In the hardware implementation, ten cameras with FIFOs are placed to form a circle with a certain degree to obtain 360degrees video image. All the video image data are firstly stored in the FIFO, which ensures the synchronization of the ten video sources when fetched by Fetching module. After the processing by FPGA, the final panoramic is output to the display through VGA module in real-time. It can be easily reconfigured or redesigned to support cameras arrayed in different ways in 2 dimensions, so the design can support 2 dimensional video stitching, vertical stitching and horizontal stitching.
After algorithm design, software simulation, hardware simulation and timing analysis, the whole system is debugged on the FPGA hardware platform. Through debugging and optimization, line-based video stitching algorithm is well realized.