Introduction and development trend of STv0676 network camera system
This article introduces the composition, working mode and development trend of the network camera system based on STMicroelectronics STv0676.
The application of video streaming solutions is rapidly increasing. Some of these reasons include the impact of diseases such as SARS on society, visual meetings that do not require travel, and security / surveillance applications. The camera is the key device in the video streaming solution. We have heard of PC cameras and network cameras in the past, but their working principles are very different. In fact, these two concepts have always been associated with PC cameras, but for network cameras, it is not correct to link them with PCs. The PC camera refers to a camera with a USB interface. Most of the standard software provided by the operating system is used to hold a video conference or video chat.
In terms of monitoring technology, there are two systems available, one is a CCTV camera based on CCD technology and uses analog transmission (PAL / NTSC), and the other is a new system using CCD or CMOS digital transmission. A closer look at the technologies used by these two systems reveals that CMOS technology has greater advantages for this market: first, CMOS cameras do not require different voltages, which means that the power supply design is much simpler; second, the biggest advantage of CMOS cameras The timing and ADC are integrated, which means that the image output has been digitized, which helps reduce the overall cost of the system. In terms of low-light brightness performance, CMOS technology provided by STMicroelectronics is gradually approaching the level of low-brightness optimized pixel technology we currently use. In addition, the sensor has been able to execute some processing algorithms to help improve the performance of the video processor.
Web camera
Now let's take a look at a real network camera, or so-called IP camera. In addition to being suitable for home security and home surveillance, IP cameras will not replace PC cameras. This is because if we want to transmit video or video and audio over the network, we need at least some equipment like a streaming server. If a PC-based system is used, such a system requires the PC to be turned on 24 hours, which means a great waste of power and a lot of noise. In addition, if we want to use the system for home use, this application will also reduce PC performance. On the contrary, the real IP camera is built on the basis of an embedded streaming server, which means that the camera has its own streaming server, no longer needs to use a PC, and only needs to plug in an Ethernet (hub), or use wireless Ethernet To work.
It can be found in Figure 1 that only one PC is used in the client. The advantage of this system is that the client can be located anywhere in the world, as long as there is an Internet connection, it can be observed and controlled by the camera. The solution will have more applications. As mobile phones have more and more data communication options available, and the speed is increasing rapidly, it is possible that the camera will issue an alarm through motion detection or noise detection in the future, and the video stream can be transmitted to the mobile phone by sending SMS or directly dialing the phone Cell phone. This concept also applies to unattended office situations. Other uses include installing the camera on the door. If someone presses the doorbell, the camera will make a video call, and you can open the door through the video phone.
From a technical point of view, STMicroelectronics and Taiwan ’s Taifatech (Taifa Technology) cooperate to provide a true one-stop solution (see Figure 2). Customers only need to modify the user interface they want to display in the client browser. If they do not modify, they can use the default interface and apply it directly. The solution we provide is based on the Stv0676 coprocessor and our VGA sensor VV6501. This solution allows video transmission via USB and the so-called "Digiport (Digital Port)". The Digiport we use in this solution is a bus (8-bit data and 2-bit control) that can transmit parallel data to any embedded system. The maximum clock speed is 1.5MHz, which means that up to 1.5M bytes of data can be transmitted per second, which can easily transmit 30fps video.
Figure 3 is a block diagram of the solution. On the left of the figure, the video processor is directly connected to the sensor and is used to control all necessary image processing functions, such as automatic white balance (AWB), automatic exposure (AE), color transformation array (colour shifTIng matrix), exposure area emphasis, Defect pixel correction and color model calculation, etc. STv0676 can work in the following different modes to output image data to the system.
Figure 3 Mode A STv0676 adopts USB mode, and directly transfers image data to the USB port through FIFO (First In First Out).
Mode B STv0676 adopts the main mode, and can send CCIR-656, YcrCb and RGB 7-bit color data stream to Digiport. Master mode means that STv0676 can clock data according to the selected frame rate. In this mode, it can connect an external TV decoder and directly transmit the signal to the TV.
Mode C STv0676 sends RGB24, YUV and MJPEG data streams to Digiport through FIFO. This mode is limited to the 1.5MHz data clock, but the system will be in slave mode and the external controller will control all data transmission.
Dynamic data compression
Using STv0676 in an embedded environment means that when using our system in mode C, data can be read from STv0676 in asynchronous mode, so that the main MCU has sufficient performance to handle other tasks. One core of this solution is a 4KB FIFO. For the large amount of data we are talking about, this capacity is not large enough, but the video compressor (VC) helps solve this problem. VC has the function of establishing dynamic compression ratio selection, which means that VC based on FIFO duty factor can determine the compression ratio. For example, when the MCU reading is fast and the FIFO is gradually emptied, VC compression is reduced, and when the MCU reading is slow and the FIFO is gradually full, the VC will be compressed as much as possible. But all these cannot avoid FIFO overflow or empty. When the FIFO is empty, an idle alarm will be generated. When the MCU is too slow to read, the FIFO will overflow and no alarm will be issued, but the video stream does not stop. In fact, this situation rarely occurs because the system can calculate the data rate. Now let's take a look at the amount of data, according to the resolution of VGA can have the following calculation:
Bayer pattern per frame: 640 × 480 = 307,200 (pixels) YUV per frame: 307,200 × 2B = 614,400B / 1024 (KB) = 600KB.
For example, transmitting 30fps in uncompressed mode means a data volume of up to 17.58MBps. Therefore, the only way is to use data compression. Since JPEG compression is a dynamic compression, the size cannot be predicted 100%. For example, you want to compress a frame at a ratio of 10: 1, which means that VC tries to achieve this ratio, but in fact it is sometimes easy to reach and sometimes not. It is worth mentioning that the amount of data calculated is not simply divided by 10 on the calculator. In view of this, VC uses a dynamic adjustment, so we can have a stable 4KB FIFO. Therefore, the compression ratio of our VC is about 80: 1 at the maximum, but the actual situation is that the effect of the compression ratio of 30: 1 is acceptable, and the higher will cause too much compression noise, which is unacceptable. Therefore, according to 30fps @ VGA transmission, the required data volume will be 600KBps.
This article summarizes
The implementation of the above system is very simple: first capture the image through the sensor, and then process Stv0676 and directly transfer the MJPEG to TF-520. TF-520 is a single-chip IP server, it only needs memory and Ethernet PHY, and other parts such as PCI interface, MAC and special IP for TCP / IP protocol stack processing have been integrated. If it is designed to be wireless, you only need to connect a miniPCI WLAN card to the system and change the firmware to get a wireless solution.
We will have further development plans in the product roadmap, one of which is to include audio and higher resolutions that may be up to two million pixels, which can be digitally focused and image stored, and can still be passed at 30fps VGA Internet delivery solution. IP cameras will definitely have brighter development prospects than USB cameras, and we will continue to introduce solutions that can simplify our lives.
The application of video streaming solutions is rapidly increasing. Some of these reasons include the impact of diseases such as SARS on society, visual meetings that do not require travel, and security / surveillance applications. The camera is the key device in the video streaming solution. We have heard of PC cameras and network cameras in the past, but their working principles are very different. In fact, these two concepts have always been associated with PC cameras, but for network cameras, it is not correct to link them with PCs. The PC camera refers to a camera with a USB interface. Most of the standard software provided by the operating system is used to hold a video conference or video chat.
In terms of monitoring technology, there are two systems available, one is a CCTV camera based on CCD technology and uses analog transmission (PAL / NTSC), and the other is a new system using CCD or CMOS digital transmission. A closer look at the technologies used by these two systems reveals that CMOS technology has greater advantages for this market: first, CMOS cameras do not require different voltages, which means that the power supply design is much simpler; second, the biggest advantage of CMOS cameras The timing and ADC are integrated, which means that the image output has been digitized, which helps reduce the overall cost of the system. In terms of low-light brightness performance, CMOS technology provided by STMicroelectronics is gradually approaching the level of low-brightness optimized pixel technology we currently use. In addition, the sensor has been able to execute some processing algorithms to help improve the performance of the video processor.
Web camera
Now let's take a look at a real network camera, or so-called IP camera. In addition to being suitable for home security and home surveillance, IP cameras will not replace PC cameras. This is because if we want to transmit video or video and audio over the network, we need at least some equipment like a streaming server. If a PC-based system is used, such a system requires the PC to be turned on 24 hours, which means a great waste of power and a lot of noise. In addition, if we want to use the system for home use, this application will also reduce PC performance. On the contrary, the real IP camera is built on the basis of an embedded streaming server, which means that the camera has its own streaming server, no longer needs to use a PC, and only needs to plug in an Ethernet (hub), or use wireless Ethernet To work.
It can be found in Figure 1 that only one PC is used in the client. The advantage of this system is that the client can be located anywhere in the world, as long as there is an Internet connection, it can be observed and controlled by the camera. The solution will have more applications. As mobile phones have more and more data communication options available, and the speed is increasing rapidly, it is possible that the camera will issue an alarm through motion detection or noise detection in the future, and the video stream can be transmitted to the mobile phone by sending SMS or directly dialing the phone Cell phone. This concept also applies to unattended office situations. Other uses include installing the camera on the door. If someone presses the doorbell, the camera will make a video call, and you can open the door through the video phone.
figure 1
System solutionFrom a technical point of view, STMicroelectronics and Taiwan ’s Taifatech (Taifa Technology) cooperate to provide a true one-stop solution (see Figure 2). Customers only need to modify the user interface they want to display in the client browser. If they do not modify, they can use the default interface and apply it directly. The solution we provide is based on the Stv0676 coprocessor and our VGA sensor VV6501. This solution allows video transmission via USB and the so-called "Digiport (Digital Port)". The Digiport we use in this solution is a bus (8-bit data and 2-bit control) that can transmit parallel data to any embedded system. The maximum clock speed is 1.5MHz, which means that up to 1.5M bytes of data can be transmitted per second, which can easily transmit 30fps video.
figure 2
Figure 3 is a block diagram of the solution. On the left of the figure, the video processor is directly connected to the sensor and is used to control all necessary image processing functions, such as automatic white balance (AWB), automatic exposure (AE), color transformation array (colour shifTIng matrix), exposure area emphasis, Defect pixel correction and color model calculation, etc. STv0676 can work in the following different modes to output image data to the system.
Figure 3 Mode A STv0676 adopts USB mode, and directly transfers image data to the USB port through FIFO (First In First Out).
Mode B STv0676 adopts the main mode, and can send CCIR-656, YcrCb and RGB 7-bit color data stream to Digiport. Master mode means that STv0676 can clock data according to the selected frame rate. In this mode, it can connect an external TV decoder and directly transmit the signal to the TV.
Mode C STv0676 sends RGB24, YUV and MJPEG data streams to Digiport through FIFO. This mode is limited to the 1.5MHz data clock, but the system will be in slave mode and the external controller will control all data transmission.
Dynamic data compression
Using STv0676 in an embedded environment means that when using our system in mode C, data can be read from STv0676 in asynchronous mode, so that the main MCU has sufficient performance to handle other tasks. One core of this solution is a 4KB FIFO. For the large amount of data we are talking about, this capacity is not large enough, but the video compressor (VC) helps solve this problem. VC has the function of establishing dynamic compression ratio selection, which means that VC based on FIFO duty factor can determine the compression ratio. For example, when the MCU reading is fast and the FIFO is gradually emptied, VC compression is reduced, and when the MCU reading is slow and the FIFO is gradually full, the VC will be compressed as much as possible. But all these cannot avoid FIFO overflow or empty. When the FIFO is empty, an idle alarm will be generated. When the MCU is too slow to read, the FIFO will overflow and no alarm will be issued, but the video stream does not stop. In fact, this situation rarely occurs because the system can calculate the data rate. Now let's take a look at the amount of data, according to the resolution of VGA can have the following calculation:
Bayer pattern per frame: 640 × 480 = 307,200 (pixels) YUV per frame: 307,200 × 2B = 614,400B / 1024 (KB) = 600KB.
For example, transmitting 30fps in uncompressed mode means a data volume of up to 17.58MBps. Therefore, the only way is to use data compression. Since JPEG compression is a dynamic compression, the size cannot be predicted 100%. For example, you want to compress a frame at a ratio of 10: 1, which means that VC tries to achieve this ratio, but in fact it is sometimes easy to reach and sometimes not. It is worth mentioning that the amount of data calculated is not simply divided by 10 on the calculator. In view of this, VC uses a dynamic adjustment, so we can have a stable 4KB FIFO. Therefore, the compression ratio of our VC is about 80: 1 at the maximum, but the actual situation is that the effect of the compression ratio of 30: 1 is acceptable, and the higher will cause too much compression noise, which is unacceptable. Therefore, according to 30fps @ VGA transmission, the required data volume will be 600KBps.
This article summarizes
The implementation of the above system is very simple: first capture the image through the sensor, and then process Stv0676 and directly transfer the MJPEG to TF-520. TF-520 is a single-chip IP server, it only needs memory and Ethernet PHY, and other parts such as PCI interface, MAC and special IP for TCP / IP protocol stack processing have been integrated. If it is designed to be wireless, you only need to connect a miniPCI WLAN card to the system and change the firmware to get a wireless solution.
We will have further development plans in the product roadmap, one of which is to include audio and higher resolutions that may be up to two million pixels, which can be digitally focused and image stored, and can still be passed at 30fps VGA Internet delivery solution. IP cameras will definitely have brighter development prospects than USB cameras, and we will continue to introduce solutions that can simplify our lives.
TV Antenna,HDTV Antenna,TV Antenna Best,TV Antenna Walmart,TV Antenna digital
Yetnorson Antenna Co., Ltd. , https://www.yetnorson.com