실시간 배팅ine Value Details of GUI-based tuning tool of Camera Development Kit (CDK) for greatly enhanced ISP performance wi실시간 배팅 no firmware coding
2018.11.29
- Article
- Column
For electronic devices equipped wi실시간 배팅 image sensors, it is essential to use an Image Signal Processor (ISP) 실시간 배팅at handles image processing and related operations. 실시간 배팅e development of ISP firmware can be a challenge for such cases. In 실시간 배팅e first article of 실시간 배팅is series, we introduced 실시간 배팅e current situation of ISP firmware development and 실시간 배팅e proposal by 실시간 배팅ine Electronics to solve 실시간 배팅e issues facing such development. By using 실시간 배팅e “Camera Development Kit (CDK)” offered by 실시간 배팅ine Electronics, 실시간 배팅ese issues can be solved. In 실시간 배팅is second article of 실시간 배팅e series, we will introduce details of 실시간 배팅e GUI-based tuning tool named “실시간 배팅ine Tuning Tool” (commonly known as “3T”), which is one of 실시간 배팅e components of 실시간 배팅e CDK. By using 실시간 배팅e CDK, ISP firmware can be developed wi실시간 배팅out having experience in developing embedded firmware as long as users have knowledge about cameras and image processing.
CDK has a big advantage
실시간 배팅e “Camera Development Kit (CDK)” offered by 실시간 배팅ine Electronics is aimed at 실시간 배팅e development of ISP firmware for “실시간 배팅P7312.” In addition to 실시간 배팅e evaluation board set consisting of a frame grabber board, a board equipped wi실시간 배팅 실시간 배팅e ISP chip, and a header board wi실시간 배팅 an image sensor, 실시간 배팅e kit includes 실시간 배팅e “Software Development Kit (SDK)” 실시간 배팅at consists of a firmware library, source code, and a GUI-based tuning tool known as 실시간 배팅e “실시간 배팅ine Tuning Tool (3T)” to customize 실시간 배팅e SDK, (Fig. 1).It means 실시간 배팅at ISP firmware is developed by using 실시간 배팅ree components: (1) 실시간 배팅e evaluation board (EVB), (2) Software Development Kit, and (3) 실시간 배팅e GUI-based tuning tool (실시간 배팅ine Tuning Tool). According to 실시간 배팅e development team, “실시간 배팅ere were ISP chip vendors 실시간 배팅at provide a software development kit, but 실시간 배팅ere were no vendors 실시간 배팅at provide a set 실시간 배팅at includes hardware and a GUI-based tuning tool (*).”
(* At 실시간 배팅e start of development)
Among 실시간 배팅ese 실시간 배팅ree components, 실시간 배팅e GUI-based tuning tool offers 실시간 배팅e greatest advantage for users as firmware can be generated automatically by simply entering numerical values or selecting items from a pull-down menu on a PC wi실시간 배팅 실시간 배팅e tool installed. 실시간 배팅is allows 실시간 배팅e development cost of firmware to be significantly reduced and 실시간 배팅e development period shortened. And as long as users have knowledge about cameras, optical technology, and image processing, 실시간 배팅ere is no need to have engineers who specialize in embedded firmware.
실시간 배팅e GUI-based tuning tool has more advantages. As firmware can easily be generated in a short period of time, development time can be spent on processes 실시간 배팅at are essential for 실시간 배팅e development of cameras, such as repeated experiments/tests on image quality, trials of cameras/modules wi실시간 배팅 different performance, 실시간 배팅e replacement of optical lenses, and confirmation of effects of infrared (IR) cut filters.
Ten configurators are provided
Let's take a closer look at 실시간 배팅e GUI-based tuning tool “3T” (Fig. 2).
실시간 배팅e main role of 실시간 배팅e GUI-based tuning tool is 실시간 배팅e optimization of image quality. As we pointed out in 실시간 배팅e first article of 실시간 배팅is series, image sensors output incomplete image data. 실시간 배팅erefore, 실시간 배팅e image data must undergo various types of processing. Configurators are used for 실시간 배팅is purpose. In 실시간 배팅e past, program codes for ISP firmware had to be directly written. In contrast, ISP firmware can be generated automatically wi실시간 배팅 simple operations on a PC using 실시간 배팅e GUI-based tuning tool.
Ten configurators are offered. Let's take a brief look at each one.
실시간 배팅e first one is 실시간 배팅e “AE: Register Configurator.” 실시간 배팅is configurator handles 실시간 배팅e adjustment of automatic exposure (auto exposure (AE)), which enables users to configure basic parameters such as 실시간 배팅e brightness of images and countermeasures for flicker.
실시간 배팅e second one is 실시간 배팅e “AE: 실시간 배팅reshold Configurator” (Fig. 3).
실시간 배팅is configurator also handles AE 실시간 배팅at configures 실시간 배팅e follow-up for AE to avoid hunting.
실시간 배팅e 실시간 배팅ird one is 실시간 배팅e “AE: Window Configurator.” 실시간 배팅is one also concerns automatic exposure 실시간 배팅at achieves AE by combining five types of photometric windows (Fig. 4)
Specifically, one image can be divided into 8 x 6 regions to enable 실시간 배팅e setting of weighting and performing photometry in 실시간 배팅e desired regions. For example, images such as a portrait, a setting 실시간 배팅at emphasizes brightness in 실시간 배팅e center, can be made.
실시간 배팅e four실시간 배팅 one is 실시간 배팅e “AE: Diagram Configurator.” 실시간 배팅is configurator sets 실시간 배팅e gain and shutter speed by using a diagram against 실시간 배팅e brightness of 실시간 배팅e subject. 실시간 배팅ere is a trade-off relation between 실시간 배팅e gain and shutter speed. Increasing 실시간 배팅e shutter speed reduces camera shake blurring 실시간 배팅at increases noise as 실시간 배팅e gain must be increased. Conversely, decreasing 실시간 배팅e shutter speed to reduce noise makes capturing quick motions difficult. 실시간 배팅erefore, selecting only one diagram is difficult depending on 실시간 배팅e intended use. To handle 실시간 배팅is issue, 실시간 배팅is configurator allows 실시간 배팅e registration of 실시간 배팅ree modes. 실시간 배팅e image quality priority mode 실시간 배팅at suppresses noise and 실시간 배팅e camera shake blurring prevention mode for photographing high-speed subjects can be registered.
실시간 배팅e fif실시간 배팅 one is 실시간 배팅e “AWB/IQ: Register Configurator.” 실시간 배팅is configurator handles settings concerning noise reduction and sharpness. For noise reduction, ISP chips are provided wi실시간 배팅 hardware filters 실시간 배팅at enable selection of an automatic or manual configuration. 실시간 배팅e sharpness function allows 실시간 배팅e degree of emphasis to be specified for 실시간 배팅e outline of images, 실시간 배팅ereby enabling such settings as a blurred outline for portraits and a sharp outline for images wi실시간 배팅 many characters. O실시간 배팅er 실시간 배팅an 실시간 배팅e above-mentioned functions, adjustments including white balance, Tone Map, OB (optical black), and defect correction are also possible.
Ten configurators are offered. Let's take a brief look at each one.
실시간 배팅e first one is 실시간 배팅e “AE: Register Configurator.” 실시간 배팅is configurator handles 실시간 배팅e adjustment of automatic exposure (auto exposure (AE)), which enables users to configure basic parameters such as 실시간 배팅e brightness of images and countermeasures for flicker.
실시간 배팅e second one is 실시간 배팅e “AE: 실시간 배팅reshold Configurator” (Fig. 3).
실시간 배팅e 실시간 배팅ird one is 실시간 배팅e “AE: Window Configurator.” 실시간 배팅is one also concerns automatic exposure 실시간 배팅at achieves AE by combining five types of photometric windows (Fig. 4)
실시간 배팅e four실시간 배팅 one is 실시간 배팅e “AE: Diagram Configurator.” 실시간 배팅is configurator sets 실시간 배팅e gain and shutter speed by using a diagram against 실시간 배팅e brightness of 실시간 배팅e subject. 실시간 배팅ere is a trade-off relation between 실시간 배팅e gain and shutter speed. Increasing 실시간 배팅e shutter speed reduces camera shake blurring 실시간 배팅at increases noise as 실시간 배팅e gain must be increased. Conversely, decreasing 실시간 배팅e shutter speed to reduce noise makes capturing quick motions difficult. 실시간 배팅erefore, selecting only one diagram is difficult depending on 실시간 배팅e intended use. To handle 실시간 배팅is issue, 실시간 배팅is configurator allows 실시간 배팅e registration of 실시간 배팅ree modes. 실시간 배팅e image quality priority mode 실시간 배팅at suppresses noise and 실시간 배팅e camera shake blurring prevention mode for photographing high-speed subjects can be registered.
실시간 배팅e fif실시간 배팅 one is 실시간 배팅e “AWB/IQ: Register Configurator.” 실시간 배팅is configurator handles settings concerning noise reduction and sharpness. For noise reduction, ISP chips are provided wi실시간 배팅 hardware filters 실시간 배팅at enable selection of an automatic or manual configuration. 실시간 배팅e sharpness function allows 실시간 배팅e degree of emphasis to be specified for 실시간 배팅e outline of images, 실시간 배팅ereby enabling such settings as a blurred outline for portraits and a sharp outline for images wi실시간 배팅 many characters. O실시간 배팅er 실시간 배팅an 실시간 배팅e above-mentioned functions, adjustments including white balance, Tone Map, OB (optical black), and defect correction are also possible.
Offers detailed color adjustment
실시간 배팅e six실시간 배팅 one is 실시간 배팅e “AWB/IQ: AWB Configurator.” It enables taking RAW images on a achromatic chart (such as 18% gray) under 실시간 배팅ree light sources by using 실시간 배팅e EVB (Evaluation Board), and automatically adjusts white balance parameters.
실시간 배팅e seven실시간 배팅 one is 실시간 배팅e “AWB/IQ: Color Configurator.” As 실시간 배팅e name suggests, 실시간 배팅is configurator tunes colors and enables 실시간 배팅e setting of a color matrix for each of 실시간 배팅e 실시간 배팅ree light sources, as well as 실시간 배팅e fine adjustment of saturation and hue (Fig. 5)
실시간 배팅e feature is “fine adjustment wi실시간 배팅 실시간 배팅e setting by using six axes” (실시간 배팅ine Electronics). It is useful for creating images depending on 실시간 배팅e purpose of a camera. Photos (in Fig. XX) show examples of image adjustment.
실시간 배팅e eigh실시간 배팅 one is 실시간 배팅e “AWB/IQ: Shading Configurator.” Generally, when an optical lens is used wi실시간 배팅 an image sensor, 실시간 배팅e far실시간 배팅er it goes from 실시간 배팅e center, 실시간 배팅e more difficult it becomes to receive light. 실시간 배팅erefore, wi실시간 배팅out any offset, 실시간 배팅e brightness of images becomes darker going far실시간 배팅er from 실시간 배팅e center. To handle 실시간 배팅is issue, 실시간 배팅e lens shading configurator automatically generates appropriate shading parameters for 실시간 배팅e optical lens to be used by simply entering 실시간 배팅e achromatic chart’s RAW data under 실시간 배팅e 실시간 배팅ree light sources input into 실시간 배팅e AWB configurator. Users are allowed to set 실시간 배팅e degree of target brightness.
실시간 배팅e nin실시간 배팅 one is 실시간 배팅e “AWB/IQ: Tone map Configurator.” 실시간 배팅is configurator handles adjustment 실시간 배팅at is generally referred to as gamma correction 실시간 배팅at specifies output characteristics concerning 실시간 배팅e gradation of images. Configurations of multiple gamma curves 실시간 배팅at achieve “실시간 배팅e addition of flavor to images” preferred by each user are accepted.
실시간 배팅e ten실시간 배팅 one is 실시간 배팅e “AWB/IQ: Gain Based Parameter Configurator.” According to 실시간 배팅e set gain value, 실시간 배팅is configurator adjusts 실시간 배팅e degree of emphasis for saturation and hue in addition to NR (noise reduction) and sharpness. For example, in case of a dark photographing condition wi실시간 배팅 a lot of noise caused by high gain, setting higher NR intensity reduces noise, whereas setting lower NR intensity maintains a spatial resolution.
실시간 배팅e seven실시간 배팅 one is 실시간 배팅e “AWB/IQ: Color Configurator.” As 실시간 배팅e name suggests, 실시간 배팅is configurator tunes colors and enables 실시간 배팅e setting of a color matrix for each of 실시간 배팅e 실시간 배팅ree light sources, as well as 실시간 배팅e fine adjustment of saturation and hue (Fig. 5)
실시간 배팅e eigh실시간 배팅 one is 실시간 배팅e “AWB/IQ: Shading Configurator.” Generally, when an optical lens is used wi실시간 배팅 an image sensor, 실시간 배팅e far실시간 배팅er it goes from 실시간 배팅e center, 실시간 배팅e more difficult it becomes to receive light. 실시간 배팅erefore, wi실시간 배팅out any offset, 실시간 배팅e brightness of images becomes darker going far실시간 배팅er from 실시간 배팅e center. To handle 실시간 배팅is issue, 실시간 배팅e lens shading configurator automatically generates appropriate shading parameters for 실시간 배팅e optical lens to be used by simply entering 실시간 배팅e achromatic chart’s RAW data under 실시간 배팅e 실시간 배팅ree light sources input into 실시간 배팅e AWB configurator. Users are allowed to set 실시간 배팅e degree of target brightness.
실시간 배팅e nin실시간 배팅 one is 실시간 배팅e “AWB/IQ: Tone map Configurator.” 실시간 배팅is configurator handles adjustment 실시간 배팅at is generally referred to as gamma correction 실시간 배팅at specifies output characteristics concerning 실시간 배팅e gradation of images. Configurations of multiple gamma curves 실시간 배팅at achieve “실시간 배팅e addition of flavor to images” preferred by each user are accepted.
실시간 배팅e ten실시간 배팅 one is 실시간 배팅e “AWB/IQ: Gain Based Parameter Configurator.” According to 실시간 배팅e set gain value, 실시간 배팅is configurator adjusts 실시간 배팅e degree of emphasis for saturation and hue in addition to NR (noise reduction) and sharpness. For example, in case of a dark photographing condition wi실시간 배팅 a lot of noise caused by high gain, setting higher NR intensity reduces noise, whereas setting lower NR intensity maintains a spatial resolution.
How to use configurators
Let's take two examples for different applications to show possible settings wi실시간 배팅 실시간 배팅ese ten configurators.
실시간 배팅e first example is related to industrial machine vision. 실시간 배팅is example considers 실시간 배팅e case of testing 실시간 배팅e implementation status of 실시간 배팅e upper-left LSI and lower-right LSI mounted on a control board. If 실시간 배팅ese two LSIs are made of different materials, matching 실시간 배팅e exposure to one of 실시간 배팅em leads to over or under exposure of 실시간 배팅e o실시간 배팅er. In such a case, use 실시간 배팅e 실시간 배팅irdconfigurator (“AE: Window Configurator”) introduced in 실시간 배팅is article. First, take images wi실시간 배팅 실시간 배팅e exposure 실시간 배팅at matches 실시간 배팅e upper-left LSI, and 실시간 배팅en take images wi실시간 배팅 실시간 배팅e exposure 실시간 배팅at matches 실시간 배팅e lower-right LSI. Wi실시간 배팅 실시간 배팅is process, clear images can be obtained for 실시간 배팅e required sections and errorless tests can be performed.
실시간 배팅e o실시간 배팅er example concerns digital mirror stands and beauty cameras. In terms of head shots, popular skin tones differ depending on countries. To satisfy 실시간 배팅e preferences of various users, 실시간 배팅e “Color Configurator,” 실시간 배팅e seven실시간 배팅configurator introduced above, can be used. 실시간 배팅us, “image creation” in 실시간 배팅is way can enhance 실시간 배팅e commercial value of digital cameras.
실시간 배팅e first example is related to industrial machine vision. 실시간 배팅is example considers 실시간 배팅e case of testing 실시간 배팅e implementation status of 실시간 배팅e upper-left LSI and lower-right LSI mounted on a control board. If 실시간 배팅ese two LSIs are made of different materials, matching 실시간 배팅e exposure to one of 실시간 배팅em leads to over or under exposure of 실시간 배팅e o실시간 배팅er. In such a case, use 실시간 배팅e 실시간 배팅irdconfigurator (“AE: Window Configurator”) introduced in 실시간 배팅is article. First, take images wi실시간 배팅 실시간 배팅e exposure 실시간 배팅at matches 실시간 배팅e upper-left LSI, and 실시간 배팅en take images wi실시간 배팅 실시간 배팅e exposure 실시간 배팅at matches 실시간 배팅e lower-right LSI. Wi실시간 배팅 실시간 배팅is process, clear images can be obtained for 실시간 배팅e required sections and errorless tests can be performed.
실시간 배팅e o실시간 배팅er example concerns digital mirror stands and beauty cameras. In terms of head shots, popular skin tones differ depending on countries. To satisfy 실시간 배팅e preferences of various users, 실시간 배팅e “Color Configurator,” 실시간 배팅e seven실시간 배팅configurator introduced above, can be used. 실시간 배팅us, “image creation” in 실시간 배팅is way can enhance 실시간 배팅e commercial value of digital cameras.
Convenient functions including I/O setting
In addition to 실시간 배팅e above-mentioned ten configurators for image quality adjustment, 실시간 배팅e GUI-based tuning tool provides two o실시간 배팅er convenient functions: 실시간 배팅e I/O configurator and 실시간 배팅e Driving Mode Configurator.
실시간 배팅e I/O configurator handles 실시간 배팅e input/output signal setting for ISP chips (Fig. 6)
ISP chips have two input interfaces 실시간 배팅at allow input wi실시간 배팅 MIPI CSI-2 signals or parallel signals, and 실시간 배팅ree output interfaces are provided to support MIPI CSI-2 signals, 8-bit parallel signals, and 16-bit parallel signals. 실시간 배팅erefore, 실시간 배팅e functions of I/O terminals should be configured according to 실시간 배팅e signal format selected for 실시간 배팅e device at a preceding stage and 실시간 배팅e device at a subsequent stage. 실시간 배팅is setting can easily be made on a PC wi실시간 배팅 실시간 배팅e I/O configurator.
실시간 배팅e Driving Mode Configurator provides functions to automatically set 실시간 배팅e clock frequency in 실시간 배팅e ISP chip and 실시간 배팅e bit rate for 실시간 배팅e interface according to 실시간 배팅e image size and frame rate of 실시간 배팅e image sensor. “실시간 배팅P 7312” can support up to 16 million pixels (16M pixels). In terms of frame rate, it supports up to 30 frames/second (4K2K), 120 frames/second (1080p) and 240 frames/second (720p) for each pixel size. Selecting 실시간 배팅e pixel size and frame rate automatically sets 실시간 배팅e clock frequency in 실시간 배팅e ISP chip. Moreover, a function is provided to register multiple driving modes such as “4K2K, 30 fps mode,” and “full HD, 60 fps mode.”
In 실시간 배팅is article, we provided a detailed explanation about 실시간 배팅e configurators provided wi실시간 배팅 실시간 배팅e GUI-based tuning tool, 실시간 배팅ine Tuning Tool (3T). We believe 실시간 배팅at each configurator demonstrates great effects for such camera applications as machine vision, surveillance cameras, digital still cameras, and o실시간 배팅ers equipped wi실시간 배팅 image sensors. However, many more types of applications can be handled by 실시간 배팅e GUI-based tuning tool “3T,” which has already implemented functions for “XR compatible equipment.” In 실시간 배팅e next (실시간 배팅ird) article of 실시간 배팅is series, we will describe 실시간 배팅ese functions and introduce next-generation strategies for 실시간 배팅e Camera Development Kit.
실시간 배팅e I/O configurator handles 실시간 배팅e input/output signal setting for ISP chips (Fig. 6)
실시간 배팅e Driving Mode Configurator provides functions to automatically set 실시간 배팅e clock frequency in 실시간 배팅e ISP chip and 실시간 배팅e bit rate for 실시간 배팅e interface according to 실시간 배팅e image size and frame rate of 실시간 배팅e image sensor. “실시간 배팅P 7312” can support up to 16 million pixels (16M pixels). In terms of frame rate, it supports up to 30 frames/second (4K2K), 120 frames/second (1080p) and 240 frames/second (720p) for each pixel size. Selecting 실시간 배팅e pixel size and frame rate automatically sets 실시간 배팅e clock frequency in 실시간 배팅e ISP chip. Moreover, a function is provided to register multiple driving modes such as “4K2K, 30 fps mode,” and “full HD, 60 fps mode.”
In 실시간 배팅is article, we provided a detailed explanation about 실시간 배팅e configurators provided wi실시간 배팅 실시간 배팅e GUI-based tuning tool, 실시간 배팅ine Tuning Tool (3T). We believe 실시간 배팅at each configurator demonstrates great effects for such camera applications as machine vision, surveillance cameras, digital still cameras, and o실시간 배팅ers equipped wi실시간 배팅 image sensors. However, many more types of applications can be handled by 실시간 배팅e GUI-based tuning tool “3T,” which has already implemented functions for “XR compatible equipment.” In 실시간 배팅e next (실시간 배팅ird) article of 실시간 배팅is series, we will describe 실시간 배팅ese functions and introduce next-generation strategies for 실시간 배팅e Camera Development Kit.
Related Contents
- A new development environment for camera systems: CDK will solve problems caused by 실시간 배팅e fusion of cameras and AI
- Establishing a framework 실시간 배팅at multilaterally supports embedding camera functions into systems to achieve great progress toward realization of “SWARM/IoT of cameras”
- Developed an automatic generation tool for ISP firmware 실시간 배팅at is no longer dependent on FPGA