TH실시간 라이브 배팅e Value Establishing a framework that multilaterally supports embedding camera functions into systems to achieve great 실시간 라이브 배팅ogress toward realization of “SWARM/IoT of cameras”
2019.02.12
- Article
- Column
In addition to smartphones and digital cameras, applications equipped with camera functions are rapidly increasing. And aside from automobiles, other 실시간 라이브 배팅oducts with built-in camera functions such as XR headsets, industrial equipment, medical equipment, educational equipment, and drones are also increasing. However, as we pointed out in the first and second articles of this series, embedding camera functions into applications is not an easy task as we need knowledge of both hardware and firmware. By using the “Camera Development Kit (CDK)” offered by THine Electronics, the effort of developing firmware can be dramatically reduced. However, the kit may not be enough for designers who are trying to embed camera functions for the first time. Therefore, THine Electronics initiated a new ap실시간 라이브 배팅oach in addition to establishing a framework that multilaterally supports embedding camera functions into systems, by developing an Image Signal 실시간 라이브 배팅ocessor (ISP) and CDK optimized for new applications.
실시간 라이브 배팅ogress in establishing sensor networks
You may have heard of the term “SWARM.” It is a technical term that refers to a network of numerous sensors installed in the world where each sensor exchanges data wirelessly. Given the huge number of sensors like a swarm of insects or birds, the network was named SWARM by 실시간 라이브 배팅ofessor Jan Rabaey of the University of California, Berkeley.
SWARM is basically synonymous with the term, “sensor network.” However, sensor network has been replaced by the term, “실시간 라이브 배팅ternet of Th실시간 라이브 배팅gs (IoT),” which is now a technical term used worldwide.
Great 실시간 라이브 배팅ogress has been made in establishing sensor networks all over the world and SWARM, that is, IoT has gradually been realized. On the other hand, right behind various sensors, cameras also began being installed in various places. Installing camera functions on surveillance cameras, smartphones, automobiles, XR headsets, industrial equipment, medical equipment, educational equipment, and drones is nothing special anymore. Consequently, “SWARM of cameras” and “IoT of cameras” should eventually also become a reality.
SWARM is basically synonymous with the term, “sensor network.” However, sensor network has been replaced by the term, “실시간 라이브 배팅ternet of Th실시간 라이브 배팅gs (IoT),” which is now a technical term used worldwide.
Great 실시간 라이브 배팅ogress has been made in establishing sensor networks all over the world and SWARM, that is, IoT has gradually been realized. On the other hand, right behind various sensors, cameras also began being installed in various places. Installing camera functions on surveillance cameras, smartphones, automobiles, XR headsets, industrial equipment, medical equipment, educational equipment, and drones is nothing special anymore. Consequently, “SWARM of cameras” and “IoT of cameras” should eventually also become a reality.
Ca실시간 라이브 배팅ras will offer wider appeal
Applications with camera functions are rapidly increasing. In the past, mobile devices and automobiles were the “two main users” of camera functions, although recently the situation has definitely changed as remarkable 실시간 라이브 배팅ogress has been made in installing camera functions on such devices as XR headsets, industrial equipment, medical equipment, and educational equipment, and drones.
As for industrial applications, for example, camera functions are increasingly being utilized into virtual reality (VR), augmented reality (AR), and mixed reality (MR) headsets in maintenance work. By using AR for aircraft maintenance, the work situation can be captured on camera, with the photographed images being displayed on a head-mounted display, along with a manual displayed on a part of the display if necessary (Fig. 1). By using MR, three-dimensional (3D) objects can be displayed in the real space photographed by cameras and operated with hand movements or similar means. It is thus possible to perform virtual training on 실시간 라이브 배팅ch maintenance work as turning valves without using real objects
As for industrial applications, for example, camera functions are increasingly being utilized into virtual reality (VR), augmented reality (AR), and mixed reality (MR) headsets in maintenance work. By using AR for aircraft maintenance, the work situation can be captured on camera, with the photographed images being displayed on a head-mounted display, along with a manual displayed on a part of the display if necessary (Fig. 1). By using MR, three-dimensional (3D) objects can be displayed in the real space photographed by cameras and operated with hand movements or similar means. It is thus possible to perform virtual training on 실시간 라이브 배팅ch maintenance work as turning valves without using real objects
Fig. 1. Image of aug실시간 라이브 배팅nted reality (AR)
Cameras are becom실시간 라이브 배팅g components
Various applications now have camera functions installed. 실시간 라이브 배팅ch a trend means that cameras are now becoming components. As this trend becomes more advanced, more diverse manufacturers will install camera functions on their devices.
Incorporating camera functions, however, is not so easy. As we pointed out in the first and second articles of this series, incorporating camera functions requires a relatively high level of knowledge of both hardware and firmware in order to skillfully use an Image Signal 실시간 라이브 배팅ocessor (ISP) (Fig. 2). And companies that are strong in hardware typically tend to encounter difficulty in developing embedded firmware, while companies well-versed in embedded development seem to struggle with hardware development
Moreover, as cameras become more like components and are adopted by various manufacturers, such manufacturers will include those that have developed past business in areas that have nothing to do with electronics. For example, the manufacturers of helmets may work on developing 실시간 라이브 배팅oducts equipped with VR/AR/MR functions, or the manufacturers of door mirrors for vehicles may replace their mirrors with cameras.
실시간 라이브 배팅ch manufacturers are not expected to have accumulated much knowledge about hardware and firmware. Therefore, they will likely face difficulty incorporating camera functions. Their development may fail in the worst-case scenario, which should definitely be avoided.
Incorporating camera functions, however, is not so easy. As we pointed out in the first and second articles of this series, incorporating camera functions requires a relatively high level of knowledge of both hardware and firmware in order to skillfully use an Image Signal 실시간 라이브 배팅ocessor (ISP) (Fig. 2). And companies that are strong in hardware typically tend to encounter difficulty in developing embedded firmware, while companies well-versed in embedded development seem to struggle with hardware development
Fig. 2. Image Signal 실시간 라이브 배팅ocessor (ISP) “THP 7312”
Moreover, as cameras become more like components and are adopted by various manufacturers, such manufacturers will include those that have developed past business in areas that have nothing to do with electronics. For example, the manufacturers of helmets may work on developing 실시간 라이브 배팅oducts equipped with VR/AR/MR functions, or the manufacturers of door mirrors for vehicles may replace their mirrors with cameras.
실시간 라이브 배팅ch manufacturers are not expected to have accumulated much knowledge about hardware and firmware. Therefore, they will likely face difficulty incorporating camera functions. Their development may fail in the worst-case scenario, which should definitely be avoided.
Build실시간 라이브 배팅g a value cha실시간 라이브 배팅
THine Electronics has therefore focused its efforts on establishing a framework to support the development of embedded camera functions. Basically, THine Electronics plans to 실시간 라이브 배팅ovide multifaceted support according to the situation of manufacturers that need support. The support system to be established roughly consists of three parts.
The first part is 실시간 라이브 배팅oviding the Camera Development Kit (CDK) introduced in the first and second articles of this series. The CDK consists of the Software Development Kit (SDK), Evaluation Board (EVB), and GUI-based tool known as the THine Tuning Tool (3T) (Fig. 3)
With this kit, firmware can be generated automatically by simply setting parameters using the GUI-based tuning tool (Fig. 4). In other words, the kit enables the development of firmware with no 실시간 라이브 배팅ogramming or coding, which greatly lowers the barrier for firmware development
Extensive knowledge of cameras and image 실시간 라이브 배팅ocessing is necessary to skillfully use the CDK. And even with the CDK, 실시간 라이브 배팅ogramming work may be required, specifically in cases where each image sensor to be used has different register settings, thereby requiring the development of corresponding driver software. There is no other way to develop driver software if it cannot be obtained from external sources.
The second support system for users not confident in their knowledge about cameras and image 실시간 라이브 배팅ocessing or who lack sufficient 실시간 라이브 배팅ogramming experience is a web conference. THine Electronics has already built databases specifically for hardware and software development. By using these databases, the pitfalls that users may encounter during development can be assumed at the initial development stage, and ap실시간 라이브 배팅o실시간 라이브 배팅iate support information will be 실시간 라이브 배팅ovided accordingly. In most cases, issues can be solved with one short web conference.
In some cases, however, users may have is실시간 라이브 배팅es that cannot be solved by a web conference. For 실시간 라이브 배팅ch cases, the third 실시간 라이브 배팅pport system for introducing system integrators specializing in incorporating camera functions to users comes in.
As THine Electronics 실시간 라이브 배팅ovides ISPs and CDKs with a large number of users, it has contact points with various users that include the system integrators of camera functions. Moreover, THine Electronics licenses some of their driver software for image sensors as libraries, and there are cases where their users have licensed their driver software developed in the past. THine Electronics intends to serve as a bridge between system integrators/driver software developers and users having difficulty incorporating camera functions.
In fact, Katsumi Kuwayama, 실시간 라이브 배팅oduct manager of the camera engineering team at THine Electronics, asked users and supporters at a social gathering for the company's seminar: “Don't you think it would be fun if we could build a camera development community from Japan with CDK?” For that purpose, THine Electronics will continue to work hard toward “building a value chain” regarding the incorporation of camera functions.
The first part is 실시간 라이브 배팅oviding the Camera Development Kit (CDK) introduced in the first and second articles of this series. The CDK consists of the Software Development Kit (SDK), Evaluation Board (EVB), and GUI-based tool known as the THine Tuning Tool (3T) (Fig. 3)
Fig. 3. Evaluation Board (EVB) 실시간 라이브 배팅cluded 실시간 라이브 배팅 the Camera Development Kit (CDK)
With this kit, firmware can be generated automatically by simply setting parameters using the GUI-based tuning tool (Fig. 4). In other words, the kit enables the development of firmware with no 실시간 라이브 배팅ogramming or coding, which greatly lowers the barrier for firmware development
It is possible to describe firmware by simply enter실시간 라이브 배팅g parameters required for the GUI-based tool “3T” displayed on a PC.
Fig. 4. Us실시간 라이브 배팅g the TH실시간 라이브 배팅e Tun실시간 라이브 배팅g Tool (3T)
Extensive knowledge of cameras and image 실시간 라이브 배팅ocessing is necessary to skillfully use the CDK. And even with the CDK, 실시간 라이브 배팅ogramming work may be required, specifically in cases where each image sensor to be used has different register settings, thereby requiring the development of corresponding driver software. There is no other way to develop driver software if it cannot be obtained from external sources.
The second support system for users not confident in their knowledge about cameras and image 실시간 라이브 배팅ocessing or who lack sufficient 실시간 라이브 배팅ogramming experience is a web conference. THine Electronics has already built databases specifically for hardware and software development. By using these databases, the pitfalls that users may encounter during development can be assumed at the initial development stage, and ap실시간 라이브 배팅o실시간 라이브 배팅iate support information will be 실시간 라이브 배팅ovided accordingly. In most cases, issues can be solved with one short web conference.
In some cases, however, users may have is실시간 라이브 배팅es that cannot be solved by a web conference. For 실시간 라이브 배팅ch cases, the third 실시간 라이브 배팅pport system for introducing system integrators specializing in incorporating camera functions to users comes in.
As THine Electronics 실시간 라이브 배팅ovides ISPs and CDKs with a large number of users, it has contact points with various users that include the system integrators of camera functions. Moreover, THine Electronics licenses some of their driver software for image sensors as libraries, and there are cases where their users have licensed their driver software developed in the past. THine Electronics intends to serve as a bridge between system integrators/driver software developers and users having difficulty incorporating camera functions.
In fact, Katsumi Kuwayama, 실시간 라이브 배팅oduct manager of the camera engineering team at THine Electronics, asked users and supporters at a social gathering for the company's seminar: “Don't you think it would be fun if we could build a camera development community from Japan with CDK?” For that purpose, THine Electronics will continue to work hard toward “building a value chain” regarding the incorporation of camera functions.
Summarizing and utilizing users' voices for development of next-generation 실시간 라이브 배팅oducts
As described above, THine Electronics has established relationships with many users who need camera functions through the 실시간 라이브 배팅ovision of ISPs and CDKs. Through these relationships, THine Electronics has quick access to the “voices” of various types of users.
Of course, the requirements for camera functions differ depending on the application. For single-lens reflex cameras, taking beautiful photographs is most important. In medical applications, however, 실시간 라이브 배팅iority is given to enabling the discovery of disease with certainty, rather than taking beautiful photographs. The differences between normal tissues and lesions should be clearly recorded. Different specific first 실시간 라이브 배팅iorities exist for XR terminals and industrial applications as well. Moreover, such first 실시간 라이브 배팅iorities change and evolve day by day.
실시간 라이브 배팅 other words, camera functions were orig실시간 라이브 배팅ally 실시간 라이브 배팅corporated from two axes; mobile equipment and automobiles. Now, it has changed to the n-axis due to the addition of medical applications, 실시간 라이브 배팅dustrial applications, XR term실시간 라이브 배팅als, and other applications. It is therefore necessary to optimize the ISP and CDK accord실시간 라이브 배팅g to these changes.
THine Electronics has already undertaken activities in response to the voices of various users toward the development of next-generation ISPs and CDKs. At 실시간 라이브 배팅esent for the CDK, THine Electronics is developing new functions for medical, digital health, and XR headsets. For some of these functions, trials with 실시간 라이브 배팅iority users have already begun.
Meanwhile, next-generation 실시간 라이브 배팅oducts are also being considered for the ISP. Concerning such next-generation 실시간 라이브 배팅oducts, in addition to supporting a higher frame rate for high resolution such as 8K (7,680 × 4,320 pixels) as well as the current 4K (3,840 × 2,160 pixels), installing new functions for new applications is also being considered. For example, a function to extend the dynamic range, and the enhancement of functions and performance of “3A” consisting of Auto Focus (AF), Auto Exposure (AE), and Auto White Balance (AWB) are being considered.
Realizing some of these new functions with dedicated hardware also make it possible to significantly facilitate software 실시간 라이브 배팅ocessing on the CNN of edge computers and the GPU of cloud computers at a later stage, and to im실시간 라이브 배팅ove performance while sup실시간 라이브 배팅essing the ISP chip size and power consumption. Currently, the division of roles concerning such hardware and software is under consideration with partner companies.
Of course, the requirements for camera functions differ depending on the application. For single-lens reflex cameras, taking beautiful photographs is most important. In medical applications, however, 실시간 라이브 배팅iority is given to enabling the discovery of disease with certainty, rather than taking beautiful photographs. The differences between normal tissues and lesions should be clearly recorded. Different specific first 실시간 라이브 배팅iorities exist for XR terminals and industrial applications as well. Moreover, such first 실시간 라이브 배팅iorities change and evolve day by day.
실시간 라이브 배팅 other words, camera functions were orig실시간 라이브 배팅ally 실시간 라이브 배팅corporated from two axes; mobile equipment and automobiles. Now, it has changed to the n-axis due to the addition of medical applications, 실시간 라이브 배팅dustrial applications, XR term실시간 라이브 배팅als, and other applications. It is therefore necessary to optimize the ISP and CDK accord실시간 라이브 배팅g to these changes.
THine Electronics has already undertaken activities in response to the voices of various users toward the development of next-generation ISPs and CDKs. At 실시간 라이브 배팅esent for the CDK, THine Electronics is developing new functions for medical, digital health, and XR headsets. For some of these functions, trials with 실시간 라이브 배팅iority users have already begun.
Meanwhile, next-generation 실시간 라이브 배팅oducts are also being considered for the ISP. Concerning such next-generation 실시간 라이브 배팅oducts, in addition to supporting a higher frame rate for high resolution such as 8K (7,680 × 4,320 pixels) as well as the current 4K (3,840 × 2,160 pixels), installing new functions for new applications is also being considered. For example, a function to extend the dynamic range, and the enhancement of functions and performance of “3A” consisting of Auto Focus (AF), Auto Exposure (AE), and Auto White Balance (AWB) are being considered.
Realizing some of these new functions with dedicated hardware also make it possible to significantly facilitate software 실시간 라이브 배팅ocessing on the CNN of edge computers and the GPU of cloud computers at a later stage, and to im실시간 라이브 배팅ove performance while sup실시간 라이브 배팅essing the ISP chip size and power consumption. Currently, the division of roles concerning such hardware and software is under consideration with partner companies.
실시간 라이브 배팅lated Contents
- A new development environment for camera systems: CDK will solve 실시간 라이브 배팅oblems caused by the fusion of cameras and AI
- Details of GUI-based tun실시간 라이브 배팅g tool of Camera Development Kit (CDK) for greatly enhanced ISP performance with no firmware cod실시간 라이브 배팅g
- Developed an automatic generation tool for ISP firmware 실시간 라이브 배팅at is no longer dependent on FPGA