For Reviewers

Recently, various artificial intelligence (AI) applications have been widely implemented in commercial industries according to their requirements [1–3]. AI technology is the science behind machine intelligence. Traditional algorithms for a machine system set specific rules that define the output for each input case, whereas AI algorithms build their own rule systems to achieve better decisions for all cases. Therefore, machines with AI algorithms can self-solve tasks that were once entirely dependent on humans.

The advent of deep learning has strengthened the practicality of machine learning and expanded the realm of AI. As a sub-concept of machine learning, deep learning has been developed using artificial neural networks inspired by the structure and function of the brain [4]. Deep learning algorithms present the ability to learn and improve themselves based on observations and data provided; consequently, better decisions are achieved in each case without human intervention and assistance. Many deep learning algorithms [4,5] have been studied to enhance the efficiency of the learning process. In particular, the convolutional neural network (CNN) algorithm [6] has achieved significant progress in image recognition, which extracts and learns object features in multiple steps to generate a feature map. The performance of the CNN algorithm depends on the amount of data learned in various conditions. It is prone to making incorrect decisions and errors in unexpected conditions that are not learned.

In machine vision systems using deep learning algorithms, one or more image sensors are used to recognize object features with various edge detection methods [7-10]. Owing to the operating characteristics of the image sensor, the target object and its features are represented differently according to the light intensity condition. Thus, even with embedding advanced deep learning algorithms, the machine vision system cannot provide the right object features under conditions of unlearned light intensity. That is, the overall recognition performance is limited by the image sensors. For this reason, when we look closely to the imaging mechanism, the way to further improve object recognition performances may be obtained in a wider scope; if the image sensor can represent the target object regardless of the light condition surrounding the target object, the object recognition performance would be effectively increased even without the deep learning process for generating a feature map for all light intensities.

In this study, we propose an object recognition machine vision system based on deep learning with adopting, as a well-known technique in CISs, multiple-exposure sensing method. By effectively applying the multiple-exposure sensing technique [11-13], the prototype machine vision system increases the object recognition rate without deep learning information in high and low illumination conditions. If the proposed machine vision system fails to recognize the target object, the image sensor operates in a multiple-exposure sensing mode to detect the object that is blocked in the near dark or bright region. If the object is found, the captured images in the multiple-exposure mode are synthesized for a wide dynamic range of image information [14]. Accordingly, even without deep learning data in near dark or bright region, the object recognition rate can be effectively improved resulting in reducing the cost of data processing and computing resources.

The remained of this paper is organized as follows. Section 2 describes the proposed machine vision system applying the multiple-exposure sensing method. The experimental results and discussions of the prototype system are presented in Section 3, finally, the conclusions presented in Section 4.

Fig. 5 shows the prototype machine vision system model that provides information related to the target object. To verify the performance of the prototype system, deep learning was conducted using training and testing images in the ratio of 8:2. After supervised learning for image classification, the number of training repetitions was set to 200,000. To verify the object recognition performance according to the light intensity, the light intensity range of the light source should be larger than the dynamic range of the image sensor in the machine vision system. As shown in Fig. 6, a commercial light source (XT11X) was used to generate a wide light intensity range of 15–3200 lm. Here, the maximum dynamic range of the image sensor was approximately 60 dB. The various exposure times of the image sensor can be applied by adjusting its frame rate. When the frame rate is reduced, the ratio of the illumination time for imaging pixels is proportionally increased in the total image sensing time, resulting in long-exposure image sensing; conversely, when the frame rate is increased, the ratio of the illumination time is reduced, resulting in short-exposure image sensing.

Fig. 5. 
Prototype object recognition machine vision model.

Fig. 6. 
Light source for generating wide light intensity range.

Fig. 7 shows the measurement results of the prototype machine vision system applying the multiple-exposure imaging method. Compared to the conventional system operating only in the normal mode, the prototype system operates in the MES mode, and perceives the target object with a recognition rate of over 87% and 77% in the near-dark (under 2 lux) and bright (over 130,000 lux) conditions, respectively. The amount of light intensity at a distance of approximately 1 cm from the light source was measured using an illuminometer (TES-1335). In addition, the WDR image and its edge information are generated by synthesizing short-and long-exposure images captured in the MES mode. Thus, both dark and bright areas can be expressed regardless of the performance of the image sensors in the prototype system.

Fig. 7. 
Measurement results of the prototype machine vision system with multiple exposure imaging method.

Fig. 8 shows a graph of the amount of recognizable illuminance. Although deep learning was performed only with sample images in the light intensity range of 70 lux to 1,000 lux, the prototype machine vision system with the multiple-exposure image sensing method demonstrated a high object recognition performance from 2 lux up to a light intensity of over 140,000 lux. Here, the visible light intensity range of the machine vision system can be calculated as in [8]:

Fig. 8. 
Measured object recognition performance.

The measured accuracy of object recognition according to the light intensity is shown in Fig. 9. Compared to the typical machine vision system, when the MES method is applied, the better object recognition rate with high accuracy is verified in a wider light intensity range.

Fig. 9. 
Measured accuracy of object recognition according to the light intensity.

In general, the recognition rate and accuracy of the machine vision system depend on the amount of deep learning data and the performance of the image sensor. Nevertheless, even with the object recognition resources (based on deep learning data) with a light intensity range of only 23 dB (the light range of 70–1,000 lux), the prototype machine vision system with multiple-exposure image sensing method was verified to improve the object recognition performance with a light intensity of up to 96 dB (the light range of 2–140,000 lux). Moreover, by synthesizing the images captured in the MES mode, the prototype machine vision system provides a wide dynamic range performance beyond the dynamic range of the image sensor. Thus, this is a meaningful result because the prototype system can reduce the amount of deep learning data and represent object features beyond the dynamic range of the image sensor. The prototype machine vision system with the multiple-exposure image sensing method has advantages in terms of the cost of data processing and computing resources.

This work presents a prototype machine vision system with a high object recognition performance. By applying the multipleexposure sensing method, the prototype machine vision system can recognize the target object in a wide light intensity range without conducting the deep learning process under all light intensity conditions. Even in near-dark and bright conditions where the object feature is invisible, the prototype system shows a high object recognition performance. Thus, the proposed object recognition technique can be utilized to enhance the application of a machine vision system based on deep learning.