Artificial neural network (ANN) which has four or more layers structure is called deep NN (DNN) and it is recognized as a promising machine learning technique. Convolutional neural network (CNN) is widely used and powerful structure for image recognition. It is also known that support vector machine (SVM) has a superior ability for binary classification in spite of only having two layers. The authors already have developed a CNN&SVM design and training tool for defect detection of resin molded articles, while the effectiveness and the validity have been proved through several CNNs design, training and evaluation. The tool further enables to facilitate the design of a CNN model based on transfer learning concept. In this paper, a pick and place robot is introduced while implementing a visual feedback control and a transfer learning-based CNN. The visual feedback control enables to omit the complicated calibration between image and robot coordinate systems, also the transfer learning-based CNN allows the robot to estimate the orientation of target objects for dexterous picking operation. The effectiveness of the system is evaluated through experimental pick and place tests using an articulated robot named DOBOT.