An image retrieval system is an image search engine is most useful for human day to day life. But still, the image retrieval systems are working in the traditional manner. Nowadays increasing the image database in a huge amount of size with the heterogeneous category. At the same time increasing the demands of users in a various manner. The most demanding fields in society are healthcare, agriculture, trademark, and crime. In healthcare field diagnose the disease, in crime field investigate the criminals, in trademark field to make an analysis to recognize the right product and in agriculture field to find the disease affected fruit images. The previous image retrieval systems are working with limitations like choose any one of the fields, any one of image file format and any one of the image feature and also any one of the medical image modality. To overcome the above limitations, the Pattern Based Image Retrieval (PBIR) system has been proposed. The PBIR system works by pattern recognition techniques. A pattern is a visible entity of image. Recognition is a label learning process. It considers mainly the patterns of the image for finding a similar group of images from the heterogeneous image database. And also include all directions of image features for image retrieval processing. The PBIR system is used to find uncertain parts of the image during augmentation steps. The augmentation work is an essential to increase the quality of images and make more reliable for feature extraction. Because currently we are using streaming image data. The images are having many unknown disturbances during capture the image. In this paper, we are mainly focusing image augmentation processes are a demosaicing algorithm, gray slice, gradient magnitude and pattern detection, Viola-Jones face detection algorithm for improving the PBIR system performance. And also PBIR system working with four image file formats are.JPG, BMP, GIF, PNG. The three main medical image modalities are CT scan, MRI scan and PET scan images. The image database of PBIR system is 3D DICOM MRI image, Brand logo image, Mango fruit image and ATM crime image. Finally, Image Quality Assessment (IQA) has been carried out using various evaluation measures are found better in the performance accuracy. And also include the results of this processes for further research directions.