In numerous scenarios of forensic applications, only a partial face image is available for recognition purposes. Hence, the systems working on components of the face and partial face images have gained great importance. This paper proposes a novel approach for component-based face recognition and association under transfer learning and demonstrates that the knowledge gained from complete face images is transferred to classify components of the face. Three important components of a face, viz. ears, lips, and nose, are used for association and recognition. These components are unique, stable, and unaltered by the change in poses and expressions. Association is found between the face and nose, face and lips, as well as face and ears. Although, these face components and the face itself are from different domains, they share common information which is utilized to transfer the knowledge gained from one domain to another. Similarly, different types of half-faces are considered: left, right, upper, and lower half and left, right, upper, and lower diagonal for the association between complete and partial face images. Because of the association between a face and its different components, the proposed method can be applied to holistic face recognition, component-based face recognition and partial face recognition.