A method for enhancing the detection of fraudulent audio data is provided that includes capturing audio data of a user speaking during an authentication transaction, dividing the audio data into segments, determining a quality control vector for each segment, and determining whether each segment is of adequate quality. Moreover, the method includes calculating a voice replay score and a voice cloning detection score for each adequate quality segment, and determining, by a trained machine learning model operated by the electronic device, a weight for each adequate quality segment. Furthermore, the method includes applying the weight determined for each adequate quality segment to the voice replay and voice cloning scores calculated for the respective adequate quality segment, calculating a decision score, and comparing the decision score against a threshold value. In response to determining the decision score satisfies the threshold value, determining the captured audio data is genuine.