Songsored

The idea

I’m building Songsored to find selected words in Moroccan rap audio, with the longer-term idea of using those detections to mute specific portions of a song. The current work focuses on detecting words in short vocal clips.

What I’m working on

The pipeline brings together audio curation, word alignment, mel-spectrogram features and a PyTorch CNN. I’ve also compared the CNN with a speech-encoder approach.

A large part of the work is reviewing false positives and checking whether a model recognises the target word or relies on something else in the recording.

Headline results

Across five training runs, frozen XLS-R features with a trained linear head reached 75.1% mean validation accuracy, compared with 62.9% for the log-mel CNN (BCE 0.554 versus 0.620).

Both used 1,297 one-second training clips from 449 recordings and 41 validation clips from 8 other recordings. Each run used 24,600 Adam updates; the lowest validation-loss checkpoint was selected. Recording-separated validation is small and repeatedly reused, not an independent test. Equal updates do not mean equal compute.

A useful failure case

In a later ordered-pooling experiment, moving the crop to place the target 200 ms earlier lowered recall from 85.6% to 22.4%. Shifted-crop training raised recall at that position to 52.8%, while original-position recall fell to 75.2%.

This diagnostic reuses the same 25 eligible positive clips from 21 recordings, with approximate word boundaries. It changes surrounding audio as well as word position and does not measure false alarms. The later experiment uses expanded training data, so it is distinct from the comparison above.

Where it stands

This is an ongoing personal project. Clip-level experiments and error analysis are distinct from a reliable full-song detector; the latter remains work in progress.