CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations
Gabriel Meseguer-Brocal, Yuexuan Kong, Romain Hennequin
cs.SD · cs.AI · cs.LG · eess.AS · eess.SP
Quick verdict
Merges JEPA with contrastive learning on one shared backbone, no EMA teacher, reporting strong tonal MIR gains; convincing task-level wins but unsupported stability and efficiency claims.
Summary
CoJEPA unifies joint-embedding predictive architecture (JEPA) and contrastive learning in a single shared backbone that serves as both teacher and student, removing the EMA teacher. Contrastive loss (NT-Xent) is applied to the class token of two full views; JEPA loss drives a predictor to reconstruct latent targets of masked views. The authors pretrain a 7.1M-parameter ViT-1D on roughly 2M proprietary tracks and evaluate linear probes at layers 4, 8, and 12 on eight MIR tasks. CoJEPA beats or matches both individual methods at every depth on most tasks, with strong tonal/harmonic gains: key 0.640 weighted accuracy versus 0.550 (contrastive) and 0.245 (JEPA), chord F-measure 0.399 versus 0.258 and 0.211, tagging MAP 0.436. Unlike JEPA (early-layer peak) and contrastive (last-layer), it peaks at intermediate depth and surpasses the 46x larger MusicFM on key, pitch, and chords, while losing tagging and beat tracking. Key limitations: single-run results with one fixed seed and no variance; no ablations of loss weighting or contrastive temperature; fully proprietary pretraining data; and a MusicFM comparison restricted to its final layer.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Applying NT-Xent to the class token of a shared JEPA backbone regularizes the latent space enough to remove the EMA teacher entirely without collapse.
- CoJEPA outperforms or matches both JEPA and contrastive learning alone across most global and local MIR tasks and depths, with the largest gains on key (0.640 vs 0.550/0.245), chords (0.399 vs 0.258/0.211), and tagging (0.436 MAP).
- CoJEPA representations peak at intermediate depth (layer 8), a layer-wise behavior distinct from JEPA (early peak) and contrastive (last-layer peak).
- A 7.1M-parameter model with this combined objective beats the 330M-parameter MusicFM on key, pitch, and chords, supporting the thesis that objective design can substitute for scale.
- The combination requires no extra backbone parameters and no task-specific architectural changes.
Strengths
Parameter-free objective hybrid
Section 3.3 uses one shared backbone in teacher mode (full views, detached JEPA targets) and student mode (masked views), so contrastive gradients regularize the latent space and the EMA teacher is eliminated; the 3.5M predictor is the only added component (Section 4.3).
Layer-wise evaluation design
Probing at layers 4, 8, and 12 (Section 5.3) surfaces a replicable structural finding: JEPA peaks early (pitch 0.959 to 0.699 with depth), contrastive crystallizes at the last layer (key 0.243 to 0.550), and CoJEPA peaks at layer 8.
Large tonal/harmonic margins over both parents
Competitiveness at 7.1M parameters
Hedged, largely accurate claims in Section 6
Weaknesses
Central stability claim is asserted, not demonstrated
Missing ablations on the combining recipe
Single-run, single-seed evaluation of tight margins
Internal inconsistencies in the setup
Asymmetric scale comparison
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
CoJEPA — Structured Preliminary Review
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CoJEPA and how does it differ from standard JEPA?
CoJEPA is a self-supervised training recipe for music representation learning that combines a JEPA objective (predicting masked audio tokens in latent space) with a contrastive objective (NT-Xent on the class token) in a single shared backbone. Unlike standard JEPA, it does not use an EMA teacher: the contrastive loss stabilizes training, and both objectives act on the same model with no extra backbone parameters.
How well does CoJEPA perform compared to MusicFM?
At 7.1M parameters versus MusicFM's 330M, CoJEPA reports better results on key estimation (0.640 vs 0.558 weighted accuracy), pitch (0.983 vs 0.969), and chords (0.399 vs 0.193 F-measure), while MusicFM stays ahead on tagging (0.454 vs 0.436 MAP) and beat tracking (0.866 vs 0.793 F-measure). The comparison uses MusicFM's last layer only, which the authors themselves flag as potentially unfavorable.
Why does removing the EMA teacher matter?
EMA teacher-student setups in JEPA bring extra hyperparameters (momentum schedules), extra memory, and training instability from the moving-target dynamic. The paper argues that a contrastive loss on the class token provides the needed regularization directly, so one shared model can serve as both teacher and student. This stabilization claim is argued conceptually rather than demonstrated with ablations in the paper.
Is CoJEPA's code or pretraining data available?
No. The paper states that pretraining used an in-house dataset of about 2 million full tracks (Deezer data), and no code, model weights, or data release is mentioned, so full replication by external researchers is not currently possible.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table 2 arrives heavily fragmented from PDF extraction: column order (seq/CLS per model per layer) was reconstructed from the Section 6 narrative rather than read directly from table geometry, so only narrative-consistent values were cited in this review.
- Several sentences in the provided text end mid-word (e.g., the Literature review section ending 'learn represen-', the Class token description ending 'And it is'), so a few wording details of the method section could not be verified verbatim.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.