MA-Retriever

MA-Retriever is the dense retriever released with MetaSyn. It fine-tunes BAAI/bge-large-en-v1.5 for retrieving articles linked to the published included-study lists of systematic reviews, scoping reviews, and meta-analyses.

Resources

• Paper: arXiv:2606.17041
• Dataset: THUIR/MetaSyn
• Code and evaluator: THUIR/MetaSyn

Use

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BFTree/MA-Retriever")

def encode_protocol(research_question, population, intervention_or_exposure,
                    comparison, outcome):
    fields = [
        ("Research Question", research_question),
        ("Population", population),
        ("Intervention or Exposure", intervention_or_exposure),
        ("Comparison", comparison),
        ("Outcome", outcome),
    ]
    protocol = ". ".join(f"{name}: {value}" for name, value in fields if value)
    query = "Represent this sentence for searching relevant passages: " + protocol
    return model.encode(query, normalize_embeddings=True)

def encode_article(title, abstract):
    document = f"Title: {title}. Abstract: {abstract}"
    return model.encode(document, normalize_embeddings=True)

The protocol query concatenates the research question, Population, Intervention or Exposure, Comparison, and Outcome. It does not use the source-review title. Documents use title and abstract. Retrieval uses cosine similarity over normalized embeddings.

Training

We used 40 reviews sampled from the training split to select the epoch count. Recall@20 is the primary metric, with Recall@100 and Recall@200 used as tie-breakers. The released checkpoint was retrained for one epoch using the full training split with Multiple Negatives Ranking Loss, batch size 64, learning rate 2e-5, maximum sequence length 512, and seed 718. After excluding articles linked to a test review, 334 of the 336 training reviews contribute positive pairs.

There are 5,585 constructed training pairs. The sentence-transformers training loader used 5,568 examples in complete batches during the epoch; this accounts for the smaller sample count shown by automatically generated trainer metadata.

Test retrieval

All metrics are macro-averaged over the 86 held-out reviews after removing the source review itself from each ranking.

Metric	K=5	K=10	K=20	K=50	K=100	K=200
Recall@K	24.0%	38.8%	53.5%	75.3%	84.2%	91.7%
Precision@K	47.9%	42.0%	34.0%	22.6%	14.0%	8.4%

The source-review split is disjoint, and every article linked to a test review is excluded as a positive retriever-training example.

Limitations

The model is trained on reviews from Nature Portfolio and a PubMed-centered corpus. Performance may not transfer unchanged to other databases, languages, or domains. A high-recall pool still requires protocol-based screening; model scores should not be treated as inclusion decisions.

Citation

@misc{metasyn2026,
  title         = {Benchmarking {LLM} Agents on Meta-Analysis Articles from {Nature} Portfolio},
  author        = {Anzhe Xie and Weihang Su and Yujia Zhou and Yiqun Liu and Min Zhang and Qingyao Ai},
  year          = {2026},
  eprint        = {2606.17041},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2606.17041}
}