Title : Recovery and ranking of experimentally validated T-cell epitopes from HIV-1 p24, HCMV pp65, and Bordetella pertussis PtxS1 by HLA class I and II prediction tools
Abstract:
Background: Computational prediction of peptide–HLA interactions is widely used to prioritize candidate epitopes, yet workflow choices can alter which candidates are retained. We benchmarked commonly used HLA class I and II prediction workflows against curated positive reference datasets of experimentally supported peptide–HLA associations from HIV-1 p24, HCMV pp65, and Bordetella pertussis PtxS1.
Methods: Reference datasets contained 258, 73, and 10 unique HLA class I allele–peptide pairs and 95, 21, and 2 HLA class II (HLA-DR) pairs for p24, pp65, and PtxS1, respectively. Six HLA class I and five HLA class II prediction workflows were evaluated using threshold-based recovery, micro- and macro-averaged allele-level recovery, inter-tool overlap, and rank-based FRANK analysis. Comparative analyses were restricted to shared evaluable allele sets, with additional pairwise evaluability requirements for FRANK. IEDB prediction modes were combined for allele-level recovery estimates. FRANK measures the fraction of source-antigen peptides receiving better scores than a validated epitope; lower values indicate better prioritization. PtxS1 was evaluated descriptively because of limited reference data. Unmatched predictions were not classified as false positives because experimentally validated negative pairs were unavailable.
Results: Micro-averaged HLA class I recovery ranged from 12.3% to 28.1% for p24 and from 25.9% to 59.3% for pp65 across predictor categories. For HLA class II, combined IEDB workflows recovered 73.7% of evaluable p24 pairs, while all three predictor categories recovered 93.8% of pp65 pairs. Micro- and macro- averaged recovery estimates differed by no more than 8.3 percentage points across the evaluated datasets. Descriptive micro-averaged recovery for PtxS1 ranged from 50% to 100% in both HLA classes, although reference data were sparse. For p24 and pp65, prediction tools recovered partially overlapping sets of experimentally supported peptide–HLA pairs. Inter-tool similarity, measured using the Jaccard index and overlap coefficient, was lower when predictions absent from the curated reference datasets were also included. FRANK analysis showed better prioritization by IEDB NetMHCpan 4.1 EL than DTU NetMHCpan 4.1 for pp65 (Holm-adjusted p = 0.0009), and by IEDB NetMHCIIpan 4.1 EL than DTU NetMHCIIpan 4.1 for p24 (Holm-adjusted p = 0.047).
Conclusions: Implementation, scoring mode, HLA coverage, and selection criteria influence recovery and prioritization of experimentally supported peptide–HLA associations. Combining threshold-based and rank-based analyses while accounting for predictor evaluability provides a more informative basis for candidate selection before experimental validation.
