Papers
Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does the integration of synthetic chart data augmentations in instruction-tuned datasets like MMC-Instruction affect LMM generalization performance across ChartQA and FigureQA benchmarks,. 0 claims were…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What is the impact of FlashSpeech's zero-shot speaker adaptation on word error rate degradation when evaluated on out-of-domain emotional speech datasets like CREMA-D. 0 claims were extracted from source…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: Can the inference speedup demonstrated by FlashSpeech be replicated in large language models for code generation tasks without compromising HumanEval pass@1 scores. 10 claims were extracted from source…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does the scaling of instruction-tuned datasets (e.g., MMC-Instruction) beyond 1M instances influence the generalization of LMMs across different chart types, as measured by accuracy on benchmarks. 14 claims…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How does the top-1 accuracy of training-free k-NN classification using synthetic video features compare to fine-tuned baselines on the NVGesture dataset when evaluated across different large. 0 claims were…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: What is the impact of increasing the scale of MMC-Instruction (e.g., 1M vs. 600k instances) on the robustness of LMMs to distributional shifts in chart types, as measured by accuracy on unseen. 10 claims were…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: What is the impact of visual encoder resolution on the accuracy of multimodal models when interpreting complex diagrams in code generation tasks. 7 claims were extracted from source literature; 1 was…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does chain-of-thought prompting affect the robustness of large multimodal models against adversarial perturbations in diagram-based reasoning benchmarks. 0 claims were extracted from source literature; 0 were…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: Can training-free reasoning compression methods like ARS maintain performance on code generation tasks evaluated by HumanEval while reducing token usage. 0 claims were extracted from source literature; 0 were…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: How does fine-tuning MMC-Instruction-trained LMMs on domain-specific chart datasets improve their performance on benchmarks like ChartQA, as compared to zero-shot capabilities. 12 claims were extracted from…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: Can feature representations from synthetic video training generalize to unseen gesture classes in large pre-trained models without fine-tuning, as measured by top-1 accuracy on the NVGesture dataset. 0 claims…
Abstract: This report synthesises findings from 8 peer-reviewed papers addressing the following research question: How do open-source multimodal models compare to proprietary models on diagram-based coding benchmarks like HumanEval-V. 14 claims were extracted from source literature; 2 were independently verified against…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How robust are LMMs trained on MMC-Instruction to distributional shifts in chart types or domains, as quantified by cross-domain accuracy when tested on unseen chart datasets. 12 claims were extracted from source…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: Comprehensive comparison of frontier large language models on mathematical reasoning code generation and scientific knowledge v13. 18 claims were extracted from source literature; 0 were independently verified…
Abstract: This report synthesises findings from 12 peer-reviewed papers addressing the following research question: What are the state-of-the-art large language model results on reasoning benchmarks published recently v13. 16 claims were extracted from source literature; 1 was independently verified against retrieved…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: Which frontier language models achieve highest scores on GPQA Diamond Humanity Last Exam and difficult reasoning benchmarks v13. 14 claims were extracted from source literature; 0 were independently verified…
Abstract: This report synthesises findings from 11 peer-reviewed papers addressing the following research question: What is the relationship between language model perplexity and downstream reasoning task performance v13. 12 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does context length affect language model performance on multi-document reasoning and summarization v13. 10 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What are the limitations of current language model evaluation benchmarks for measuring reasoning v13. 10 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: How do language models compare to human experts on professional knowledge and science benchmarks v13. 15 claims were extracted from source literature; 7 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 15 peer-reviewed papers addressing the following research question: What is the effect of model size on language model performance on logical reasoning tasks v13. 9 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How do language models handle multi-hop reasoning chains in scientific question answering v13. 14 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: What prompting strategies maximize language model accuracy on graduate-level science questions v13. 12 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…
Abstract: This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does synthetic training data improve language model performance on mathematical reasoning benchmarks v13. 0 claims were extracted from source literature; 0 were independently verified against retrieved…
Abstract: This report synthesises findings from 16 peer-reviewed papers addressing the following research question: How does extended thinking time affect language model accuracy on competition-level mathematics v13. 0 claims were extracted from source literature; 0 were independently verified against retrieved documents. An…