[Eval Results] smollm2-360m-instruct on smalleval/mmlu-nano:mmlu_high_school_mathematics.jsonl - 24.0% Accuracy

[sauravpanda]

Evaluation Results

Metadata

• Dataset: smalleval/mmlu-nano:mmlu_high_school_mathematics.jsonl
• Model: smollm2-360m-instruct
• Evaluation Type: evaluation

Performance Metrics

• Accuracy: 24.0%
• Total Questions: 250
• Average Latency: 0.00ms
• Tokens/Second: 0.00
• Memory Usage: 265.22MB

System Information

• Browser: chrome 131
• OS: MacIntel
• CPU: 10 cores
• RAM: 8GB
• GPU: Not Available
• Timestamp: 2025-01-23 05:38:11

Evaluation Results Jan 23 2025.json

[sauravpanda]

insert

[sauravpanda]

insert

[github-actions]

Failed to create PR. Error: 403 {"message": "GitHub Actions is not permitted to create or approve pull requests.", "documentation_url": "https://docs.github.com/rest/pulls/pulls#create-a-pull-request", "status": "403"}