Folks at Middleware have tried to analyse Llama 3.1 vs GPT-4o. They ingest a lot of time-series data from live sources like Github.
They tried to analyse on:
- Mathematical Accuracy
- Data Analysis
- Actionability
- Summarisation
They have a DORA metrics product which show: Lead Time for Changes, Deployment Frequency, Mean Time to Recover (MTTR), Change Failure Rate (CFR) for engineering teams. (https://github.com/middleware/middleware)
I think still it is not the best or efficient way to compare. What do you think? Any suggestions on comparisions for specific use cases like these.
1 comment
[ 4.9 ms ] story [ 14.0 ms ] threadThey tried to analyse on:
- Mathematical Accuracy - Data Analysis - Actionability - Summarisation
They have a DORA metrics product which show: Lead Time for Changes, Deployment Frequency, Mean Time to Recover (MTTR), Change Failure Rate (CFR) for engineering teams. (https://github.com/middleware/middleware)
I think still it is not the best or efficient way to compare. What do you think? Any suggestions on comparisions for specific use cases like these.