← Back to post

Edit history

Most recent

The pace of advancement is still going strong, especially in math and “reasoning”. For example the frontier math benchmark is showing big strides for the frontier models. A year ago they were only getting <10% of questions, now sol from openai is scoring 89% and they’re reporting the unreleased astra is at 98% .

They even had to create a new frontier math benchmark called erdos that all previous models scored 0% on and astra got 3% , which may not look like much but is a huge relative jump.

Original

The pace of advancement is still going strong, especially in math and “reasoning”. For example the frontier math benchmark is showing big strides for the frontier models. A year ago they were only getting <10% of questions, now sol from openai is scoring 89% and they’re reporting the unreleased astra is at 98% .