LLMs do suck at math, if you look into it, the o1 models actually escape the LLM output and write a python function to calculate the output, I’ve been able to break their math functions by asking for functions that use math not in the standard Python library.
I know someone also wrote a wolfram integration to help solve LLMs math problems.
Terrence Tao (one of the most famous and active mathematician) recently wrote his thoughts in Mastodon on o1 mathematical capabilities. Interesting read: https://mathstodon.xyz/@tao/113132502735585408
LLMs do suck at math, if you look into it, the o1 models actually escape the LLM output and write a python function to calculate the output, I’ve been able to break their math functions by asking for functions that use math not in the standard Python library.
I know someone also wrote a wolfram integration to help solve LLMs math problems.
Wow that’s really clever actually. Basically using the library as digital scratch paper
Terrence Tao (one of the most famous and active mathematician) recently wrote his thoughts in Mastodon on o1 mathematical capabilities. Interesting read: https://mathstodon.xyz/@tao/113132502735585408