I was thinking the same thing. Probably why the timeline is so fast too with only giving people a month’s notice of the API costs. And could also be true of twitter.
ChatGPT and other LLMs are gaining a lot of value from information freely available online and sites with large user generated text submissions like Reddit/twitter want a piece of the pie.
The problem with this line of thinking is “why is 3rd party API data any less valuable than 1st party API data for AI training?” While this may be true, I don’t see this particular move being motivated by AI. They still have all the API calls and interactions even if they aren’t being made by Reddit’s own apps.
I may be wrong here, but what I mean is they have ways to stop LLM companies from web scraping all of Reddit. The only other way the likes of chatGPT can get all the info is through to API which is currently free. So I think Reddit might be doing is saying this information isn’t free so pay X amount for access to our data.
Obviously 3rd party apps like Apollo won’t pay that, but Google and OpenAi probably will.
I’m not too sure what you mean by api data being worth less or more, it’s all the same data.
“TL;DR: Pushshift is in violation of our Data API Terms and has been unresponsive despite multiple outreach attempts on multiple platforms, and has not addressed their violations. Because of this, we are turning off Pushshift’s access to Reddit’s Data API, starting today”
I was thinking the same thing. Probably why the timeline is so fast too with only giving people a month’s notice of the API costs. And could also be true of twitter.
ChatGPT and other LLMs are gaining a lot of value from information freely available online and sites with large user generated text submissions like Reddit/twitter want a piece of the pie.
The problem with this line of thinking is “why is 3rd party API data any less valuable than 1st party API data for AI training?” While this may be true, I don’t see this particular move being motivated by AI. They still have all the API calls and interactions even if they aren’t being made by Reddit’s own apps.
I may be wrong here, but what I mean is they have ways to stop LLM companies from web scraping all of Reddit. The only other way the likes of chatGPT can get all the info is through to API which is currently free. So I think Reddit might be doing is saying this information isn’t free so pay X amount for access to our data.
Obviously 3rd party apps like Apollo won’t pay that, but Google and OpenAi probably will.
I’m not too sure what you mean by api data being worth less or more, it’s all the same data.
This makes sense. I get the argument now. Thanks!
Almost certainly true. Historically, you’d just grab the dumps from pushshift.
https://www.reddit.com/r/modnews/comments/134tjpe/reddit_data_api_update_changes_to_pushshift_access/
“TL;DR: Pushshift is in violation of our Data API Terms and has been unresponsive despite multiple outreach attempts on multiple platforms, and has not addressed their violations. Because of this, we are turning off Pushshift’s access to Reddit’s Data API, starting today”