Pretty much the question. I heard about Usenet a while back but never managed to wrap my head around it.
I’ll try to give an ELI5 kind of answer here.
Before the Internet, “networks” were mostly one-offs you would dial into with a modem. Big or small, users would dial into the systems to enjoy whatever content was available on them.
The Internet was created as a way to connect multiple, disparate network nodes like these. Now, instead of just letting people access your content, you could now let them access other people’s content as well.
There were lots of programs made to do this. IRC for chatting, Archie and Gopher for searching FTP sites for downloads you might want. There was also Usenet - a threaded discussion forum. The discussions looked a lot like Lemmy - there were subject lines and when you clicked on them there was threaded discussion you could read and participate in.
When this was all initially going on the Internet was mostly text-based. We may have been accessing Usenet from our Windows 3.1 laptops (I used a program called Agent), but all these programs were doing was trading text. Slowly though, bandwidth started creeping up.
As bandwidth began to creep up, people realized that huge text posts to Usenet could be used to post things like photos encoded to text. And thus was uuencoding born - and it didn’t stop at photos. But because Usenet posts are limited in size, big files would get posted as multiple parchives - in multiple sections/posts that could be stitched back together into a whole again.
It was in this way that Usenet - a system designed for conversation - became a way to trade files.
Meanwhile the web happened. Discussion quickly moved to the web because you didn’t have to download a separate program to view web forums. At the time, web forums were inherently inferior (they couldn’t do threaded discussion) but they were also inherently superior (they could be moderated). Yeah, Usenet was unmoderated and because of this it was basically a huge pile of dogshit by the time the web got huge.
Usenet did continue to flourish though - as this sort of Frankenstein file-sharing system. The problem is that most Usenet servers were hosted by ISPs because they wanted to host discussions - not file-sharing. So they shut their Usenet servers down. But the file sharing was just too useful to die, so dedicated Usenet providers popped up and picked up the slack where the local ISPs left off. It wasn’t hard. Usenet is just a protocol - anybody can adhere to it and create a node.
And clients changed too - from the readers I used like Agent, to new readers that recognized that people using Usenet aren’t looking for discussion anymore. They’re looking for an easy way to find the files they want and a program that will seamlessly stitch together all those PAR files behind the scenes for them to get it.
This was the purpose behind Newzbin, which was an elaborate way to access the remaining Federation of (now mostly dedicated, paid) Usenet servers and easily find and download all they had to offer. It was super easy and worked very well, so naturally, it was fucked into oblivion by Hollywood in 2010.
The great thing about Usenet though, is you can’t kill it by killing off one node. The other great thing is that it’s pretty stupidly complicated by today’s standards, so it still exists because it’s been largely forgotten while Hollywood focuses on stuff like torrenting.
If you want to access Usenet, you will need to purchase access to a company that runs a Usenet server and get client software that can help you find and stitch together those PAR files. I am out of the loop, so I am afraid I cannot help you any further with that. But hopefully if you know the history of it and how it works in theory, it should help.
Bravo! Outstanding explanation. I got lost on like the second sentence though. If you have the time, can you also ELI5
dial into with a modem
It’s a common enough expression but what does it even mean?
This is it. This is the comment that makes me realize that I’m old.
Wow, this caught me off guard. I read an article about kids being given old tech and they knew people waved Polaroids (even though it doesn’t enhance development) from pop culture. Are young people really unaware that we used to dial with a modem to connect to the net?
No I meant do analog modems use tones to transmit information, or how does it work?
Like what is the process of “dial with a modem to connect to the net”.
Modem is short for modulator/demodulator. It took a data stream and encoded it onto tones, multiple (audible) frequencies were used to increase data rates. You had to dial in to either a server, another computer, or an ISP. If you picked up the phone you would hear what sounded like white noise.
I lost many a download, or ‘online’ game to my mother picking up the phone to make a phone call.
That’s very clear, thanks. I remember reading about the original phreakers, one guy had perfect pitch so he could just sing the tone to open a long distance line and then dial away. I think he got sued by one of the American phone companies for stealing.
There are some amazing stories about the cat and mouse games regarding phone phreaking. The “captain crunch whistle” is great. All their attempts at security, beaten by a bit of audible brute force, via a cereal box toy.
Think of it like a “fast”, automated telegraph that speaks in audible binary instead of Morse code.
audible binary
Thanks yea, that’s exactly what I was wondering.
It might actually be hexadecimal or some other encoding but you get the point.
Watch Wargames. The main character uses the Ethernet somewhat realistically compared to more modern “depictions” of “hackers,” and has to use an absolutely ancient modem to connect to the network.
Omg “a 1983 American techno-thriller film”… sounds awesome B-)
And probably more relevant today than it was back then.
That I can help. Back in the day, to access someone’s else computer you literally hooked up yours to a phone line. Your desktop would then dial to some number you got that had another computer listening to answer, and they would start a “conversation”. Your computer sent what to us would sound like noise, the receiving computer would listen to that noise and answer back. Voilá, you’re connected to a network!
It’s binary with low tones representing 0 and high tones representing 1. Thanks for the link, that’s just what I wanted to know!
what 5 yrs old can listen to a wall of text?
deleted by creator
They weren’t always premium. Your local ISP or college often ran a server for their users in the old days.
It’s funny seeing this thread with it’s descriptions written as if describing the traditions of some long-gone ancient civilization.
Which part of that isn’t true
I mean it’s… checks math… over 30 years old. That is ancient in technology timeline.
It’s an old federated message board system. Message boards are called “newsgroups “. It predates the web so it’s usually accessed via a special client app. To use it you’d need:
- A Usenet client app, called a newsreader. See Wikipedia. Many are probably abandoned by now.
- An account with a Usenet provider. A search engine will point you to several options. There used to be some free ones. If there still are, it would be a good way to try it out. But note that the free ones often don’t carry all of the newsgroups — they omit the binary groups, which are known to carry pirated software and, let’s say, diverse video content.
It’s federated? Okay, that makes a lot more sense. I thought usenet providers were like isps, connecting to a single, central host or something. I didn’t realize they were federated systems.
It’s really neat to think of Usenet as ‘federated’ considering that’s a new term for most of us.
My preferred options are: Binsearch, astraweb, and newsbin
You get what you pay for, a bargain IMHO
Check out FidoNet for another example of federation.
If I recall correctly, ISPs were often running their own Usenet servers. This meant that traffic didn’t leave their networks and thus they paid less network interchange fees.
These days maybe only niche ISPs in some parts of the world might be running Usenet servers. Majority of them are run by specific companies created specifically for the purpose.
deleted by creator
Heh. I used to run leafnode as my own, in house, single person server.
I’m quite surprised to find out it’s still alive and maintained.
I’d add to this and say you need:
A Usenet client such as SabNZBD (like qbittorrent or similar downloading client, but for Usenet)
A Usenet provider such as Astraweb, Newshosting, UsenetServer, Easynews, etc (or a paid subscription giving you access to a number of servers, kind of like a private torrent site)
And a tracker or indexer such as NZBGeek, NZB Finder, omgwtfnzbs, DrunkenSlug, etc (similar to a library index that helps you find what you want in the sea of information)
You can set services like these up with programs that use these tools to pull what you want automatically, such as the Arrs (Sonarr, Radarr, Lidarr, etc) or you can manually search either the provider directly through their own search engine, or through an indexer’s refined search engine.
Get a free account from https://www.eternal-september.org/.
Then in the desktop version of Thunderbird , “Add a newsgroup account” with your login details from eternal september.
Find some groups to subscribe to. Try comp.lang.python for example. Once you subscribe you can see the posts inside it.
You’ll probably see lots of spam. To remove the spam you need to use Thunderbirds Message Filters functionality - remove all messages where their Message-ID header contains “googlegroups”.
Not having any experience with this I find it hilarious that most of the spam comes from googlegroups
Heh, yeah.
Google does a great job stopping spam in Gmail so they could stop it if they wanted but… Just haven’t. Super lame.
I’m just amazed they haven’t shut down Google groups by now…
Someone made the calculation that the data harvested for advertisers is more valuable than the hosting/maintenance expense they would recoup by killing it. Plus it’s tasty food for Bard.
Usenet is a decades-old distributed message sharing system. It’s like an old school message board. To access it, you need a newsreader. Mozilla Thunderbird is one such example.
I have not accessed newsgroups in several years, so I don’t know how active it is today. But it used to be the go-to source for “warez” and bootleg media and porn. Oh yeah, and discussions threads on myriad topics. :)
Slashdot, digg, Reddit, lemmy, 4chan, etc. are all spiritual descendents of usenet.
The software tech for usenet is old, slow, and has a learning curve. You might find it frustrating to navigate and use. However, modern newsreaders probably hide some of the complexity.
Usenet has its own protocol (NNTP) where digests of messages get passed from server to server, eventually making it out to all (or most) servers that host a particular group (like alt, sys, gov, etc.). In essence, it’s a federated digital bulletin board of bulletin boards. Many servers don’t participate in some groups such as alt.binary.*.
Usenet pre-dates the world wide web, and even pre-dates Gopher. It was designed such that a Usenet server could spend most of its time disconnected from the Internet and accumulate local posts that would then be federated in a digest when the server dialed up and connected to other servers.
The main NNTP network eventually made its way to a centralized web-accessible service and most places that used to provide an NNTP server (which was most ISPs in the 90s) eventually shut their servers down and only provided gateway and email services.
The protocol still exists though, and there’s still a small connected network.
In reality, Lemmy is the spiritual descendant of Usenet.
It was designed such that a Usenet server could spend most of its time disconnected from the Internet and accumulate local posts that would then be federated in a digest when the server dialed up and connected to other servers.
…Would this have been local posts of an individual, or sometimes a group in a LAN or something? The way you describe it here puts me in the mind of recent stuff like Scuttlebutt, albeit that’s more clearly individual-focused.
What’s scuttlebutt?
And no, not individual and not LAN. WAN. A Usenet server could easily service hundreds of folks if not thousands. It would collect all their posts and then aggregate upwards.
This gives a brief overview of Scuttlebutt with a link to a more technical breakdown.
That said, I remain confused by the other person’s description, as I’m not sure how it’s accumulating posts while “disconnected from the Internet”. I follow how it works when connected, but not so much how it would work as they’ve described it, at least in the disconnected circumstances, unless it’s sorta how I asked.
The Internet is a network of networks. Nowadays, everyone tends to have always-on connections to the entire thing, but back in the day, many of the networks spent a lot of time disconnected from each other. Usenet was designed to mostly transparently handle this by the local network having an aggregating server that would collect all the local activity and share it with other Usenet servers when it could reach them.
Remember that even the local activity was people connecting with teletype terminals and dialing up over modems from remote systems. Long distance trunking fees were a big deal, and Internet routing had to deal with the possibility that there was currently no route to the destination address.
Thanks for clarifying! So it does work roughly as I was thinking, that’s cool!
You can subscribe to Easynews. It’s Usenet turned into a website. There’s a built in search engine (supports regular expressions), retention going back to 2008, spam and malware filtering, and multiple servers located in the US and Europe. You choose whether to use the web or a Usenet client. Probably the easiest way to use this neglected corner of the Internet.
Even Usenet gets censored, but there a window of a couple days between posting and takedown where the file is available. We see this a lot with major studios who pay investigators to identify infringing material. To get around this, some uploaders are encrypting their content, and you’ll need the description key.
Usenet used to be freely available through most ISPs. But now, even though there are still a lot of usenet groups active for discussion, it is also used for file “sharing”. So hosting Usenet groups can be very data intensive, and ISPs stopped hosting them.
Google hosts a free portal into usenet at groups.google.com , but I just went there and saw that even they decided it’s too much trouble and will be stopping soon.
There are a few subscription based usenet servers, they vary from $5 to $10 a month. Easynews.com , giganews.com are two that seem to still be open.
Yeah Google bought a really great thing and ran it for a very long time as their own version of stackoverflow. I bet stackoverflow was a response to Google’s monopoly on groups answering questions.
Two long answers and yet neither of them answered how to get started. Sorry I don’t know how either.
Same, just stopping by to see if anyone has any answers. Like, there needs to be a “how to usenet for dummies” guide that sets you up with the best usenet stuff and explains what each thing does and why you care.
Edit: also, in the era of free services, why do you still have to pay for Usenet access?
Edit 2: apparently my question is too dumb for “no stupid questions” lol.
in the era of free services, why do you still have to pay for Usenet access?
Bandwidth costs money. And usenet isn’t collecting and selling your info to recoup those costs like most “free” services. A lot of ISPs do offer free usenet access to customers, albeit somewhat neutered. You can browse the bulletin boards but most of the file downloads are going to be filtered out. That’s the part you have to pay for.
Okay, but why the bandwidth costs? Is Usenet a separate network entirely? Do you have to use a VPN to connect to it? I’m honestly curious about it, it just seems alien compared to everything else. Why are there different usenet providers and not a single “Usenet” company?
Edit: someone else said that usenet is federated, which makes a lot more sense. I thought it was like a single host that the providers connected to, like isps.
Usenet and the message boards being referred to are ‘proto-internet’ services. Think BBS, where your computer dialed into a service, and you could interact with that builiten board, the messages and users on it, as well as any files it had available for download.
Usenet had newsgroups that were very diverse and specific, and originally were just like message boards, but at some point, the major remaining Usenet servers started just sharing to each other, or maybe more appropriately, they would reference each other.
As someone mentioned before, it’s a protocol just like HTTP. There’s a bunch of servers all hosting webpages made in hypertext, and we just jump between them with links. Likewise, there’s a bunch of servers out there hosting newsgroups, but you have to find a gateway to get started. The reason there’s no ‘one’ company is akin to asking why all websites aren’t hosted/owned by one company.
If anything… It’s kinda like lemmy/fediverse stuff. You make an account with one instance, but since the protocols are the same, you can use your account on that one instance to talk to the whole fediverse network, multiple instances.
Why it costs is because at this point, it’s an archive. A huge archive, of not just text discussions, but also all the files that have been posted since a very long time ago. And just like the currently ‘free’ archive.org, it costs money to host all of that. Usenet is a bit less resource intensive than a modern website, so it can just basically sit… But they just ask that you pay to access it, pay to have an account. In this case, you’re paying to access a network that is separated from the rest of the internet at large.
Usenet is a decentralized network that works very much like the fediverse here. Anything uploaded to one server gets sent out to all the servers; that’s why you can log onto lemmy.world and see posts made on say lemm.ee. Likewise something posted to one usenet server gets sent out to all the others. And these servers are being operated by individual entities (again, much like lemmy). Whoever owns the server you’re downloading from is having to pay for everything it sends out. Which is why most of the free ones will let you access the text portions but not the file downloads.
A VPN isn’t necessary. But you do need a special program to connect to usenet. It’s built into a lot of older email clients (thunderbird is a good option).
There’s no one single usenet company for the same reason there’s no one single lemmy instance. (I’m sure someone has explained it better somewhere else in this post).
Usenet costs money the same way as your email (should) cost money. They’re hosting a large amount of content, they’re paying fees to their ISPs to let them send that content to you, and they’re fighting spam and other crap.
If you want to do nothing more than chat with folks on Usenet then you don’t need a VPN. Just like if your only download Ubuntu from torrents you don’t need vpn. Anything else and a vpn is a good idea. That said, I recommend you should always use VPN. Your isp is not your friend. They will steal your data. Heck, even your VPN isn’t your friend. Regularly read reviews of your VPN and ensure they’re privacy friendly and charge for their service.
Do not fall into the Onavo trap.
Simple. Usenet is an email inbox anyone can read.
What it looked like was an email program with a list of subject names like mail folders, each containing subject lines of conversation threads. The threads were fully branched, replies under the correct messages, like Lemmy. Not a simple list, like email.
Also unlike email, the messages were posted publicly instead of to you.
There was a list of newsgroup names for different subjects, you’d pick which of those to get messages from to appear as the “mail folders”.
The names were in a hierarchy, so computer subjects were comp.something, hobbies/recreation were rec.something etc. a bit like website names, only back to front, general to more specific, e.g uk.rec.sheds, alt.startrek.fanfic , rec.humor, rec.humor.funny.
You’d download messages from (and upload your replies to) a server and it would share messages with other servers, like Lemmy federation. So each group would be a merge of all messages from all around the world. Effectively there would only be ONE alt.folklore.urban for instance.
Usually your isp would run a server and you’d use that.
At first it wasn’t mainly used as a way to share binary files encoded as text messages, but eventually that took over, isps dropped having servers and big paid ones took over.
Oh, one really cool thing, newsreader programs would usually show you which message threads had new messages, so it was easy to keep up with interesting conversations.
BTW, OP and others… Incredible Doom is the name of an online and print graphic novel series that has Usenet at its core. It’s a lovely piece of art created by very talented artist. Check it out from your local library or used book store…
Or… cough… from your local Usenet group dealing in comix warez.
