Category Archives: How To Usenet

What is a newsgroup

I have seen a lot of discussions on forums and even on Usenet itself starting with questions like “what is a news group”, “what are newsgroups” or more commonly “what is a newsgroup”.  A Usenet newsgroup is a discussion group for a specific topic, which is shared on Usenet.  It is not a forum, or a blog, or even a file-sharing or online storage application, although it can look like any of those depending on what newsgroup you are reading. There are big differences between Usenet and other types of discussion sites and file-sharing applications.

A forum is typically hosted on a single web server, in order to post to that forum you typically have to sign up for an account to post questions or answers. Usenet is hosted on literally thousands of newsservers around the world, each server has it’s own copy of articles that it carries.  Most servers require users to sign up for an account, but there are some with free accounts and a few that still allow anonymous connections.  If a persons Usenet account allows posting, they can post questions and answers to a newsgroup and it is copied to all the servers in the network.  This gives excellent redundancy, which is a definite benefit, but there is a lack of control that most forums allow, such as being able to block some discussions or delete certain comments.

A blog is also typically hosted on a single web server using software like wordpress. Usually the blog owner is the only person allowed to post, although many sites allow others to leave comments. Bloggers typically post discussions about topics they are interested in, such as how their day went or what their cat is up too, or in this case “what are news groups”. A person could do exactly the same thing in a Usenet newsgroup, with all the same Usenet benefits and concerns listed above. One big difference between a newsgroup and a blog is that once an article is posted, you cannot edit it, and sometimes getting it deleted from all those servers is difficult, many servers ignore cancel requests for various reasons, including prevent vandalism.

File-sharing applications allow people to host binaries on their personal computers and share them with other people over the internet using a p2p (peer-to-peer) connection so users can copy files to and from each others machines.  Although Usenet was not originally designed to share  binaries, it was not long after its inception that people figured out how to use it for that.  There are a couple of glaring differences between Usenet and P2P, the biggest being the distributed way it stores files.  Although P2P usually has good distribution of files, if everyone with the binary you are interested in has deleted it from their PC or are simply not running the application, you cannot download the binary.  Another issue can be if the person that has parts of the file you want has a slow connection, it can take a very long time to download it. Getting a binary from a Usenet servers is easier and usually faster, if the binary is listed you can download it all from your newsserver, and the download speed is constant.  Most modern Usenet providers have very fast internet connections and can feed binaries to multiple users at the same time without any significant loss of download speed.

Online storage applications allow a user to securely save a backup copy of files to the providers server.  The provider usually has redundant servers and performs backups to protect the users data so they can recover at any time in the future.  They also usually have applications for many different devices including PCs, Macs and Smart phones. Online storage is really geared towards allowing a person or a company to backup their files somewhere else so they won’t lose them in the case of a fire or other catastrophe. Usenet allows users to post files, and due to its distributed nature it seems like a fairly safe way to store your personal information, but there is no inherent security; a file posted to a newsgroup can be viewed by anyone with a newsreader.  Users can encrypt their postings to prevent prying eyes from looking at personal information, but some servers may delete your files automatically based on a normal expiration schedule.  Because of the lack of security and guarantee of data retention, Usenet cannot be relied on as a file storage medium for critical data for an extended period of time.

What is Usenet

Usenet has been available at universities and corporations since long before the internet was available to people in their homes.  It has always been a great medium for sharing thoughts and ideas, and many of the earliest “open source” projects shared their software using the various comp.source.* newsgroups.  Today, it has moved from it’s original discussion driven design to include all types of digital media, including various types of executables, videos, music and images which can usually be found in the alt.binaries.* groups.  Using modern newsreader software and a good Usenet provider the Usenet provides excellent speed and reliability which is often missing from other file-sharing options, such as BitTorrent or LimeWire.

What is Usenet?

For a detailed history of Usenet I recommend that you got to Wikipedia.  They have a very large, detailed and … well… boring explanation of the history of Usenet.  Most people that are reading this posting really don’t care about the history, they want to know about what it is today.

Okay, maybe a little history is in order; the original design of Usenet was loosely based on the old Bulletin Board Systems of the late 70s and early 80s.  One major difference between Usenet and a BBS is that instead of everyone connecting to a central computer via a modem to access data, Usenet systems each get a copy of the articles allowing users to read them locally.  This design worked well for universities and corporations, and helped to reduce the cost of operating a Usenet server.  Typical operation was to set up your server to call a couple of neighboring servers (hopefully local) and exchange whatever articles they had, those servers would talk to their neighbors.  Eventually articles would get to someone with a hard-line to another city and the sharing would continue; many universities had permanent connections with other universities, and some large corporations had their offices inter-connected as well.

The Usenet still operates in this fashion, but the large inter-connected companies are now called ‘Tier-1 Providers”.  There are anywhere from 4-10 of the Tier-1 providers, depending on whom you believe.  Most  the Tier-1 providers are increasing their storage so that their article retention and completion are nearly 100% for the last few years.  The primary exception to this is articles which get removed in response to DMCA take down notices from copyright owners.

What can I do with Usenet

Usenet was originally designed for the sharing of ideas, and there are still many newsgroups that are dedicated to just that.  Although their traffic is slowly dying off due to technologies such as forums, and blogs, many still have very lively conversations from time to time.  Since the advent of the Internet, Usenet has become more of a file sharing medium, allowing users to post images, music, videos or executables for other users to enjoy.

Is Usenet legal

Most binaries posted are legal for you to download and use, but there are also some that are posted in violation of their copyright.  Most people who use Usenet are smart enough to know that if you find a movie that was shown at the theater, it is almost certainly not legal to download. However, it can sometimes be tricky to figure out if an video, image or MP3 file are legal.  Movie trailers are usually legal to download; many websites post some of their content as a teaser to get new members which is legal to download; some bands have privately published music which they allow people to download for free, these are legal; sometimes a user joins a website to download the content and post it to Usenet which is not legal.  Regardless of how copyright material ends up on Usenet, when it happens, the copyright holder must send a “DMCA take down” message to the Usenet provider, and the Newsgroup server is legally obligated to remove those articles.

Downloading Usenet Binaries

Many years ago, the Usenet only allowed 7-bit text messages to ensure that communications was possible between any type of machine, using any kind of modem.  It was not long before someone figured out how to encode 8-bit data into a 7-bit format so that binaries could be sent using Usenet with no modifications to the Newsgroup server software.  The original programs for encoding and decoding articles were named uuencode and uudecode, respectively.  Modern versions of Usenet are now 8-bit safe and even allow for multi-byte text in UTF-8 format, but various encoding methods for articles are still used.

In the old days, to download a Usenet binary, you had to locate all its parts on the server and download them to your computer.  After all the parts were on your computer you would feed them to a decoding program, which usually gave you the binary or an archive containing your binary.  If your binary was in an archive, you then used another application to extract your binary from the archive.  Needless to say, this could be a long and tedious process, and often not all the parts were available on your Usenet server.

Over the years, the applications and Newsgroup servers have gotten much more sophisticated.

  • Usenet server software was upgraded to be able to work with other languages than English, and in the process was made 8-bit safe so that encoding from 8-bit to 7-bit was no longer required, decreasing article sizes by about 40%
  • Web sites such as Newzbin started popping up which created NZB files providing users with an exact list of all articles required to recreate a binary
  • Encoding software began creating NZB and SFV files automatically and posting them with the binaries
  • An application called parchive created ‘PAR’ files which could be used to repair incomplete files using complex algorithms that could not only detect errors in the parts, but could even recreate missing parts using the ‘PAR’ file and the parts that were successfully downloaded.

Most modern newsreaders have integrated newsgroup search features, and almost all of them support the NZB standard file format, both of which make finding all the parts of a Usenet binary much simpler.  In addition, most newsgroup readers also directly support decoding, repairing and extracting binaries into a directory for you, so all you have to do is tell it what to download with an NZB file and where to store it.  Another nice feature of some newsgroup browsers is that they can work with multiple servers, so if a part is missing from one server, they will attempt to get it from an alternate server.

Putting it all together

For most users interested in downloading from Usenet, the steps are now pretty simple.

  1. Choose a Usenet provider
  2. Choose a newsreader
  3. Use a newsgroup search website or your newsreader’s integrated search to find the digital media you want
  4. if you are using a website, import the NZB from the website
  5. click download
  6. when the download completes, use your new binary

Free Usenet Binaries

The truth about free Usenet

After you have spent some time trying the various free servers on my newsgroup search site, it should be apparent that it is no longer possible to download from the alt.binaries newsgroups for free. Many of us thought we had it all nailed down 20 years ago and were set; now we are going to the commercial Newsgroup providers because our university or corporate provider have stopped giving us access for free, due to increased cost and in some cases, legal concerns. Continue reading Free Usenet Binaries

Alternate Usenet Ports

In the past, many ISPs (Internet Service Providers) began blocking or throttling peoples Usenet connections to outside Newsgroup Servers.  This was done because most ISPs provided Usenet and they wanted to discourage people from reading Newsgroups from other providers which could use a lot of the ISPs Internet bandwidth.  Although most ISPs no longer provide Usenet on their own networks, it is still common for many ISPs to block or throttle Usenet for the same reason, bandwidth.In order to be able to provide better service to more people, most NNTP server providers allow alternate ports that a subscriber can use to connect to their Newsgroup servers.  By using alternate ports it is more difficult for an ISP to use technologies, such as traffic-shaping to place caps on only Usenet users.  If you have noticed that your Usenet downloads seem significantly slower than downloading in your web browser, your ISP might be throttling your connection, and using an alternate port could help you to get around it.

Some ISPs have used software in the past which is able to recognize the protocol that is being used on a port, and based on the type of traffic they can throttle or block your Usenet.  Although NNTP and HTTP use plain-text messages to transfer data back and forth, the commands and responses (although using similar result codes) are different enough to be easily differentiated. Because of this, and privacy concerns, Usenet providers also allow users to connect to their NNTP servers securely, using 256-Bit SSL connections, which render the content of dialog between a Usenet client and the Usenet server, unreadable or recognizable.

Below is a table with various commercial Usenet providers and the optional ports you can use to connect to them.

Alternate NNTP Ports
Usenet Provider Server Addresses Encrypted Ports Standard
80, 443, 563, 8080 119
443, 563 23, 119,  1818, 8080
443, 563, 8080 21, 22, 80, 119
Easynews Web
81,443 80
Giganews Usenet
443, 563 23, 80, 119
80, 81, 563 23, 25, 119, 443, 8080
Newsgroup Direct
80, 81, 563 23, 25, 119, 443, 8080
NewsGuy 443, 563 80 , 119, 8080
Newshosting 443, 563 23, 25, 80, 119, 3128
80, 81, 465, 563, 993 20,23, 53, 119, 443, 2000, 8080, 9000, 9001, 9002
PowerUsenet 443, 563 23, 80, 119
443, 563 23, 80, 119
80, 81, 563 23, 25, 119, 443, 8080 443, 563 23, 80, 119
443, 563, 8080 23, 25, 119, 3128, 8000, 9000

Ports Used

Almost all of the ports used by Newsgroup servers as alternates are actually defined for a specific purpose. This is done so that ISPs will be more reluctant to simply block the port and thus interfere with other users ability to use those services. Below is a table which tells what service typically is run on each port.

Defined Ports
Port Service Description
119 NNTP Network News Transfer Protocol
20 FTP-DATA FTP Data Channel
21 FTP File Transfer Protocol
22 SSH Secure Shell Login
23 TELNET Telnet Login
25 SMTP Simple Mail Transfer Protocol
53 DNS Domain Name Service
80 HTTP HyperText Transfer Protocol
Web Browsing
81 Not Defined
465 URD URL rendezvous directory for SSM
993 IMAPS Internet Message Access Protocol over TLS/SSL
1818 ETFTP Enhanced Trivial File Transfer Protocol
2000 Used by Cyrus and/or Cisco Equipment
3128 Typically used for Web Proxies
8000 Not defined
8080 HTTP-ALT Alternate port for HTTP
9000 CSLISTENER CSlistener
9001 ETLSERVICEMGR ETL Service Manager
9002 DYNAMID DynamID authentication