Tag Archives: howto

What is a newsgroup

I have seen a lot of discussions on forums and even on Usenet itself starting with questions like “what is a news group”, “what are newsgroups” or more commonly “what is a newsgroup”.  A Usenet newsgroup is a discussion group for a specific topic, which is shared on Usenet.  It is not a forum, or a blog, or even a file-sharing or online storage application, although it can look like any of those depending on what newsgroup you are reading. There are big differences between Usenet and other types of discussion sites and file-sharing applications.

A forum is typically hosted on a single web server, in order to post to that forum you typically have to sign up for an account to post questions or answers. Usenet is hosted on literally thousands of newsservers around the world, each server has it’s own copy of articles that it carries.  Most servers require users to sign up for an account, but there are some with free accounts and a few that still allow anonymous connections.  If a persons Usenet account allows posting, they can post questions and answers to a newsgroup and it is copied to all the servers in the network.  This gives excellent redundancy, which is a definite benefit, but there is a lack of control that most forums allow, such as being able to block some discussions or delete certain comments.

A blog is also typically hosted on a single web server using software like wordpress. Usually the blog owner is the only person allowed to post, although many sites allow others to leave comments. Bloggers typically post discussions about topics they are interested in, such as how their day went or what their cat is up too, or in this case “what are news groups”. A person could do exactly the same thing in a Usenet newsgroup, with all the same Usenet benefits and concerns listed above. One big difference between a newsgroup and a blog is that once an article is posted, you cannot edit it, and sometimes getting it deleted from all those servers is difficult, many servers ignore cancel requests for various reasons, including prevent vandalism.

File-sharing applications allow people to host binaries on their personal computers and share them with other people over the internet using a p2p (peer-to-peer) connection so users can copy files to and from each others machines.  Although Usenet was not originally designed to share  binaries, it was not long after its inception that people figured out how to use it for that.  There are a couple of glaring differences between Usenet and P2P, the biggest being the distributed way it stores files.  Although P2P usually has good distribution of files, if everyone with the binary you are interested in has deleted it from their PC or are simply not running the application, you cannot download the binary.  Another issue can be if the person that has parts of the file you want has a slow connection, it can take a very long time to download it. Getting a binary from a Usenet servers is easier and usually faster, if the binary is listed you can download it all from your newsserver, and the download speed is constant.  Most modern Usenet providers have very fast internet connections and can feed binaries to multiple users at the same time without any significant loss of download speed.

Online storage applications allow a user to securely save a backup copy of files to the providers server.  The provider usually has redundant servers and performs backups to protect the users data so they can recover at any time in the future.  They also usually have applications for many different devices including PCs, Macs and Smart phones. Online storage is really geared towards allowing a person or a company to backup their files somewhere else so they won’t lose them in the case of a fire or other catastrophe. Usenet allows users to post files, and due to its distributed nature it seems like a fairly safe way to store your personal information, but there is no inherent security; a file posted to a newsgroup can be viewed by anyone with a newsreader.  Users can encrypt their postings to prevent prying eyes from looking at personal information, but some servers may delete your files automatically based on a normal expiration schedule.  Because of the lack of security and guarantee of data retention, Usenet cannot be relied on as a file storage medium for critical data for an extended period of time.

Downloading Usenet Binaries

Many years ago, the Usenet only allowed 7-bit text messages to ensure that communications was possible between any type of machine, using any kind of modem.  It was not long before someone figured out how to encode 8-bit data into a 7-bit format so that binaries could be sent using Usenet with no modifications to the Newsgroup server software.  The original programs for encoding and decoding articles were named uuencode and uudecode, respectively.  Modern versions of Usenet are now 8-bit safe and even allow for multi-byte text in UTF-8 format, but various encoding methods for articles are still used.

In the old days, to download a Usenet binary, you had to locate all its parts on the server and download them to your computer.  After all the parts were on your computer you would feed them to a decoding program, which usually gave you the binary or an archive containing your binary.  If your binary was in an archive, you then used another application to extract your binary from the archive.  Needless to say, this could be a long and tedious process, and often not all the parts were available on your Usenet server.

Over the years, the applications and Newsgroup servers have gotten much more sophisticated.

  • Usenet server software was upgraded to be able to work with other languages than English, and in the process was made 8-bit safe so that encoding from 8-bit to 7-bit was no longer required, decreasing article sizes by about 40%
  • Web sites such as Newzbin started popping up which created NZB files providing users with an exact list of all articles required to recreate a binary
  • Encoding software began creating NZB and SFV files automatically and posting them with the binaries
  • An application called parchive created ‘PAR’ files which could be used to repair incomplete files using complex algorithms that could not only detect errors in the parts, but could even recreate missing parts using the ‘PAR’ file and the parts that were successfully downloaded.

Most modern newsreaders have integrated newsgroup search features, and almost all of them support the NZB standard file format, both of which make finding all the parts of a Usenet binary much simpler.  In addition, most newsgroup readers also directly support decoding, repairing and extracting binaries into a directory for you, so all you have to do is tell it what to download with an NZB file and where to store it.  Another nice feature of some newsgroup browsers is that they can work with multiple servers, so if a part is missing from one server, they will attempt to get it from an alternate server.

Putting it all together

For most users interested in downloading from Usenet, the steps are now pretty simple.

  1. Choose a Usenet provider
  2. Choose a newsreader
  3. Use a newsgroup search website or your newsreader’s integrated search to find the digital media you want
  4. if you are using a website, import the NZB from the website
  5. click download
  6. when the download completes, use your new binary

Alternate Usenet Ports

In the past, many ISPs (Internet Service Providers) began blocking or throttling peoples Usenet connections to outside Newsgroup Servers.  This was done because most ISPs provided Usenet and they wanted to discourage people from reading Newsgroups from other providers which could use a lot of the ISPs Internet bandwidth.  Although most ISPs no longer provide Usenet on their own networks, it is still common for many ISPs to block or throttle Usenet for the same reason, bandwidth.In order to be able to provide better service to more people, most NNTP server providers allow alternate ports that a subscriber can use to connect to their Newsgroup servers.  By using alternate ports it is more difficult for an ISP to use technologies, such as traffic-shaping to place caps on only Usenet users.  If you have noticed that your Usenet downloads seem significantly slower than downloading in your web browser, your ISP might be throttling your connection, and using an alternate port could help you to get around it.

Some ISPs have used software in the past which is able to recognize the protocol that is being used on a port, and based on the type of traffic they can throttle or block your Usenet.  Although NNTP and HTTP use plain-text messages to transfer data back and forth, the commands and responses (although using similar result codes) are different enough to be easily differentiated. Because of this, and privacy concerns, Usenet providers also allow users to connect to their NNTP servers securely, using 256-Bit SSL connections, which render the content of dialog between a Usenet client and the Usenet server, unreadable or recognizable.

Below is a table with various commercial Usenet providers and the optional ports you can use to connect to them.

Alternate NNTP Ports
Usenet Provider Server Addresses Encrypted Ports Standard
Alt.Binaries east.AltBinaries.com
80, 443, 563, 8080 119
Astraweb news.astraweb.com
443, 563 23, 119,  1818, 8080
Easynews news.easynews.com
443, 563, 8080 21, 22, 80, 119
Easynews Web members.easynews.com
81,443 80
Giganews Usenet news.giganews.com
443, 563 23, 80, 119
Newsdemon news.newsdemon.com
80, 81, 563 23, 25, 119, 443, 8080
Newsgroup Direct news.newsgroupdirect.com
80, 81, 563 23, 25, 119, 443, 8080
NewsGuy news.newsguy.com 443, 563 80 , 119, 8080
Newshosting news.newshosting.com 443, 563 23, 25, 80, 119, 3128
Ngroups us.ngroups.net
80, 81, 465, 563, 993 20,23, 53, 119, 443, 2000, 8080, 9000, 9001, 9002
PowerUsenet news.powerusenet.com 443, 563 23, 80, 119
SuperNews news.supernews.com
443, 563 23, 80, 119
ThunderNews news.thundernews.com
80, 81, 563 23, 25, 119, 443, 8080
Usenet.net news.usenet.net 443, 563 23, 80, 119
UseNetServer news.usenetserver.com
443, 563, 8080 23, 25, 119, 3128, 8000, 9000

Ports Used

Almost all of the ports used by Newsgroup servers as alternates are actually defined for a specific purpose. This is done so that ISPs will be more reluctant to simply block the port and thus interfere with other users ability to use those services. Below is a table which tells what service typically is run on each port.

Defined Ports
Port Service Description
119 NNTP Network News Transfer Protocol
20 FTP-DATA FTP Data Channel
21 FTP File Transfer Protocol
22 SSH Secure Shell Login
23 TELNET Telnet Login
25 SMTP Simple Mail Transfer Protocol
53 DNS Domain Name Service
80 HTTP HyperText Transfer Protocol
Web Browsing
81 Not Defined
465 URD URL rendezvous directory for SSM
993 IMAPS Internet Message Access Protocol over TLS/SSL
1818 ETFTP Enhanced Trivial File Transfer Protocol
2000 Used by Cyrus and/or Cisco Equipment
3128 Typically used for Web Proxies
8000 Not defined
8080 HTTP-ALT Alternate port for HTTP
9000 CSLISTENER CSlistener
9001 ETLSERVICEMGR ETL Service Manager
9002 DYNAMID DynamID authentication