A Usenet newsgroup is a discussion group within the Usenet system for messages posted by users about a specific topic.  This term is often confused with forums or blogs, but they are different in many ways. In order to read newsgroups you need access to a Usenet server and newsreader software.

Usenet was originally based on the old bulletin board systems (BBS) of the late 70’s and early 80’s.  The primary difference is that a BBS required users to dial in to a centralized computer to read all the articles, and they were not (typically) shared between different BBS’.  Usenet turned that design on it’s head, articles are stored and copied between servers around the world, with each maintaining a separate repository of articles for all the newsgroups.  When servers communicated with each other they would trade missing articles so that all servers eventually would have all articles in their repository.  This design allowed users to dial in to a newsgroup server and share ideas with people from all around the world.

Newsgroup Hierarchies

Newsgroups are grouped together in hierarchies which help users to locate newsgroups they are interested in more easily on the Usenet.  Originally all newsgroups were in the net.* hierarchy, and locating a news group you were interested in could be difficult. In 1987 some of the major Usenet backbone sites got together and rearranged all the newsgroups in what was called the “big-7”, they were:

• comp.* – computer related newsgroups
• news.* – Usenet related newsgroups
• rec.* – recreational activity related newsgroups
• sci.* – newsgroups for scientific topics
• soc.* – social issue newsgroups
• talk.* – contentions issue related newsgroups suck as religion and politics
• misc.* – miscellaneous discussions that didn’t fit any of the above

In 1995 the “Big 7” was changed to the “Big 8” with the addition of the humanities.* hierarchy for discussion topics such as the arts and philosophy. All of the “Big 8” groups were controlled by a group from the major backbone operators and they allowed users to make suggestions for creating or deleting groups, and those requests were formalized by choosing an appropriate name and group charter.  Finally, the proposal was put to a vote by the group.  Often, in the end, the answer was “No” which frustrated a lot of users who pushed for the creation of a new hierarchy that was not controlled by the “Big 8” group and eventually the alt.* hierarchy was created.

Unlike the “Big 8” which requires users to present a Request for Discussion (RFD) to create a newsgroup, the alt.* hierarchy is the wild west.  Users that want to create their own newsgroup simply put alt.<topic> in the newsgroup to post to and many servers will propagate it to all their peers.  The major shortfall in this design is that; while “Big 8” newsgroups will be carried by almost all Usenet servers, the decision to pass on unknown groups is up to the individual server operators, and some will refuse to carry or propagate those groups.  The upshot of this is that you can create a group, but no one may ever see it but you.

Other Hierarchies

There are many other hierarchies now on the Usenet, many of which are operator or country/language based.  Because of the flexible design of the Usenet software, an operator can create newsgroups which are only available to their users and will not be passed to other servers.  Also, for many non-English users, groups have been created that allow discussion in their country or native language.  Most of these hierarchies have a regulating body (or individual) which maintains the list of allowed newsgroups.  Some of the major hierarchies include:

• cn.* – for Chinese newsgroups
• de.* – for German newsgroups
• fr.* – for French newsgroups
• it.* – for Italian newsgroups
• kr.* – for Korean newsgroups
• tw.* – for Taiwanese newsgroups

Types of Newsgroups

There are predominantly two types of newsgroups, binary or text.  Both work the same way from a server standpoint, and there is no major differences in the naming conventions except that binary groups often have ‘binaries’ somewhere in the name, such as ‘alt.binaries.pictures.rail’ which carries pictures of trains by newsgroup readers.  Often there is a discussion newsgroup for a binary group which will have an identical name but have ‘.d’ added to the end denoting that it is for discussion of postings to the binary newsgroup.

Newsgroups are usually about specific topics, and some are even moderated to ensure that off-topic posts are not posted to a discussion, while others allow users to post articles about a wide variety of topics.  There are currently over 110,000 newsgroups, most of which do not see any activity except for SPAM, but estimates are that about 20,000 of the newsgroups are still regularly used by people for their intended purposes.  A lot of the other 90,000+ are newsgroups created in the alt.* hierarchy either intentionally, or often through a typographical error while typing in the newsgroup to post or cross-post to.

Binary Newsgroups

The Usenet was originally designed to work with US-ASCII characters, and was at that time a very US-centric system.  The Usenet originally only connected a few Universities together, and that was expanded to include corporations, high-schools and some individuals.  It wasn’t long before some users wanted to be able to send binary data in newsgroups and a method (called UUencode) was created which converted 8-bit binary files to 7-bit US-ASCII.  Users that wanted to use the binary data could decode it (using UUdecode) and have a copy of the original file.

The modern Usenet software is designed to work with many international character sets and can transfer those articles without risk of damage to 8-bit data, but articles are still often sent in 7-bit clean formats.  There are also newer programs that can create articles that use 8-bit data such as yEnc.  yEnc’s 8-bit articles are much more efficient at transferring binaries than the older UUencode articles because the conversion from 8-bit to 7-bit has an approximate overhead of about 40%, whereas the overhead for yEnc is much lower, usually as low as 1-2%.

Another restriction of the original Usenet servers was often article size, this restriction required that larger binaries be broken up into multiple parts which they poster would then post manually.  Users that wanted the original binary would download all the parts, and put them back together manually before decoding. This process was tedious and error prone, plus servers often would not get all the parts of a posting, or might delete parts before the user had time to download all of them.  The high error rate caused the ‘.d’ discussion groups to often be filled with requests for “part 13 of 53”.  Modern newsgroup readers are able to handle multi-part binary postings internally and the users usually needs to just pick the binary they want and come back later for the completed download.