Category Archives: Usenet

Articles about Usenet in general

Google Group vs Usenet Newsgroup

Earlier today I got an email from a user that was wanting to know the difference between googlegroups and Usenet newsgroups.  They went on further to ask which was a better choice for them based on some criteria.  The were looking for images, so my immediate response for them was “go with a Usenet server”, but then I realized the reasons were as important as my answer.

Short History

Usenet newsgroups have been around since the late 70s/early 80s.  I have operated a server since just before the “great renaming” back in 1987.  My personal experience is that the Usenet has been a wonderful tool for sharing of all kinds from the beginning.

Close to the beginning some people got the bright idea to keep copies of all the articles posted to Usenet.  The Deja News archive of Usenet articles, allowed visitors to their website to perform extensive searches and people could sign up for a free account to post articles in text newsgroups.  It was a wonderful tool, until they tried to make money from it.  In about 1999 they tried to monetize it, and everything went sideways.

Googlegroups

On Feb 12, 2001 Google bought the archives and other resources that were left to the, by then failed, Deja News.  Google used those archives to start Googlegroups.  Google groups are fundamentally the same as Usenet newsgroups, and many of the groups are still peered with Tier-1 providers on the internet.  So, users that post to a Google group about ‘dog behavior’ might get a response from someone using a Usenet server reading ‘rec.pets.dogs.behavior’, and vice versa.

Probably due to the governing board in charge of the “Big 8” hierarchies on Usenet, Google groups created by their users are not created on the Usenet.  Because of this, not all groups will get peered with Usenet servers.  This limits the amount of low-bandwidth or nonsense groups in Usenet, but also lessens the possible amount of content it could provide.

Okay… which is better

Neither, both, either… really depends on your needs.  All things being equal, either should work for someone that is just looking for a discussion group about a topic.  There are many considerations that can change that though.

Considerations
Google Group Usenet Newsgroup
Create Group Y Y1
Subscription N Y2
Software Required N Y/N3
Mailing List4 Y N
Binaries (alt.binaries.*) N Y
Document Sharing5 Y Y

1 – It is possible to create a newsgroup in the alt.* hierarchy just by typing its name into the newsgroup field of your newsgroup client.  If you want to create a newsgroup in the “Big 8” hierarchies you have to prove there is need/interest through a vetting process.  Few new newsgroups are created any more in the “Big 8”.

2 – It is possible to get a free account from some Usenet providers, these are usually speed or size limited and do not have binaries, but if your goal is to join a discussion group, that would be sufficient

3 – there are a several options for reading the Usenet newsgroups

  1. You can purchase a newsreader for Usenet, typical cost is around $20 and some require a subscription for advanced search features.
  2. Some Usenet providers (such as EasyNews) have a web-based interface which is actually much more sophisticated than the one provided by Googlegroups, they support searching for and previewing binaries
  3. there are free newsreaders, but they are normally designed more for Usenet enthusiasts than for casual users

4 – there is no mailing list on Usenet unless you create one… which is very angrily frowned upon.  A posting to a newsgroup is visible to all subscribers; you probably have a better chance of someone reading a posting in a newsgroup than getting passed their spam filter for mailing them directly

5 – document sharing in Google groups is limited.  Although document sharing is not directly supported on Usenet, you can post binaries in some groups.  You could post a document in an alt.binaries.* group, then reference it in an article in a discussion newsgroup.  There is a program (called uBackup) which can be used to “store” documents on the Usenet.

What is a Newsgroup

A Usenet newsgroup is a discussion group within the Usenet system for messages posted by users about a specific topic.  This term is often confused with forums or blogs, but they are different in many ways. In order to read newsgroups you need access to a Usenet server and newsreader software.

Usenet was originally based on the old bulletin board systems (BBS) of the late 70’s and early 80’s.  The primary difference is that a BBS required users to dial in to a centralized computer to read all the articles, and they were not (typically) shared between different BBS’.  Usenet turned that design on it’s head, articles are stored and copied between servers around the world, with each maintaining a separate repository of articles for all the newsgroups.  When servers communicated with each other they would trade missing articles so that all servers eventually would have all articles in their repository.  This design allowed users to dial in to a newsgroup server and share ideas with people from all around the world.

Newsgroup Hierarchies

Newsgroups are grouped together in hierarchies which help users to locate newsgroups they are interested in more easily on the Usenet.  Originally all newsgroups were in the net.* hierarchy, and locating a news group you were interested in could be difficult. In 1987 some of the major Usenet backbone sites got together and rearranged all the newsgroups in what was called the “big-7”, they were:

  • comp.* – computer related newsgroups
  • news.* – Usenet related newsgroups
  • rec.* – recreational activity related newsgroups
  • sci.* – newsgroups for scientific topics
  • soc.* – social issue newsgroups
  • talk.* – contentions issue related newsgroups suck as religion and politics
  • misc.* – miscellaneous discussions that didn’t fit any of the above

In 1995 the “Big 7” was changed to the “Big 8” with the addition of the humanities.* hierarchy for discussion topics such as the arts and philosophy. All of the “Big 8” groups were controlled by a group from the major backbone operators and they allowed users to make suggestions for creating or deleting groups, and those requests were formalized by choosing an appropriate name and group charter.  Finally, the proposal was put to a vote by the group.  Often, in the end, the answer was “No” which frustrated a lot of users who pushed for the creation of a new hierarchy that was not controlled by the “Big 8” group and eventually the alt.* hierarchy was created.

Unlike the “Big 8” which requires users to present a Request for Discussion (RFD) to create a newsgroup, the alt.* hierarchy is the wild west.  Users that want to create their own newsgroup simply put alt.<topic> in the newsgroup to post to and many servers will propagate it to all their peers.  The major shortfall in this design is that; while “Big 8” newsgroups will be carried by almost all Usenet servers, the decision to pass on unknown groups is up to the individual server operators, and some will refuse to carry or propagate those groups.  The upshot of this is that you can create a group, but no one may ever see it but you.

Other Hierarchies

There are many other hierarchies now on the Usenet, many of which are operator or country/language based.  Because of the flexible design of the Usenet software, an operator can create newsgroups which are only available to their users and will not be passed to other servers.  Also, for many non-English users, groups have been created that allow discussion in their country or native language.  Most of these hierarchies have a regulating body (or individual) which maintains the list of allowed newsgroups.  Some of the major hierarchies include:

  • cn.* – for Chinese newsgroups
  • de.* – for German newsgroups
  • fr.* – for French newsgroups
  • it.* – for Italian newsgroups
  • kr.* – for Korean newsgroups
  • tw.* – for Taiwanese newsgroups

Types of Newsgroups

There are predominantly two types of newsgroups, binary or text.  Both work the same way from a server standpoint, and there is no major differences in the naming conventions except that binary groups often have ‘binaries’ somewhere in the name, such as ‘alt.binaries.pictures.rail’ which carries pictures of trains by newsgroup readers.  Often there is a discussion newsgroup for a binary group which will have an identical name but have ‘.d’ added to the end denoting that it is for discussion of postings to the binary newsgroup.

Newsgroups are usually about specific topics, and some are even moderated to ensure that off-topic posts are not posted to a discussion, while others allow users to post articles about a wide variety of topics.  There are currently over 110,000 newsgroups, most of which do not see any activity except for SPAM, but estimates are that about 20,000 of the newsgroups are still regularly used by people for their intended purposes.  A lot of the other 90,000+ are newsgroups created in the alt.* hierarchy either intentionally, or often through a typographical error while typing in the newsgroup to post or cross-post to.

Binary Newsgroups

The Usenet was originally designed to work with US-ASCII characters, and was at that time a very US-centric system.  The Usenet originally only connected a few Universities together, and that was expanded to include corporations, high-schools and some individuals.  It wasn’t long before some users wanted to be able to send binary data in newsgroups and a method (called UUencode) was created which converted 8-bit binary files to 7-bit US-ASCII.  Users that wanted to use the binary data could decode it (using UUdecode) and have a copy of the original file.

The modern Usenet software is designed to work with many international character sets and can transfer those articles without risk of damage to 8-bit data, but articles are still often sent in 7-bit clean formats.  There are also newer programs that can create articles that use 8-bit data such as yEnc.  yEnc’s 8-bit articles are much more efficient at transferring binaries than the older UUencode articles because the conversion from 8-bit to 7-bit has an approximate overhead of about 40%, whereas the overhead for yEnc is much lower, usually as low as 1-2%.

Another restriction of the original Usenet servers was often article size, this restriction required that larger binaries be broken up into multiple parts which they poster would then post manually.  Users that wanted the original binary would download all the parts, and put them back together manually before decoding. This process was tedious and error prone, plus servers often would not get all the parts of a posting, or might delete parts before the user had time to download all of them.  The high error rate caused the ‘.d’ discussion groups to often be filled with requests for “part 13 of 53”.  Modern newsgroup readers are able to handle multi-part binary postings internally and the users usually needs to just pick the binary they want and come back later for the completed download.