Network News Downloading
[ Home ] [ Back] [ Next ]
How to download from Network News

Introduction 1. Introduction
SocksCap32 Setup 2. Connecting
Attachments 3. Attachments
Joining 4. Joining
Viewing 5. Viewing
Resources 6. Resources

1. Introduction :-
To download files from a newsgroup, you will need access to that newsgroup first. So you'll need a program to read newsgroups and a NNTP server that carries the newsgroup you want to open.

If your ISP doesn't have a NNTP server, or it has but it doesn't carry the newsgroup you like, or it only gets a few posts for that newsgroup, you can try out a free NNTP servers from the list in the resources section.

A competent news reader application is Forte Agent, it has a small footprint, can also automatically rejoin large files and its administration is easily managed. Alternatively the news reading functions built into your browser whether Internet Explorer or the Netscape Browser work just as well, but they are not a satisfactory long term solution, since they leave a url audit trail, which some who value their anonymity online will wish to avoid.

Some news articles arrive whole, others arrive in parts, each part possibly encrypted, or compressed or both. So recognizing that you may need to perform one or all of the the following operations Joining , Decryption , De-compression is part of the skill in successfully downloading files from Network News.

2. Connecting:-
To open a newsgroup from a browser, you have to enter in the location field (the one in where you normally enter the URL of a web page) the "news" protocol followed by the DNS or the IP address of the NNTP server you intend to use, and the name of the newsgroup you want to open, such as in this example:

news://testnews.execpc.com/alt.binaries.pictures.animated.gifs

To open a newsgroup from a news reader program (Agent Forte) select from your "Subscribed Groups" your news group. then from the online menu select "Get New Headers in Selected Groups", this will refresh your news reader with new articles. Now simply select and double-click an article to retrieve it from your news server.

The content of a newsgroup looks similar to your normal mailbox: you can read the posts by clicking on them. If they have an attachment it usually will appear as a link at the bottom of the message. All you have to do is to right click it and save it to disk. If they are chunks of a large file, name them with a progressive number (for example "File1", "File2", "File3", or just "1", "2", "3"...), to make things easier when you'll have to rejoin them.

If the attachment is a picture it will be shown directly, unless it is encoded, or you have disabled this function, then you can right click the picture and save it to disk.


3. Attachments:-
The Internet was not designed to carry binary (program and other non-text files) files. It is able only to transfer messages made up of conventional text (printable ASCII) characters. In order to get around that limitation, UUencode and other methods were created. Known as "attachments" these binary attachments are binary files attached to an otherwise pure text file, and are encoded in some way, usually your news agent just decodes them automatically for you, still some encoding can't be decoded automatically, so other methods are used instead.

These solutions all perform the same basic operation: they encode the non-transferable binary file into ASCII characters that Network News or the e-mail system can handle. The person receiving the message can then decode the strings of characters to recreate the original file. Perhaps you have seen such one of these completely unintelligible messages; here's an example:

_=_
_=_ Part 001 of 001 of file encoded.txt
_=_
begin 666 encoded.txt
M5&5S="$-"@T*1V5N=&QE(%)E861EBCH-"@T*5&AIDR!ICR!N;W1H:6YG(&UO
MF4E=&AA;B!A('1EAW0B9FEL92!CBF5A=&5D('1O('!R;W9I9&4B9F]D9&5R
M(A9OFB!T:&4G=F%R:6]UBR!E;F-O9&EN9R!S8VAE;65S+B!)9B!Y;W4A87)E
M('5S:6YG(&ET('1O('1EAW0L(&-O;F=R871U;R%T:6]NPR!O;B!Y;W5R(S%G
M:6QI='DA:6XP8W5T=REN9RPBS&%S=&EN9RPPAV%V:6YG+"!A;F0E9&5C;V1I :;FEG=7-I;FGA5VEN6FEP+HT*#0I%;FIOQ2$`
end

Programs such as Fastcode and WinZip can easily open and extract UUencoded, XXencoded, BinHex, and MIME files (base64, plain/text, and quoted-printable) that have been sent to you, and create Uuencoded files for you to send.

In the previous example The UUencoded data is a series of lines of ASCII text characters which are normally 60 characters long and begin with the letter "M". Since both Netscape and Agent tells you how many lines are in an article you can estimate the size of a message before opening it.

Message Size:
The approximate size in kilobytes of a message can be calculated by multiplying the number of lines by 60 and then dividing the result by 1024. This would be the size of the file you are downloading. To know how many Kb will occupied on your hard disk, you'll have to multiply it by 3 and then divide by 4. If, for example, you find a message which is 1000 lines long, you'll have to download 1000*60/1024=58.5Kb, and it will occupy 58.5*3/4=43.8Kb on disk. When UUencoded files are saved as stand alone files, generally have an ".uu" or ".uue" extension.

Manual Decode:
If you get an un decoded UUencoded attachment, you'll have to decode it "manually" first. You can proceed this way:

Copy the whole body of the message (usually with CTRL+A to select all, followed by CTRL+C to copy it), then paste it (CTRL+V) on a good text editor. Don't use the standard Windows Notepad because it can't handle large text files and have problems with some special characters.

However, once you have it in your text editor, delete all the lines which aren't part of the UUencoded file (everything above the "begin" line and below the "end" line) and save it to disk with the name you want. Now you can decode it with Fastcode by selecting it into the main window listbox then Clicking on the Binary Decode button, or from Winzip by highlighting it with the mouse then :-
  • Double-click on the file with the .UUE extension
  • WinZip will execute and decode the file. If it is a Zip archive, it will offer to open it. If it has not been compressed, it will be shown in the usual manner.
  • Use WinZip in the regular manner to Extract the file to your folder of choice.
Or using the Aladdin Expander (drag and drop the file inside this tool's window). You can also decode it from the MS-DOS command line with an uuencode/uudecode command you may find in some software archive. Remember to delete the UUencoded file you saved after decoding it, for it will be only a waste of space on your hard disk.

If the file you are downloading has been UUencoded and then split in multiple chunks, it may happen that single chunks don't have the "begin" and the "end" lines. In this case, after you have deleted all the useless lines, leaving only the ones that begin with a "M", manually add the begin/end lines, as in the following example:

begin 623 File3
M)C`A(#TJ*C,I)S0T-#@Z.2PS,$PT-4DX.3\M+B,L2%P_03U#0TA(2%=(2%-5 M5BI;6U195TU24&M,3EY%1GM86'=96F5866ML:V-F97-TRW)M;59K8HMD981F
end

Missing Parts:
First try to open that newsgroup from another NNTP server. Since news posts are passed from server to server, it may happen that your server didn't get all the posts, but you might find them in another one which has a superior file input and a longer article retention period.

If no other news server has the missing parts, then you can post a message asking the original poster or a friend who has all the posts, to post the missing parts again. This is a common practise and people are normally happy to oblige.

Remember:
If the file you have downloaded is an executable file, DELETE IT AT ONCE. Never run executable program files downloaded from a newsgroup. They will PROBABLY be trojans or viruses. Even if you trust the sender, you can't be sure if someone has not forged their identity. if you need an application go to the Software producers homesite and download from there.


4. Joining :-
How files are joined depends on the method used by the poster to send them initially. Files that are intended to be joined using another program tend to have digits appended to the filename ie filename.mpg.001.pgp, filename.mpg.002.pgp etc and in some cases an additional extension here its .pgp since each part has been encrypted. Files within a news group that have been automatically split by the news server will have the format filename.mpg [1/9], filename.mpg [2/9], filename.mpg [3/9], and your news reader should normally be able to Join these where the contents of the brackets is the article part number and the total number of parts, usually where there are missing or incomplete parts joining is unfeasable, but not impossible. So once all the articles have been retrieved, some common operations which can be performed to complete a download are Decryption, Joining, and/or Decompression and Renaming.

The Join Sequence:
This is normally determined by the reverse order of the file extensions on the target parts, running from right to left. And one or more of the following operations are normally used.

  • Decryption
  • Decompression
  • Joining
  • Renaming
So for a set of files that started with one named hanson.mp3.001.pgp Then running from right to left the Joining Sequence would be:
  1. Decrypt all the file parts using PGP
  2. Join all the files

Example Join Sequence:

Step 1. Decrypting the parts:

Highlight all the parts, then right click, selecting PGP using the Decrypt option.

Highlight the parts. Decrypting the parts.
The PGP password box should appear so just enter your password.

Enter 
the Password

Step 2. Joining the parts:

Run Mastersplitter and select the Join tab then using the "Browse" button search for the file with the .000 sequence number, here we search for "Hanson.mp3.000". Then click on the Join button to complete the file join. Joining the parts.

Now for a set of files that started with one named hanson.mp3.001.zip Then running from right to left the Joining Sequence would be:
  1. Decompress all the file parts using WinZip
  2. Join all the files
For a set of files that started with one named hanson.mp3.001.rar Then running from right to left the Joining Sequence would be:
  1. Decompress all the file parts using WinRAR
  2. Join all the files
A variation on this theme is for the poster to Encrypt or Compress the file first, then to Split the file into parts.

The simple case is that of a file being Split followed by Encryption. So for a set of files that started with one named hanson.mp3.pgp.001 Then running from right to left the Joining Sequence would be:
  1. Join all the files
  2. Decrypt the resulting files using PGP
Now for a set of files that started with one named hanson.mp3.zip.001 Then running from right to left the Joining Sequence would be:
  1. Join all the files
  2. Decompress the resulting files using WinZip
For a set of files that started with one named hanson.mp3.rar.001 Then running from right to left the Joining Sequence would be:
  1. Join all the files
  2. Decompress the resulting files using WinRar

Using Winrar to download is discussed further here Renamed Parts:
You can Rename all the files to eliminate the numerical suffix. An example might be hanson.mp3.pgp.001 renamed to hanson001.mp3.pgp. So for a set of re-named files that started with one named as hanson001.mp3.pgp Then running from right to left the Joining Sequence would be:

  1. Rename all the files in ascending numerical order i.e. from .pgp.001
  2. Join all the file parts
  3. Decrypt the resulting file using PGP
And for a set of files that started with one named hanson001.mp3.rar Then running from right to left the Joining Sequence would be:
  1. Rename all the files in ascending numerical order i.e. from .rar.001
  2. Join all the file parts
  3. Decrypt the resulting file using WinRAR
Finally for a set of files that started with one named hanson001.mp3.zip Then running from right to left the Joining Sequence would be:
  1. Rename all the files in ascending numerical order i.e. from .zip.001
  2. Join all the file parts
  3. Decrypt the resulting file using WinZip

News Reader Join:
File parts will have the format filename.ext [1/9], filename.ext [2/9], filename.ext [3/9] ... etc. The parts may have an icon image denoting a split part. First sort all the articles by author and check you have all the parts. Highlight these and ask your newsreader to download them all.

After the downloading is completed, click on the first part, hold down the SHIFT key, then click on the last part of the file. Right click and from your menu select the "Combine and Decode." operation, normally a dialog box is presented so you can arrange the parts in some kind of sequence. The program will then combine the parts. Once finished some newsreaders present you with the Filename its icon image type, size etc, and an option to run a program say Windows Media Player, or your favorite image viewer. You can save now or once you have viewed it you can save it to disk using the "Save Attachments." menu command. Some newsreaders, like Agent Forte, have an AutoJoin option where all incoming split articles are automatically joined.

Program Join:
Another popular method for joining files is to use a program that has been designed for this purpose. Files that are intended to be joined using another program tend to have digits appended to the filename ie filename.pgp.001, filename.pgp.002 etc. Files are normally split this way to allow for selective reposts. Generally the poster will indicate which program has been used to split the file. Normally the first part must must be present to have a successful join. First download all the file parts and save them to a named folder.

Now identifying the Join Sequence run the program used to split the file. Select the "Join" operation, or its equivalent. Browse to the folder containing the parts, select the first parts, which is sometimes filename.000. Then click "OK" to join. If you have been successful some dialog box will appear announcing "Join was successful". All you need do now is check that your file works fine, by applying the Decryption operation, then you can delete the original parts.

Batch File Join:
Files posted in parts, are sometimes accompanied by a .bat file. This method allows you to join files without using a joining program.

As before download and save all the parts to a named folder on your hard disk. After identifying the Join Sequence . One of the files will have the file extention .bat. Highlight and Double-click on the .bat icon. A DOS window will open and the file will start to be joined. A message will appear in the DOS window once joining has been completed. Just close the DOS window. Apply any further Join Sequence operations, then all you need do is check that your file is working fine then you can delete the parts.

Custom Join:
Custom join usually applies when the first file, a .bat file, or possibly even the joining program is not available!

Use Another Splitter
The simplest solution is to download a copy of a splitter program. Then select an arbitrary file that is similar in size to the sum of the splitted parts. Next Copy and Paste this file renaming it with the same name as your splitted file. Then run the split program inputting your "copied and renamed" file as the file to split, select the same number of parts and request that a .bat file should be created. Now delete all the file parts and the copied file then use the resulting filename.bat to join your file parts.

Rename Another .BAT File
Alternatively use an ascii type editor to edit and change an already existing .bat file, Notepad is fine here, but one with a "search and replace" command is better. First rename the .bat file to one with a .txt extension i.e. "filename.dat.bat.txt" then open it with your ascii editor.

Add Join Lines
Now check that the .bat file contains the required number of join lines. If not then use simple copying and pasting of lines followed by a sequential renumbering till you have the required total.

Rename The Original File
Then if the name of the file within the unedited .bat file is say filename.dat.001 and your split file name is filename.mp3.001, then use the search and replace facility to replace "filename.dat" with "filename.mp3".

Save and Join
Check to see that everything looks ok, then save your filename.dat.bat.txt, next rename it to filename.mp3.bat, now run it to join all your parts, verify your file works then clean up before deleting all the parts, but keep your filename.mp3.bat, since this can be used in some future join excercise.

Manual Join:
Manual join usually applies when you have a list of uu encoded files, or encoded using some other method, usually from news, which refuse to be auto-joined or have syntax problems.

First try using Fastcode32 or WinZip which both allow the decoding of files encoded using mime, uu, binhex, etc. This helps sort out the valid parts.

Click on the MS-DOS Prompt shorcut or menu option and navigate to the folder containing the file parts. Now you can use the COPY command as in the following example:

COPY /B File1+File2+File3+FileN DestFile.ext Where DestFile.ext is the name of the destination file name with the appropriate extension (.MPG for MPEG movies, .MP3 for MP3 audio files, .JPG or .GIF for pictures, etc )

The option /B is necessary because you are joining Binary files. If you try to join binary files without the /B option, they will be treated as ASCII (pure text) files, and the copy process will end as soon as the computer will meet a byte with a value of 0, because it will consider it as an End-Of-File (EOF) marker.

If you have several chunks and you can't put all of their names in a single MS-DOS command line (there's a limit of 127 characters for a single command line), you'd better join them in larger chunks first, as in the following example:

COPY /B File1+File2+File3+File4+File5 File1-5

COPY /B File6+File7+File8+File9+File10+File11+File12 File6-12

Then ...

COPY /B File1-5+File6-12 Track.mp3

Missing Parts?
A quick fix for this is to nominate the previous file as the next file to be joined. So if filename.mp3.009 is missing then first copy filename.mp3.008 renaming it to filename.mp3.009 if neccessary, then join and verify as before.


Viewing Files :-

Playing:
Microsoft Windows MediaPlayer handles most audio/video formats. Be sure to have the latest version installed.

Sound:
If the file is an MP3 you may want to use a specific player as WinAmp.

Pictures:
Pictures in almost any format can be opened with ACD-See32 (shareware) For a larger selection of graphic formats you can try a professional application such as Corel PhotoPaint. PhotoPaint will allow you can to edit pictures and adjust colors, contrast, or sharpness.

In this case you should save the file with the same name, but adding an "r" as a suffix to the file name: is a good rule to never change the name of a picture, especially if it's part of a series.


6. Resources:-
See the other faqs

  • Network News Providers
  • Network News Readers
  • Network News Agent Setup
  • Network News Posting
  • Network News Downloading

Free News Servers: -
http://freenews.maxbaud.net/
http://tools.rosinstrument .com/proxy/news.htm
http://newssearch.pilum.net/
http://www.wnabb.freeserve.co.uk/

Proxy Lists:
http://tools.rosinstrument.co m/cgi-bin/fp.pl
http://proxys4all.cgi.net/
http://www8.big.or .jp/~000/CyberSyndrome/pl1a.html
http://www.cyberarmy.com/lists/ proxy/

Proxy Checkers
http://checkpoint1.hypermar t.net/index.html
http://www.ren tal-web.com/~azuma/cgi-bin/env.cgi
http://mizuno -labo.cs.inf.shizuoka.ac.jp/~s5087/proxy.html
http://www.all-nettools.com/tool s1.htm
http://allfree.webprovider.c om/proxychk.html
http://www.tamos.com/netres/pr.cgi
http://www.all-nettools.com/

Country Check
http://www.all-nettools.com/tool s1.htm
http://www.arin.net/whois/
http://www.samspade.org/

Cracks:-

Forte Agent by Forte
Is an all round basic news agent that is easy to use and understand. It comes in two versions: Agent, which is the basic version is a capable news reader. And Forte Agent which adds email features for those that use email to communicate.

Programs:

News Hunter:-
News Hunter - Using News Hunter, you can find your own open news servers whenever you want. News Hunter is only available for Windows at the moment. You can configure News Hunter to search for specific news groups on each server it finds. It will seek out and test each server and give you a list of ones it was able to access that met your criteria. You can then configure your news reader with the newly discovered server and participate in newsgroups using a server other than your default server! News Hunter is Freeware

SocksCap32 Program
SocksCap32 will allow you to anonymise yourself or sockisfy your connection to network news and other protocols such as irc ftp etc. It is a versatile and valid alternative to using proxy servers directly.

Autopost32:-
Autopost is the easiest way to post a large number of binaries to a single newsgroup. Autopost will automatically do all the encoding for you. Using AutoPost will when it omits the NNTP Posting host line allow you to post these images with anonymity. It is important you make TEST posts to see which ones omit this particular line as not all do. Autopost is Shareware.

MasterSplitter:-
MasterSplitter is a simple to use program that allows you to set the size and number of splitted file parts and will also generate a batch file which can be used to automatically join the parts together again. MasterSplitter is shareware.

Fastcode32:-
Fastcode32 supports the following formats Mime (Base64) UUD/UUE, XXD/XXE, AtoB/BtoA and Binhex. If you have browsed newsgroups and seen files that contain weird characters, this means they are encoded. Fastcode32 will decode these files using drag and drop. It will also decode multipart MIME files. Additionally, Fastcode32 will ENCODE files for posting.

WinZip
WinZip is the most widely used archiving program. It also allows each zip archive to be password protected.

Aladdin Expander:-
Aladdin Expander, which can handle typical Mac compressed files such as .HQX or .SIT (StuffIt).

Winrar:-
WinRar is an all round first class compression program that also allows the compressed files to be password protected. This program should operate on all the industry archive types.

Media Player:-
Microsoft Windows MediaPlayer handles most audio/video formats, such as MPEG, AVI etc.

ACD-See by ACD Systems this is an all round picture viewing program that will work from inside forte agent as your picture viewer of choice.

Corel PhotoPaint by CorelPhotoPaint will allow you to edit pictures and adjust colors, for contrast and/or sharpness. To transfer pictures to other file formats, and layer text onto pictures.

WinAmp Is an all round music player.

Opera Is an excellent small footprint web browser.

Computer Partners Brief explanation of newsgroups and instructions for using Netscape Navigator or Outlook Express for reading newsgroups.

Rules for posting to Usenet Brief introduction to Usenet and appropriate guidelines to follow for posting.

Free Advice on Usenet News An editorial introduction to Usenet.

The Usenet Newbie Project Learn everything about the Usenet in one place. How to access it, how to read posts, explanation of technical terms and acronyms and how to create your own Newsgroups.


Home Back Next