How to securely cleanup
|02. Deleted Files|
|03. Secure Deletion|
|04. File Cache|
|05. Swap File|
|06. Recycle Bin History|
|07. Printer Spooler Files|
|09. Email and Attachments|
|10. Directory Entries and Shorcut Files|
|11. Browser History|
|13. Temp Files|
|15. Containers and Partitions|
The aim of this article is to bring together from a list of diverse sources methods that will ensure corporate and personal data security, explain what constitutes secure deletion and the various items that often require secure deletion. Its primary focus is Windows98 but refers to Win 2000 and WinXP where relevant. Due to the evolving nature of modern operating systems it can be considered as work in progress.
Your Operating System stores information about the programs you use, the documents and pictures you view, including many other things. Each application you use generates its own history data, access logs, temporary data files, and each file contains time and date stamps. To ensure corporate and personal data security, this information, along with any Recycle Bin activity, all of which constitutes an electronic data trail, should be securely deleted so to avoid corporate or personal information leakage.
Regular secure deletion is always a defense against information leakage, the inadvertent storage of items covered by Intellectual Property rights, Munitions or Pornography laws, since the laws in most countries allow for the accidental downloading of transgressive material, as long as it is immediately deleted! A corporate or personal policy that includes regular secure deletion can never have their legal liability questioned or misrepresented as a last minute desire to purge a computer of offending material!
The program I use for removing sensitive information is Eraser. This is an advanced security tool (for Windows), which allows you to completely remove sensitive data from your hard drive by overwriting it several times with carefully selected patterns. It can also be scheduled to run at set intervals to wipe free space or specific folders and their contents, as well as overwriting the swap file at startup.
Download Eraser, and put eraserd.exe in your C:\ folder.
There are several other overwriting tools that appear to deliver similar results, these i have itemized in section 3 along with other useful programs.
The regime of regular wiping and cleaning can become tiresome and possibly prone to information leakage. So an alternative to consider is to clone or ghost a complete hard disk. Cloning is the creation and maintenance of a complete image or copy of a computer's hard drive. This image would be the one created just after the operating system is installed, along with all your application programs. The cloned copy would retain all your system settings and assuming your hard disk had been suitably overwritten, defragged and formatted, there would be no personal or corporate information at the start of each session.
Another option would be to install windows into a RamDisk, using external media as data backups. So that once your computer session was finished and you had powered down, flushing the memory, there would be no personal or corporate information left on the computer.
It is generally accepted that "delete does not mean delete", Because when a file is created, a directory entry for that file is also created. When a file is deleted and not sent to the recycle bin, the first letter of the filename in the directory entry is changed to a special character (Hexadecimal E5). All entries for that file in the File Allocation Table are then cleared. The data contained in the file remains on the hard drive until it is overwritten. Theoretically, this data could remain on the hard drive forever!
Finding Deleted Files:
It has been established that the data contained in deleted files remains
on the hard drive. Where are they located? How can they be viewed? How
can they be recovered? All the data on the drive can be viewed using
a tool such as WinHex. From an "oops"
I would like to get that file back standpoint undelete tools exist that
may be able to retrieve a deleted file. File Rescue, and Recover NT are stand alone utilities
that claim to be able to recover deleted files. Directory Snoop has an unerase feature. It is
important to remember that files can be recovered only if they have
not been overwritten
It has been established that there is a large amount of data on a hard drive that a user does not create. It may not be visible through standard interfaces, but can be found in several locations, including slack space and unallocated space. It is important to understand these terms, since overwriting data in these areas is one of the keys to preventing data from being recovered.
Windows operating systems use fixed sized clusters to store data. An entire cluster is used even if the data being stored does not fill the cluster. The space between the end of the file and the end of the cluster is called slack space. A visual representation might look like this:
Using this concept, if a cluster is filled with data, and then the cluster is only partly overwritten, the data in slack space is recoverable. It can be viewed and recovered using a tool such as Norton Utilities DiskEdit, but is more efficiently recovered using forensics’ tools such as NTI’s GetSlack and Guidance Software’s Encase.
Unallocated space (more accurately, unallocated clusters) can be defined as clusters that are not currently allocated by the Operating System or File Allocation Table. Essentially, unallocated space contains all of the deleted files (among other types of files) on the drive that have not yet been overwritten. Once again, this data can easily be recovered using forensics’ tools such as Encase and NTI’s GetFree.
Now that it has been established that a computer hard drive can contain a wealth of information, how can a security professional insure that corporate laptops are not leaving with unprotected proprietary information, donated computers are free from confidential records and that no documents, e-mails, memos can be uncovered and used against a corporation in a lawsuit? The following sections contain steps and suggestions to reduce the risk of loss of proprietary information.
Deleted files can be recovered, using electron scanning technology. Is it possible to delete a file (and its associated files, temporary, spooler, etc.) so that it cannot be recovered? From a corporate or personal perspective, an individual will have to determine the value of his data and determine the steps that can be considered "reasonable and practical" to prevent proprietary or personal data from being stolen or recovered by competitors or groups intent on corporate espionage. The main premise for preventing data from being recovered is to overwrite it. The question becomes how many times should it be overwritten? The more often the data is overwritten, the less likely it becomes recoverable by any means. For a drive currently in use, it is necessary to overwrite slack space and unallocated space. Most secure deletion programs use one of three following overwrite methods:
Single Pass – data area is overwritten once with either 1’s, 0’s or pseudorandom data
DoD Method – the data area is overwritten with 0’s, then 1’s and then once with psuedorandom data. Many tools use variations of this, overwriting as many as seven times, using three alternating passes of 0’s and 1’s following by one pass of psuedorandom data. This is based on standards outlined in the Department of Defense Manual 5220.22 M, also known as the National Industrial Security Program Operating Manual or NISPOM. This manual outlines the steps to both "clear" and "sanitize" a "rigid non-removable disk". To clear a disk it states that you must "overwrite all addressable locations with a single character." To "sanitize" a disk you must do one of the following:
Guttman Method – the data area is overwritten 35 times. This method uses psuedorandom data to overwrite the drive and overwrites the drive taking into account the different encoding algorithms used by various hard drive manufacturers, RLL (run length limited), MFM (modified frequency modulation), PRML (partial-response, maximum-likelihood). This method of overwriting data was created by Peter Guttman, and is described in his paper, "Secure Deletion of Data from Magnetic and Solid State Memory."
Overwriting the data only reduces the likelihood of data being recovered. The more times data is overwritten, the more expensive and time consuming it becomes to recover the data. In fact Peter Guttman states "…it is effectively impossible to sanitize storage locations by simple overwriting them, no matter how many overwrite passes are made or what data patterns are written."
But for practical purposes regular secure deletion, using some variation on the DoD method, with a rolling backup procedure should ensure minimum safety of most personal and corporate data. The Guttman Method can be applied at intervening periods, and Eraser uses Gutmann overwriting.
Steps to Securely Remove Files and Associated Data from a hard drive:
Checklist of items to include for secure deletion
Most operating systems have a cache manager which organizes the block caching functionality of system memory, for file data stored on secondary media, e.g. hard disk, cd rom. This is measured in blocks of memory termed Pages and the dynamic details are normally held in a Page Table. This file caching feature is meant to speed up the efficiency of data throughput and is generally an invisible operation. Depending on the size of the File Cache either fragments or whole documents of sensitive personal or corporate information may be left in the file cache.
Since write caching has the potential to reduce the effectiveness of secure deletion software, unless the disk buffers are specifically flushed regularly while secure deletion is in progress. If you do use windows, and have disk caching enabled, it's probably a good idea to disable it.
If you have a document you are working on using Word, it may create a temporary file with your documents contents. When disk write caching is disabled, the file is written directly to your hard drive. When disk write caching is enabled, this file may not ever be written to disk provided Word deletes the temporary file before the cache is flushed to disk.
The Win95/98 swap file is named [ win386.swp ], under Windows 2000 its known as the Page File and is named [pagefile.sys] this is normally listed in the Windows folder. Its presence is essential to allow Windows to run properly. Its function is to act as a fast cache/buffer file.
If Windows runs out of physical RAM (for what ever the reason), it swaps the least recently used memory areas to your hard drive and then re-uses the RAM. Example: A program needs data that was just swapped out (for what ever the reason), Windows will swap it back into RAM. It's as basic as that, this is the way the Windows shell was designed.
Some programs use the swap file even if RAM memory is available For example, if a user has an email document open but is currently working on other applications, as the user switches to the other application, lets say a Instant Messenger. The operating system will save the email process's memory state e.g. text and all, in the swap file on the hard drive in order to free up memory for the other, most recent active application.
Swap files can be quite large, with 300 megabytes of memory not uncommon. Its normal for the swap file to contain fragments of word processing documents, email attachments, internet web pages, instant messaging and chat room conversations, and other data created and/or viewed in past work sessions. This may include Recent File history lists that are linked to that application lets say Outlook Express, or similar URL history lists relating to your previous browsing session with Internet Explorer.
The same applies to your Image Viewer which will save the names of pictures looked at in its history list, or Media Player History, which has both a public history list saved to the registry and a private one since it performs phone home tracking for MS, using COM GUID's stored within the registry. All this is evidence that pictures or files found in the Internet Temporary File cache have been viewed, and not deleted or ignored as the user may claim!
Also remember viewing a document on the screen, but which is never saved to your hard disk, because you powered down the computer, may have recoverable fragments available in the swap file!!
It you do use a swap file over writer, it is recommended that you do not let Windows manage the size of this file. If you were to shred your free disk space under Windows, and then reboot into DOS to overwrite the swap file, you could find that while the free space was being shredded, Windows had setup a large swap file (say 150MB), which it reduced down during shutdown (say to 25MB). In this case, even if you overwrote remaining swap file, you could potentially have part of what was previously stored in the swap file (125MB of it, in this example) still held on your hard disk. Understanding that the swap file is dynamic and recreated with each windows session at startup, leaving previous swap file data transparent in free space, is important to understanding why it needs to be overwritten. And not simply zeroed using a system call to the windows api. To work out the size of the swap file needed, take a look at it using flipTech SwapMon over a long period of time.
Tip: The Swap File can be placed in a partition or on a ramdisk, this means it is stationary in one location, making overwriting and cleaning simpler.
Find Using: Windows Explorer, Find Files or Folders, File Extension text search on the hard disk.
Cleanup With: Eraser.
Windows tracks files placed in the Recycle Bin by the user by creating temporary INFO files that, when recovered and assembled by investigators, present a history log documenting a user's file deletion activity. Recycle Bin INFO files and fragments are usually randomly distributed across hidden areas of a hard drive. An investigator can estimate when a user deleted particular files, the sequence of deletion and other important file metadata, even if those files had long since been removed from the Recycle Bin.
There are three places from where recycle bin history can be found.
INFO files usually contain a users file deletion history, since files automatically deleted by the operating system are not recorded in the INFO file. From this distinction in deletion method, a users intentions at the time of deletion can be inferred. Therefore any user who deletes a large number of files in a critical time period, say prior to the user being subject to a legal discovery process, or who claims the file was accidentally saved, may have his claims contradicted by the time date stamps in his file deletion history.
Cleanup With: Eraser.
In Microsoft Windows the default setting for printers is to "Spool print jobs so program finishes printing faster."
Spool is an acronym and stands for "simultaneous peripheral operations online". The significance of spooling is that the application sends the file to the hard drive first and then to the printer. Because the file is copied to the hard drive, the data it contains will remain on the drive until it is overwritten. A key security concept to remember is that even if the file is never saved, but only printed, it may be possible to recover the data in the original document. These files can be recovered using forensics tools and then viewed using an image viewer that supports enhanced metafiles (notice the data format in the screen shot).
EMF files also contain further information that can reveal when a certain file was printed, where it was located on the computer, etc. Metafiles are used most often for sharing pictures between programs through the clipboard, but can be stored in the form of clip art on the hard disk.
There are three kinds: WMF, EMF and EMF+.
Cleanup With: Eraser
Metadata can be described simply as "data about data". Although metadata is not a separate file, the data it contains is created automatically by Microsoft Office products. Understanding what is contained in metadata provides another reason to verify that sensitive files are completely removed from a drive. From a security standpoint, metadata may contain information that should not be shared outside of an organization. What can be found within Metadata? According to Microsoft Knowledge Base article Q223790 the following are examples of metadata that can be stored in documents:
Cleanup With: Eraser
Email files are a source of information leakage both at a corporate and personal level. Each email program will generate its own proprietary email file format, for example, Outlook Express 4 email files extensions are denoted as .mbx and Outlook Express 5 email files as .dbx extensions. These are normally stored within a compound file in a folder under the WINDOWS directory named inbox.mbx or inbox.dbx . If you use another email program then identify its file extension, folder location and factor it into your disk sanitization procedures.
If you rely on a Web based email provider such as hushmail or hotmail then the email content will be stored as htm or html files and whole or remnants may remain on your hard disk. Also your internet history files will log your visits, including url, to your email provider. Your outgoing and incoming emails may be stored indefinitely at your ISP or at any email server it passes through on its journey to its destination.
When computer files are transmitted over the Internet as email attachments, a process similar to the printing process occurs, where Windows generates duplicate temporary files of the files being transmitted as an attachment. For instance, if a user were to transmit a graphics file he/she created on their computer over the Internet as an email attachment, Windows would create a copy of that file in an encoded format, usually MIME (Multi-Purpose Internet Mail Extensions) or uuencode. Usually files that have been deleted and overwritten can be found in duplicate in the form of a MIME file. Any file thus recovered may yield plaintext information, which would defeat any encryption process applied during email processing! So as well as securely overwriting System folders remember and find your email programs storage folder and enter the path to emails or storage file as an entry in the Eraser Task List so it can be sanitized.
Add the following folder to the Eraser task list:
Cleanup With: Eraser
The Windows operating systems will create folder entries on a drive whenever a user moves or renames a file. The folder entries that are created and deleted during the moving and renaming processes contain information which an investigator can use to identify the user's activities, such as determining when a file was created, modified, moved, or renamed, and from what location. In systems utilizing the Windows NT file system, including Windows 2000 and XP, a forensic examination of deleted Master File Table records and index buffers will accomplish many of the same functions.
Link files, sometimes named shorcut files, provide a similar insight and information about a user's activity on a computer system. When a user accesses a particular file, Windows creates a temporary link file that points to that accessed file. This enables the user to access recently opened files by clicking on the Start button and selecting Documents, where the operating system then displays a list of recently opened data files. The user can select a file from this list, causing the file to be opened by its registered application. These link files are stored in the Windows/Recent folder, where by default fifteen such entries are maintained before being deleted by the operating system. However, forensic examiners can often recover hundreds of these files from a hard drive.
The recovery of such link files can reveal the identities of data files opened by the user, and can often provide a wealth of information about how the computer system was configured on a given date and even reveal the existence of disks with relevant information that were not produced for examination.
Add this line to your Autoexec.bat file:
Find Using: Encase, WinHex, Windows Explorer, Find Files or Folders, File Extension text search on the hard disk. .LNK , .SCT, the shorcut identifier varies depending on which type it is, ie network folder or a cd rom drive. File header search for 4C 00 00 00 ‘L’ i.e. the shortcut files Magic value in hex.
Internet cache was originally designed to "speed up" your browsing experience by storing most of the web sites that you visit as local files on your hard drive. Even if you visit a web site only once, it is stored on your hard drive as "cache" until it is removed, and is usually stored under the windows directory in the "Temporary Internet Files" folder. Anyone using forensic software can reliably re-trace your steps through the Internet, connecting and examining the sites you've visited - and even the individual pages and pictures on those sites. Simply deleting this using the DOS DEL or DELTREE command will not remove it, it must be overwritten! The same applies to any other folders under the WINDOWS directory containing sensitive corporate or personal information. e.g.
Along with storing each site you visit as internet cache, browsers will also store a full chronological report of all of the web sites you have accessed - even if you didn't "type" the URL into the browser (e.g. links and buttons).
Another location where your browser will store your preferences, i.e. 'autocomplete' site addresses, Favorites, and Recent Url Visited lists, is in the registry. This, however, leaves an audit trail behind - exposing the URL of each and every site you visit to anyone who wants to know. These lists can be removed using Window Washer. It is not recommended that you use Window Washer for overwriting the swap file or other folders such as the Windows Temporary Internet Files folder.
To securely delete the locked history file index.dat every time you boot your computer, add the following lines to the top of your Autoexec.bat file, a plain text file that is located on the C:\ directory of your hard drive. Commands listed in Autoexec.bat run in DOS mode while your computer is booting up, before Windows starts. If you delete (or wipe) the Internet Explorer history files from DOS at boot time, you can bypass the Windows locking function that Microsoft uses to force you to keep your Internet Explorer history files intact forever. Look in the Home Directory of Eraser and you should find a program named eraserd.exe. Now to enable secure deletion under dos, just move a copy of eraserd.exe to your C:\ directory, and add these lines to the top of Autoexec.bat:
Note: You can set the "passes" variable for any number you like. This tells eraserd.exe, how many layers of random data to write over the files before deleting them. A setting of 1 is sufficient to prevent software from un-deleting the wiped files; 7 or above should be sufficient to keep professionals from being able to recover the wiped data.
Cleanup With: Eraser
12. Cookies:- Cookies are small text based files that websites "give" to you, which are stored locally on your hard drive. This is done so that they can track your browser state during your current viewing session. Most of the time you are completely unaware that these files are being stored on your system, and are never told exactly what kind of information they contain. Cookies usually contain very private statistical information on your browsing habits such as where you came from, where you live, what your e-mail address is, etc. Cookies are one of the biggest information leaks in existence! There are many myths about cookies, which are best dispelled by looking at a site such as www.cookiecentral.com. DoubleClick is an agency that supplies the ads that appear on many of the net's most popular sites. Using cookies, DoubleClick can uniquely identify you, allowing a profile of the type of sites you visit to be built up. Solution: In your browser disable all cookie access and clean regularly!
Add the following folder to the Eraser task list:
and add this line to your Autoexec.bat file
Find Using: Windows Explorer in Windows Cookies Folder
In an effort to improve performance and efficiency, many applications create temporary files. Microsoft Knowledge Base Article Q211632 accurately describes temporary files, " A temporary file is a file that is created to temporarily store information in order to free memory for other purposes, or to act as a safety net to prevent data loss when a program performs certain functions." These temporary files remain open as long as the application needs them. When the application is shut down, these files are deleted, but the data they contained still remains on the hard drive. How many temporary files are created by an application? This depends on the application but Microsoft states that both Word 97 and Word 2000 create 15 temporary files during use. An important concept to remember about temporary files is that the data they contain remains on the hard drive (until it is overwritten), even if the original file (document, spreadsheet, etc.) is not saved to the drive.
The only way to know whether a particular application program makes "temporary" copies of your data for itself is to use Windows Explorer to inspect the contents of whatever directory your autoexec.bat file specifies in its SET TEMP= line, while the program is open and after using it for a while. Notepad will allow you to inspect such TEMP Space files to see if they contain text from the document on which the program is working.
Further compounding the potential for scattered copies of leaked documents is the fact that some programs define their own "temp" subdirectories for their working copies. In addition, Win9X provides the ability for custom environment settings for application programs. If these "temp" directories are on different logical drives than the TEMP directory, you must also delete and overwrite a disk's free space on those drives after such programs have been used to work on un-encrypted copies of your documents, or your sensitive information may be vulnerable to forensic analysis.
A method to avoid wiping the whole hard disk is to place the TEMP folder in a dedicated partition, or on a ramdisk, which isolates information leakage. Other application specific "temp" folders on other logical drives can be entered into the Eraser Task list.
Add the following folder to the Eraser task list:
and add this line to your Autoexec.bat file
Find Using: Windows Explorer in Windows file extension .tmp
Under Windows 95 and 98 the registry is stored in the C:\WINDOWS directory as system.dat, and user.dat. Under Windows 2000, NT and XP the registry is split into separate component security files which allows for load on demand these are stored in the C:\WINDOWS\SYSTEM32\config directory. It is in the registry that each applications, Recent File History, Undo/Redo History data and personal settings are stored. It is important to be aware that some of these settings are the same as some which you may have deleted from the windows folder e.g.C:\Windows\Recent\. The general principle is to find these settings and history strings and remove them. This can be done by hand or automated using separate .reg files or it can be done at shutdown using a program such as Window Washer. Window Washer has many application plugins for download that will save typing. Similar to File Slack the registry has a slack area too, so this will require defragging a program that will do this on WinNT , Win2K and WinXP is Resplendent Registrar on Win9X machines RegCompact will achieve a similar result.
Securely Deleting Registry Entries:
Windows keeps five backup copies of the registry. The backup registries are kept in cab archive files in the C:\WINDOWS\SYSBCKUP directory, and are named rb[number].cab. The [number] is a number that identifies the order in which these files were created. You can make these copies "roll over" very quickly, leaving nothing but sanitized versions, by making new backups after every cleanup. After you are done with regedit, you can type in "scanregw" at the "Run" prompt. When prompted, make a backup of your cleaned up registry. The ensures the offending file names will not be saved in the registries, just the cleaned up one. Don't forget that the Run text box has an MRU registry key of its own, and you can click on the little arrow to select from a list of recently typed in commands. If you are sure your registry is good and your system is stable, you can securely delete the older "contaminated" backups.
NT's registry is implemented as a database. It shares many of the characteristics including fragmentation as items are added, removed, or changed. In particular, when items are removed from the registry, there is no cleanup mechanism to recover the now unused space in the database. Microsoft released RegClean to clean up some of the debris. A alternative to Microsoft's RegClean is the freeware RegCleaner
Fragmentation can become significant over time on a busy system. Use either tool with caution. There is no tool to compress the registry, to recover lost space and reindex the fundamental database. But there is a method to achieve this result. Backup the registry and be prepared to get back to where you started if things go wrong.
Update the "%systemroot\repair" directory using the command:
"rdisk" will update repair data including the default, software and system hives. The repair version of the hives is compressed and reorganized. It is not an image copy. Adding the "/s" parameter gets the sam and security hives. The dash ("-") instructs rdisk to not make a floppy disk copy.
Expand the registry hives to a temporary location:
expand %systemroot%\repair\default._ C:\temp\reg\default
A comparison of the files sizes in "%temp%" and "%systemroot%\system32\config" can reveal significant differences in size. If the sizes are close, the hive in question did not have much lost space to recover. Usually the software hive has the greatest space recovery.
Replace the version in %systemroot%\system32\config with the newly reindexed version in %temp%. NT keeps the hives open and locked.
You can not simply copy the new version over the old versions. To get around this problem, you need to make the copy when NT is not loaded. For a FAT-based installation, bootup using DOS and make the copy using the DOS COPY command. For NTFS-based installations, I recommend using a recovery or backdoor copy of NT. Install NT on the PC in a different directory. Boot under the secondary NT and copy the hives for the inactive primary NT installation.
There is some risk in this procedure. But as the hives get large with a high amount of dead space, performance suffers. If your server or workstations gets slower and slower for no apparent reason, you may get the performance back using this approach.
Do not attempt if you are not in a position to recover back to the starting point.
An alternative is the freeware RegCleaner.
Another alternative is the donationware RegCompact.
The method described above for Windows NT is not possible as Microsoft removed "rdisk" on the basis that the Windows 2000 environment is too large to fit on a floppy disk
However, if you try to create an Emergency Recovery Disk (ERD) by going to "Start|Programs|Accessories|System Tools|Backup|Emergency Repair Disk", the registry hives hould be copied to the "%system32%\repair\Regback". From this point it may be possible to continue, as per the Windows NT instructions above. Also Resplendent Registrar works fine with Windows 2000.
Find Using: Windows Explorer
View With: Resplendent Registrar, RegEdit.exe, RegEdt32.exe.
Scramdisk will let you create a container file, protected by a password, on an existing hard drive. The software then mounts the container, labeling it with a drive letter. This virtual encrypted drive can then be accessed only with the correct password.
A partition is a separate part of your hard disk which is set apart, or ring fenced from the rest of your hard disk. Its utility is similar to that of using a container, in that information leakage is constrained to a known area on your hard disk. This makes cleaning and overwriting simpler to do. As in the above article covering RamDisks. Various files and folders can be reassigned and moved to your partition, and a secure deletion program such as Eraser can be used to overwrite its contents at the end of each session. A good program for achieving this is PartitionMagic which can resize, split, merge, delete, undelete, and convert partitions -- all without destroying data -- and is fast to use.