COLLECTED hard drive usage after XP NTFS

"Jeff Richards" wrote in message
...
| Finding the correct hard drive space (total number of sectors) is all that
| matters. In most any large capacity drive the firmware will have remapped
| some reserved sectors over identified bad sectors, and that might tell you
| something about the health of the drive in the same way that the SMART
| reporting tools can reveal similar detail. It's not relevant to your
claim
| that clearing the first few sectors of the drive is not sufficient to
force
| NTFS to give back all the space originally allocated to it.

Really,, the health huh, when some of these are apparently the files that
are created during XPs Restore Point activities.
When some of these are apparently the files created when XP backs itself up.
When some of these are apparently the protected HIVES.
When some of these are apparently from the protected system areas.

|
| Scandisk and Format operate at the file system level, and any bad sector
| identification they do is relevant to the file system only. Other
utilities
| can make more permanent identification of bad or suspect sectors, and in
| some cases these can cause problems. Some procedures for marking suspect
| sectors as bad interferes with the inbuilt sector remapping routines and
| causes the disk to use up all its reserved sectors trying to restore
| corrupted sectors that never were really bad in the first place. This is
| the result of using unsuitable tools, and has nothing to do with XP or
NTFS.
| --
| Jeff Richards
| MS MVP (Windows - Shell/User)

Sure they're not... best check again.
So you really never did any forensic reviews on the bad sectors created
after using DOS tools, right.

Pull those disks out and check them. I'd love to see I'm wrong, and I may
be.
I'm fully prepared to apologize to the world if necessary, are you?

Don't take this wrong. I'm not demeaning you, or acting facetious, I am
truly concerned with this, if this is what is occurring.

| "MEB" meb@not wrote in message
| ...
| Well fine, then you do have disks available, use a quality forensic tool
| on
| them. Now we're getting somewhere.
|
| Merely finding full hard drive space means relatively little, since
there
| are reserved sectors for S.M.A.R.T. activity AND other
| scandisk/fdisk/format like activities. The disk will use these areas
| attempting to replace inaccessible areas of the disk. That does NOT mean
| they are bad, just inaccessible.
|
| --
| MEB
| _______________
|
| "Jeff Richards" wrote in message
| ...
| | Jeff Richards
| | MS MVP (Windows - Shell/User)
| | "MEB" meb@not wrote in message
| | ...
| |
| |
| | "Jeff Richards" wrote in message
| | ...
| | | "MEB" meb@not wrote in message
| | | ...
| | | snip
| | |
| | | And to suggest that no results to your searching proves that
there's
| a
| | | conspiracy to keep the problem quiet is just nonsense.
| | | --
| | | Jeff Richards
| | | MS MVP (Windows - Shell/User)
| |
| | I note you state you have disks laying around which have been wiped
| for
| | re-use.
| | Could you use forensic tools to look for compressed and encrypted
| files
| on
| | those disks?
| | I know that's a lot to ask, your day is probably full, but it would
be
| | instrumental in my research, and this thread.
| | Make sure the tool or tools have access to the full disk and are NOT
| | limited by supposed $ BAD SECTOR $ classifications, such as would be
| | created
| | during fdisking and formatting, and scandisking after a "wipe".
| |
| | --
| | MEB
| | _______________
| |
--
MEB
http://peoplescounsel.orgfree.com/
BLOG http://peoplescounsel.spaces.live.com/ Public Notice or the "real
world"

"Most people, sometime in their lives, stumble across truth.
Most jump up, brush themselves off, and hurry on about their business as if
nothing had happen." Winston Churchill
Or to put it another way:
Morpheus can offer you the two pills;
but only you can choose whether you take the red pill or the blue one.
_______________

On Thu, 28 Sep 2006 08:46:10 +1000, Franc Zabkar
put finger to keyboard and composed:

I've checked two of my PCs. Neither has ever seen NTFS or Win XP, yet
the hard drives in both machines show a usable capacity that is
slightly less than their native capacity. In all cases it is the BIOS
(not Fdisk, sorry) that reserves the last cylinder for its own use,
whatever that may be. In addition, there is the loss of the surplus
sectors that result from the translation to CHS mode. This is because
the BIOS tries to fit the drive's geometry into 1024 cylinders or
less. In so doing, the last cylinder is often only partially filled,
so its surplus sectors are discarded. The next full cylinder is then
reserved.

I've just checked my third PC with an AMI BIOS dated 03/13/03. Unlike
earlier BIOSes (1999, 1996), the BIOS in this machine does not reserve
the last cylinder. Therefore Win98SE is able to see the drive's full
capacity (apart from the surplus sectors).

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

On Wed, 27 Sep 2006 22:41:01 -0400, "MEB" meb@not
put finger to keyboard and composed:

Another long one for the discussion (theory)::

With respect, I'm still convinced that you've created a problem where
there originally was none. I say this because in your first post in
this thread you have failed to notice that your BIOS has reserved the
last cylinder of your HD. Instead you've blamed XP/NTFS for this
reduced capacity. I suspect that you may have then used various
forensic utilities in a fruitless attempt to recover this space,
resulting in additional loss.

As to why you have not been able to get Zap or Wipe to restore your
drive to its original pristine condition, I can only speculate.
Perhaps you have attempted to run these utilities inside a Windows DOS
box? Maybe these old IBM (?) utilities have problems with modern
non-IBM drives or BIOSes? For example, some 15 years ago I used
Seagate's Find-ATA utility to retrieve the specs for 40MB IDE drives.
Today the same utility produces a screen with garbled output, even for
a Seagate HD.

On the question of your 10% loss of free space, I can only speculate
that Winhex has done its best to make sense of a file system that has
been damaged by Fdisk. A clue may lie at this URL which you provided
earlier:

http://mirror.href.com/thestarman/asm/mbr/FDISK98.htm

The author writes that the first "Verifying Drive Integrity" process
"writes to and verifies sectors full of F6 bytes on your hard drive;
.... These 'F6 sectors' are written to the 1st and 7th sectors of each
'Head' number (except for Head 0) until a calculated number of sectors
(depending upon the size of the partition) at the beginning of the
drive (or partition) have been 'verified' by FDISK."

Elsewhere ...

http://mirror.href.com/thestarman/asm/mbr/FORMAT98.htm
http://mirror.href.com/thestarman/asm/mbr/MSWIN41.htm

.... the author states that the Format command writes the FSinfo data
to sectors 2 and 8 of the first track. The FSinfo sector keeps track
of the number of "Free Clusters on Disk". The File Access Table (FAT)
tracks the usage of each cluster, ie it knows which clusters are free
and which are in use. So it would be easy for a utility (eg Scandisk)
to check whether the amount of free disc space is being correctly
reported. It could do this by scanning the FAT for free clusters and
comparing what it finds with the total recorded in the FSInfo sector.
In fact Scandisk invariably reports a free space mismatch error on my
HD after a crash. In your case you would expect a mismatch because the
1st and 7th sectors of every track that falls within your FATs would
have been "F6-ed" by Fdisk.

As for why a forensic utility can still see your old file data, I
think it is because a standard Format is essentially non-destructive.
The format process first verifies that every sector can be read, and
then writes the boot sectors, FATs, and root directory. The data area
is not disturbed. A destructive "unconditional" format, ie one that
writes F6 to every sector of the partition, requires the /u switch:

format c: /u

The other tools created to address these activities (wiping, formatting,
fdisking, etc) base their information and coding on known partition types,

I doubt that a wipe utility would care about partition types. Instead
I would think that it would treat the entire disc as raw physical
sectors.

What happens when they attempt to access unknown files, partitions, filing
systems, or file indicators?

Let's pull some information from a related activity.

From the "Eraser 5.8" (http://www.heidi.ie/eraser/ -
http://www.tolvanen.com/sami/) help file (a nice little erasing tool BTW):

I would think that compression and encryption happen at the OS level.
I can't see how they would impact on the hard disc at the sector
level.

Pursuant the above, the "ORIGINAL SYSTEM OR PROGRAM" must/should be used
to access compressed and or encrypted data areas.

I still can't see why zeroing physical sector 0 would not be
sufficient to reclaim this space.

So what is it we have been doing with all these DOS based tools?
http://mirror.href.com/thestarman/asm/mbr/FDISK98.htm for fdisk
http://mirror.href.com/thestarman/asm/mbr/FORMAT98.htm for format
http://mirror.href.com/thestarman/asm/mbr/MSWIN41.htm for boot record/sector
http://mirror.href.com/thestarman/asm/mbr/95BMEMBR.htm for MBR

The author of these pages has done some great detective work. Thanks
for the links.

Now what happens if HPA is involved in protecting certain areas of the
disk, such as some of those compressed areas or files?

If the BIOS can see the whole disc (apart from one reserved cylinder),
then isn't that enough to tell you that its capacity is not being
clipped by some entry in the HPA?

Now what happens when we use tools that "don't know Jack" about what XP
NTFS has done to the disk?

Have you examined the HPA? Have you found any evidence that anything
has been done? If so, what?

Perhaps now you can grasp why I'd like that information on XP NTFS.

|
| I'd be keen to see the CHS numbers in a BIOS HD Autodetect screen, as
| well as the CHS numbers in an MBRtool report. If you can post the
| actual raw contents of the partition table, then that may be
| interesting, too.

Most of that will not be worth anything.

Why not? At the very least it will tell you what capacity the BIOS
sees. That's where the boot process begins. The next most important
thing is the partition table. If the latter has less sectors than
reported by BIOS, then that may tell you something. At the very least
it gives us a point of reference.

If you have used these tools, you know everything (except the BIOS HD
SCREEN) you just asked for has already been modified extensively, many times
over. [The web pages will show some of the activity when they are created.]
Meandisk, for instance, uses the DEBUG MBR reset for the disk in addition
to it's complete zero wipe (which doesn't work).

Are you saying that physical sector 0 cannot be erased? Are you trying
to do this within a Windows DOS box?

The only thing of any real value would be the $Mft tables (that are still
intact) and one of the backup boot sectors. However, the backup boot and
partition table have also been changed.
Or have they?
Running TESTDISK !Search (extended search - searches the entire disk) pulls
up the NTFS partition information. However, placing that info as the
partition, causes the disk to extend far beyond it's stated CHS capacity. So
perhaps LBA is used in non-standard areas of the disk.

Does TESTDISK's NTFS partition information correctly reflect the
native capacity of the HD as specified by its manufacturer? Have you
accounted for a reserved cylinder and/or surplus sectors?

The first few times a backup partition table was used or viewed, it was
similar to the CHS of the disk, except for one cylinder less. That was wiped
by one of the tools, yet this other one still pops up.

Using a disk editor shows a large amount of information stored beyond the
CHS end boundary of the disk, including what appears to be a partition table
and a boot sector.grin

Are you seeing a wrap-around effect? If you make changes to the MBR
code or to the partition table, do these changes also appear at the
end of the disc? For example, change one of the text strings from
"system" to "System" and see what happens.

- Franc Zabkar
--
Please remove one 'i' from my address when replying by email.

INLINE Caution Hazardous material grin

"Franc Zabkar" wrote in message
...
| On Wed, 27 Sep 2006 22:41:01 -0400, "MEB" meb@not
| put finger to keyboard and composed:
|
| Another long one for the discussion (theory)::
|
| With respect, I'm still convinced that you've created a problem where
| there originally was none. I say this because in your first post in
| this thread you have failed to notice that your BIOS has reserved the
| last cylinder of your HD. Instead you've blamed XP/NTFS for this
| reduced capacity. I suspect that you may have then used various
| forensic utilities in a fruitless attempt to recover this space,
| resulting in additional loss.

Sorry Franc, but I have reset the disk to the full capacity many times.
The rest of your statement would be based upon your own false assumptions.

|
| As to why you have not been able to get Zap or Wipe to restore your
| drive to its original pristine condition, I can only speculate.
| Perhaps you have attempted to run these utilities inside a Windows DOS
| box?

You must be nuts or think I'm extremely incompetent.
Which one is it?

Maybe these old IBM (?) utilities have problems with modern
| non-IBM drives or BIOSes? For example, some 15 years ago I used
| Seagate's Find-ATA utility to retrieve the specs for 40MB IDE drives.
| Today the same utility produces a screen with garbled output, even for
| a Seagate HD.
|

Know what you mean there, I used to love Pro View (by McAfee) for disk and
file activities, but tools come and go, you just have to keep pace the best
you can, I guess.

Which old IBM utilities are you talking about, wipe and zap?
I already said they don't work.

| On the question of your 10% loss of free space, I can only speculate
| that Winhex has done its best to make sense of a file system that has
| been damaged by Fdisk. A clue may lie at this URL which you provided
| earlier:
|
| http://mirror.href.com/thestarman/asm/mbr/FDISK98.htm
|
| The author writes that the first "Verifying Drive Integrity" process
| "writes to and verifies sectors full of F6 bytes on your hard drive;
| ... These 'F6 sectors' are written to the 1st and 7th sectors of each
| 'Head' number (except for Head 0) until a calculated number of sectors
| (depending upon the size of the partition) at the beginning of the
| drive (or partition) have been 'verified' by FDISK."
|
| Elsewhere ...
|
| http://mirror.href.com/thestarman/asm/mbr/FORMAT98.htm
| http://mirror.href.com/thestarman/asm/mbr/MSWIN41.htm
|
| ... the author states that the Format command writes the FSinfo data
| to sectors 2 and 8 of the first track. The FSinfo sector keeps track
| of the number of "Free Clusters on Disk". The File Access Table (FAT)
| tracks the usage of each cluster, ie it knows which clusters are free
| and which are in use. So it would be easy for a utility (eg Scandisk)
| to check whether the amount of free disc space is being correctly
| reported.

Actually, if you've been paying attention to another thread Everest
Report presntly going on, we just had someone find out how badly a disk can
be screwed up. Let's see what we come up with there, if I can only get PCR
to lay off for a bit without offending him.
As for my disk, since I monitor the activities and occurrences, NO fdisk
has done no more damage than other programs. It's the final resetting of all
this that's going to be the real pain.

It could do this by scanning the FAT for free clusters and
| comparing what it finds with the total recorded in the FSInfo sector.
| In fact Scandisk invariably reports a free space mismatch error on my
| HD after a crash. In your case you would expect a mismatch because the
| 1st and 7th sectors of every track that falls within your FATs would
| have been "F6-ed" by Fdisk.

Good theory, however it's faulty. You haven't checked the other tools
previously referenced, which I had to place into that discussion you thought
to put into that alt.hard drive group did you? Read on before you get upset
or reply.

|
| As for why a forensic utility can still see your old file data, I
| think it is because a standard Format is essentially non-destructive.
| The format process first verifies that every sector can be read, and
| then writes the boot sectors, FATs, and root directory. The data area
| is not disturbed. A destructive "unconditional" format, ie one that
| writes F6 to every sector of the partition, requires the /u switch:
|
| format c: /u

Yeah that's right.
Like I haven't done before. This is a test of DOS and other techniques and
tools.

|
| The other tools created to address these activities (wiping, formatting,
| fdisking, etc) base their information and coding on known partition
types,
|
| I doubt that a wipe utility would care about partition types. Instead
| I would think that it would treat the entire disc as raw physical
| sectors.

They try, but only if they recognize them or can access them.
It was actually interesting to watch through the first few tests, now
it's... well have you ever done the same task dozens of times with the same
outcome?

|
| What happens when they attempt to access unknown files, partitions,
filing
| systems, or file indicators?
|
| Let's pull some information from a related activity.
|
| From the "Eraser 5.8" (http://www.heidi.ie/eraser/ -
| http://www.tolvanen.com/sami/) help file (a nice little erasing tool
BTW):
|
| I would think that compression and encryption happen at the OS level.
| I can't see how they would impact on the hard disc at the sector
| level.

So your ignoring the extended boot and partition table perhaps referenced
by the HPA.
And the "hex values" and changed disk instructions then (HINT).

|
| Pursuant the above, the "ORIGINAL SYSTEM OR PROGRAM" must/should be
used
| to access compressed and or encrypted data areas.
|
| I still can't see why zeroing physical sector 0 would not be
| sufficient to reclaim this space.

Because the protected sectors can't be accessed, perhaps?

|
| So what is it we have been doing with all these DOS based tools?
| http://mirror.href.com/thestarman/asm/mbr/FDISK98.htm for fdisk
| http://mirror.href.com/thestarman/asm/mbr/FORMAT98.htm for format
| http://mirror.href.com/thestarman/asm/mbr/MSWIN41.htm for boot
record/sector
| http://mirror.href.com/thestarman/asm/mbr/95BMEMBR.htm for MBR
|
| The author of these pages has done some great detective work. Thanks
| for the links.

YW. Don't count on more than they present and make sure you understand
EXACTLY what they present.
Then perhaps look around a bit for more info, like how hard drives are made
for instances.

|
| Now what happens if HPA is involved in protecting certain areas of the
| disk, such as some of those compressed areas or files?
|
| If the BIOS can see the whole disc (apart from one reserved cylinder),
| then isn't that enough to tell you that its capacity is not being
| clipped by some entry in the HPA?

No, your forgetting LBA and several other things related to hard drives.

|
| Now what happens when we use tools that "don't know Jack" about what XP
| NTFS has done to the disk?
|
| Have you examined the HPA? Have you found any evidence that anything
| has been done? If so, what?

Franc, do you have access to any re-formatted XP NTFS disk. I don't care if
you have installed the disk full of programs and an OS. Just use some of
these tools (be careful which ones) and LOOK AT THE DISK FOR NTFS traces.
Or beg, borrow, or steal (it's a saying) a copy of WINHEX and look at the
disk. Version 12.8 is the last one which works in 98 (if that's what your
running), it's 13+ for XP last time I checked. See what it locates for you.
(IT IS EXPENSIVE BTW) You could use the test version I think (is it still
available?).

|
| Perhaps now you can grasp why I'd like that information on XP NTFS.
|
| |
| | I'd be keen to see the CHS numbers in a BIOS HD Autodetect screen, as
| | well as the CHS numbers in an MBRtool report. If you can post the
| | actual raw contents of the partition table, then that may be
| | interesting, too.
|
| Most of that will not be worth anything.
|
| Why not? At the very least it will tell you what capacity the BIOS
| sees. That's where the boot process begins. The next most important
| thing is the partition table. If the latter has less sectors than
| reported by BIOS, then that may tell you something. At the very least
| it gives us a point of reference.

Which one do you want: the present: the one after Meandisk: the one
after.....
Your missing the point, it won't tell you anything that your looking for.
Because after fdisking and formatting the disk is reduced from it's stated
full capacity, which it finds. Scandisk it, and it will be reduced even
more. Hence, it's now around 5.6 gig, which is as far as I'm willing to take
it. It will be difficult enough to recover.

|
| If you have used these tools, you know everything (except the BIOS HD
| SCREEN) you just asked for has already been modified extensively, many
times
| over. [The web pages will show some of the activity when they are
created.]
| Meandisk, for instance, uses the DEBUG MBR reset for the disk in
addition
| to it's complete zero wipe (which doesn't work).
|
| Are you saying that physical sector 0 cannot be erased? Are you trying
| to do this within a Windows DOS box?

Are you in love with Windows or something? DOS box, what idiot would use a
DOS box for this type of activity.

Yes it can, and was, MEANDISK rewrote it.
Other programs have changed it. I've changed it following the old "tried and
true" advise. This is a test remember?
IT MAKES NO DIFFERENCE.

|
| The only thing of any real value would be the $Mft tables (that are
still
| intact) and one of the backup boot sectors. However, the backup boot and
| partition table have also been changed.
| Or have they?
| Running TESTDISK !Search (extended search - searches the entire disk)
pulls
| up the NTFS partition information. However, placing that info as the
| partition, causes the disk to extend far beyond it's stated CHS capacity.
So
| perhaps LBA is used in non-standard areas of the disk.
|
| Does TESTDISK's NTFS partition information correctly reflect the
| native capacity of the HD as specified by its manufacturer? Have you
| accounted for a reserved cylinder and/or surplus sectors?

Yes

|
| The first few times a backup partition table was used or viewed, it was
| similar to the CHS of the disk, except for one cylinder less. That was
wiped
| by one of the tools, yet this other one still pops up.

LET ME MODIFY THIS PART OF MY PRIOR POST
There apparently is still a Table around the center of the disk as well.

|
| Using a disk editor shows a large amount of information stored beyond
the
| CHS end boundary of the disk, including what appears to be a partition
table
| and a boot sector.grin
|
| Are you seeing a wrap-around effect? If you make changes to the MBR
| code or to the partition table, do these changes also appear at the
| end of the disc? For example, change one of the text strings from
| "system" to "System" and see what happens.
|
| - Franc Zabkar
| --
| Please remove one 'i' from my address when replying by email.

Franc, if I choose to do so, I can edit the entire friggin disk by hand or
by block replacement. Heck I can make the disk a full 7 gig and beyond, if
that's what I want to look at.
I have both DOS and Windows disk editing and FORENSIC tools, capable of
just about anything, INCLUDING rewriting jump and other instructions on the
disk.

Get it yet?
What are we doing here, fixing the disk or EXPOSING Microsoft's apparent
corruption of hard disks?

I did not ask for help fixing this disk, I asked for the XP NTFS info
(which I need to recover this disk or it's trial and error) and started a
general discussion of related material, and theory, hoping to spark thought
processes.
To be relevant, one would have to have tested at least one disk with a
competent disk tool for some comparison of results.
Using the "tried and true" info, is using the same OLD information which is
mostly irrelevant, it's apparently based upon DOS and the old NT kernel
NTFS.

Now, really want to help? Then test a disk or two so we can compare
results. Heck, I may be wrong.

--
MEB
http://peoplescounsel.orgfree.com/
BLOG http://peoplescounsel.spaces.live.com/ Public Notice or the "real
world"

"Most people, sometime in their lives, stumble across truth.
Most jump up, brush themselves off, and hurry on about their business as if
nothing had happen." Winston Churchill
Or to put it another way:
Morphs can offer you the two pills;
but only you can choose whether you take the red pill or the blue one.
_______________

"Franc Zabkar" wrote in message
...
| On Thu, 28 Sep 2006 08:46:10 +1000, Franc Zabkar
| put finger to keyboard and composed:
|
| I've checked two of my PCs. Neither has ever seen NTFS or Win XP, yet
| the hard drives in both machines show a usable capacity that is
| slightly less than their native capacity. In all cases it is the BIOS
| (not Fdisk, sorry) that reserves the last cylinder for its own use,
| whatever that may be. In addition, there is the loss of the surplus
| sectors that result from the translation to CHS mode. This is because
| the BIOS tries to fit the drive's geometry into 1024 cylinders or
| less. In so doing, the last cylinder is often only partially filled,
| so its surplus sectors are discarded. The next full cylinder is then
| reserved.
|
| I've just checked my third PC with an AMI BIOS dated 03/13/03. Unlike
| earlier BIOSes (1999, 1996), the BIOS in this machine does not reserve
| the last cylinder. Therefore Win98SE is able to see the drive's full
| capacity (apart from the surplus sectors).
|
| - Franc Zabkar
| --
| Please remove one 'i' from my address when replying by email.

Gotcha, thanks for the info..
Both of mine have Award BIOSs. It's important to know, but I'm looking at
the actual physical disk sectors.

BTW, don't be offended by the other post, because ultimately, if any one
bothers to actually test a disk for the discussion, we are going to have to
compare disk instruction sets as well as sector analysis.

Here's some more info you may not have:

AEFDISK INFO EXCERPT

Partition Information

ID Name
ÄÄ ÄÄÄÄ
00h empty
01h DOS 12-bit FAT
02h XENIX root file system
03h XENIX /usr file system (obsolete)
04h DOS 16-bit FAT (up to 32M)
05h DOS 3.3+ extended partition
06h DOS 3.31+ Large File System (16-bit FAT, over 32M)
07h QNX
07h OS/2 HPFS
07h Windows NT NTFS
07h Advanced Unix
08h OS/2 (v1.0-1.3 only)
08h AIX bootable partition, SplitDrive
08h SplitDrive
08h Commodore DOS
08h DELL partition spanning multiple drives
08h QNX 1.x and 2.x
09h AIX data partition
09h Coherent filesystem
09h QNX 1.x and 2.x
0Ah OS/2 Boot Manager
0Ah OPUS
0Ah Coherent swap partition
0Bh Windows 95 OSR2 with 32-bit FAT
0Ch Windows 95 OSR2 with 32-bit FAT (LBA-mode INT 13 extensions)
0Eh LBA VFAT (same as 06h but using LBA-mode INT 13)
0Fh LBA VFAT (same as 05h but using LBA-mode INT 13)
10h OPUS
11h Hidden 12-bit FAT partition, OS/2 FAT12
12h Compaq/HP Diagnostics partition (FAT compatible)
14h (using Novell DOS 7.0 FDISK to delete Linux Native part)
14h Hidden sub-32M 16-bit FAT partition
16h Hidden over-32M 16-bit FAT partition
17h Hidden HPFS partition
18h AST special Windows swap file
19h Willowtech partition
1Bh Hidden Windows 95 with 32-bit FAT
1Ch Hidden Windows 95 with 32-bit LBA FAT
1Eh Hidden Windows 95 with LBA BIGDOS
20h OFS1
21h officially listed as reserved, FSo2
23h officially listed as reserved
24h NEC DOS 3.x
26h officially listed as reserved
31h officially listed as reserved
33h officially listed as reserved
34h officially listed as reserved
36h officially listed as reserved
38h Theos v3.2 2GB partition
39h Theos v4 spanned partition
3Ah Theos v4 4GB partition
3Bh Theos v4 extended partition
3Ch PowerQuest PartitionMagic recovery partition
40h VENIX 80286
41h Personal RISC Boot
41h Power PC Reference Platform Boot
42h SFS (Secure File System) by Peter Gutmann
45h EUMEL/Elan
46h EUMEL/Elan
47h EUMEL/Elan
48h EUMEL/Elan
4Dh QNX4.x
4Eh QNX4.x 2nd part
4Fh QNX4.x 3rd part
4Fh Oberon
50h OnTrack Disk Manager, read-only partition
51h OnTrack Disk Manager, read/write partition
51h NOVELL
52h CP/M
52h Microport System V/386
53h OnTrack Disk Manager, write-only partition???
54h OnTrack Disk Manager (DDO)
55h EZ-Drive
56h GoldenBow VFeature
56h DM converted to EZ-BIOS
57h DrivePro
5Ch Priam EDisk
61h SpeedStor
63h Unix SysV/386, 386/ix
63h Mach, MtXinu BSD 4.3 on Mach
63h GNU HURD
64h PC-ARMOUR protected partition
64h Novell NetWare 2.xx
65h Novell NetWare 3.xx or 4.xx
67h Novell
68h Novell
69h Novell
70h DiskSecure Multi-Boot
71h officially listed as reserved
73h officially listed as reserved
74h officially listed as reserved
75h IBM PC/IX
76h officially listed as reserved
7Eh F.I.X
80h Minix v1.1 - 1.4a
81h Minix v1.4b+
81h Linux
81h Mitac Advanced Disk Manager
82h Linux Swap partition
82h Prime
82h Solaris x86
83h Linux native file system (ext2fs/xiafs)
84h OS/2-renumbered type 04h partition (hiding DOS C: drive)
84h Hibernation partition
85h Linux extended partition
86h NTFS volume set
87h HPFS Fault-Tolerant mirrored partition
8Ah Linux Kernel Partition (used by AiR-BOOT)
8Eh Linux Logical Volume Manager partition
92h Amoeba
93h Amoeba file system
94h Amoeba bad block table
99h DCE376 logical drive
A0h IBM Thinkpad hibernation partition / PQMagic
A0h Phoenix NoteBIOS Power Management "Save-to-Disk" partition
A1h officially listed as reserved
A3h officially listed as reserved
A4h officially listed as reserved
A5h FreeBSD, NetBSD, BSD/386
A6h OpenBSD
A7h NEXTSTEP
A9h NetBSD
AAh Olivetti Fat 12 1.44Mb Service Partition
B1h officially listed as reserved
B3h officially listed as reserved
B4h officially listed as reserved
B6h officially listed as reserved
B7h BSDI file system (secondarily swap)
B8h BSDI swap partition (secondarily file system)
BEh Solaris boot partition
C0h CTOS
C0h REAL/32 secure small partition / DR-DOS secondary
C1h DR DOS 6.0 LOGIN.EXE-secured 12-bit FAT partition
C4h DR DOS 6.0 LOGIN.EXE-secured 16-bit FAT partition
C6h DR DOS 6.0 LOGIN.EXE-secured Huge partition
C7h Syrinx Boot
CAh FAT-32 (?)
CBh reserved for DRDOS/secured (FAT32)
CCh reserved for DRDOS/secured (FAT32, LBA)
CEh reserved for DRDOS/secured (FAT16, LBA)
D0h REAL/32 secure big partition
D1h Old Multiuser DOS secured FAT12
D4h Old Multiuser DOS secured FAT16 32M
D5h Old Multiuser DOS secured extended partition
D6h Old Multiuser DOS secured FAT16 =32M
D8h CP/M-86
DBh CP/M, Concurrent CP/M, Concurrent DOS
DBh CTOS (Convergent Technologies OS)
DEh Dell diagnostic
DFh RadiSys DTS
E1h SpeedStor 12-bit FAT extended partition
E3h DOS read-only
E3h Storage Dimensions
E4h SpeedStor 16-bit FAT extended partition
E5h officially listed as reserved
E6h officially listed as reserved
EBh BeOS partition
F1h Storage Dimensions
F2h DOS 3.3+ secondary partition
F3h officially listed as reserved
F4h SpeedStor
F4h Storage Dimensions
F5h Prologue multi-volume partition
F6h officially listed as reserved
FDh Linux raid partition
FEh LANstep
FEh IBM PS/2 IML
FFh Xenix bad block table

The main characteristics of the NT file system are described by the boot
record. The file system starts at the location of the boot sector and ends
at this sector plus the sectors in the volume field. The volume is organized
in clusters of multiples of 512 bytes. The start of the volume is associated
with cluster number 0.

Any addressing within the file system is done using the cluster number
instead of a sector number. Any kind of file information is contained in a
structure called Master file table (MFT). The start of the MFT is indicated
by the boot sector's field 1st MFT cluster. The MFT is a database containing
information about every file or directory on the volume. The default size of
an entry in the MFT is 1024 bytes. Each entry describes a file or directory
on the volume (including the MFT itself) and has a record number that equals
the byte position inside the MFT divided by 1024. Each MFT entry consists of
a header and a list of attributes. The attributes describe file names, time
stamps, file sizes, data allocations and more. In the file entry details
view you can inspect any attribute of any MFT entry. Most important
attributes a

$10 Standard information: contains time stamps and DOS attributes,
$30 File name: contains the file's name for different name spaces (usually
NT's native Unicode file name and DOS compatible DOS file name),
$80 Data: if the entry represents a file, this attribute contains the file's
data.
$90 Index root: if the entry is a directory, this attribute describes the
root of a binary tree in which the directory entries are located,
$A0 Index allocation: if the entry is a directory, this attribute contains a
list of file names.
If the attributes are too large to fit into 1024 bytes, some of them will be
non-resident, meaning the value part the specific attribute will be found
outside of the entry. In this case the outside data allocation will be
described by a run list. A run list is a list of clusters used. When in the
file entry details view you can inspect a non-resident attribute's data by
following a link.

Really,, the health huh, when some of these are apparently the files that
are created during XPs Restore Point activities.
When some of these are apparently the files created when XP backs itself
up.
When some of these are apparently the protected HIVES.
When some of these are apparently from the protected system areas.

Some of WHAT? The only missing space that yo have described properly has
been thoroughly explained by Franc.

| Scandisk and Format operate at the file system level, and any bad sector
| identification they do is relevant to the file system only. Other
utilities
| can make more permanent identification of bad or suspect sectors, and in
| some cases these can cause problems. Some procedures for marking suspect
| sectors as bad interferes with the inbuilt sector remapping routines and
| causes the disk to use up all its reserved sectors trying to restore
| corrupted sectors that never were really bad in the first place. This
is
| the result of using unsuitable tools, and has nothing to do with XP or
NTFS.
| --
| Jeff Richards
| MS MVP (Windows - Shell/User)

Sure they're not... best check again
..
Not WHAT? Not really bad? That's exactly the point I made - there are some
tools that will declare sectors bad when they aren't. If this happens at
the file system level then the flagging goes away when the file system is
removed, and the space is recovered. But if the tool is 'smart' and tries
to make them permanently bad, then (a) it might succeed, and the space is
lost forever or (b) the drive firmware kicks in and tries to remap a
reserved sector, and things can go downhill from there. That's why I
commented that it is possible for these types of uutilities to 'damage' a
disk. ZAP and Wipe are not that 'smart', but in any case the issue has
nothing to do with NTFS. It's a problem of using the wrong tool..

I have seen a similar thing happen when the drive was badly configured and
two utilities got extremely confused about how many sectors, tracks and
heads there were (sectors was the critical one, IIRC). This can also happen
if overlay software is used. That's a configuration problem, and again
nothing to do with NTFS.

So you really never did any forensic reviews on the bad sectors created
after using DOS tools, right.

I have done lots of examination of bad sectors. In many cases I have
recovered data from bad sectors. I would assume that quite often the data
was some sort of XP 'special usage' area, such as system restore. But I
have never known the allocation of disk space to bad sectors to be
associated with how XP or NTFS was using that particular sector.

Pull those disks out and check them. I'd love to see I'm wrong, and I may
be.
I'm fully prepared to apologize to the world if necessary, are you?

Check them for what? That the disk capacity visible to the file system
agrees with the physical size of the disk? That's already carefully checked
in making sure the disks are fit for use.

Don't take this wrong. I'm not demeaning you, or acting facetious, I am
truly concerned with this, if this is what is occurring.

If you are quoting this because you think this describes where the problem
is coming from, then I think I can see what the problem is
snip
Note this: " If you erased cluster tips on a compressed drive and lost
significant amount of disk space, you must recompress the drive to restore
the lost space."
Erasing a a compressed drive or volume with cluster tips turned on increases
the space consumed by the compressed files, becasue one of the features of
compression is that the cluster tips are never physically stored. When they
are erased, the compression software thinks the erasing data is genuine data
and compresses it and writes it, thus vastly increasing the space required.
Recompressing the volume or partition re-writes each file without the 'tip',
putting things back to how they were.

Inappropriate use of the erasing utility will increase the size of the
compressed volume and reduce free disk space, but it is all fuly
recoverable.

snip

Let's say not the whole disk is compressed (though it basically is in
NTFS), but only certain areas of the disk. What happens when we attempt to
remove them normally? What happens if more than one compression and or
encryption scheme is used within the disk, e.g. compression within
compression or encryption within compression?
If you remove a compressed file the file entry is removed from the
compressed volume. The file contents remain, and (usually) the compressed
volume file size doesn't change. You can recompress the volume or
partition to remove unused space.

Saving a compressed file to a compressed partition (eg, zipping a file to a
compressed volume) doesn't affect anything, except that you won't save any
space other than the 'tip'.

Whether a file is encrypted or not is irrelevant, except that encruption
might change the file size and will certainly slow down file operations. It
will also make the whole file look different than the original, and this
might interfere with intelligent replacement algorithms in the compression
routine, again increasing disk access times. But it has nothing to do with
lost disk space.

Now what happens if HPA is involved in protecting certain areas of the
disk, such as some of those compressed areas or files?
It isn't.

Now what happens when we use tools that "don't know Jack" about what XP
NTFS has done to the disk?
Then you will stuff up the file system (or just get wrong information).

snip
The only thing of any real value would be the $Mft tables (that are still
intact) and one of the backup boot sectors.
The $MFT tables are just sectors of data for a recovery program. The fact
that they still exist does not indicate that the data or structures they
refer to still exist.

However, the backup boot and partition table have also been changed.
Or have they?
Maybe, maybe not. Depending on the utility you are using and exactly what
you are looking at, you might be seeing an original, you might be looking at
a backup (which might be out of date, or might refer to a completely
different disk organisation, such as an overlay) or you might be seeing a
reconstruction. The history of the disk is critical if you want to make use
of this type of data.

Running TESTDISK !Search (extended search - searches the entire disk)
pulls
up the NTFS partition information.
Partition information can be either a reconstruction or a recoverd sector.
The recovered sector might be a backup, and it could be a backup from any
previous state of the drive. It depends on what the prior process was, the
utility you are using and which partition data you are looking at.

However, placing that info as the
partition, causes the disk to extend far beyond it's stated CHS capacity.
So it's probably a reconstruction, and incorrect.

So
perhaps LBA is used in non-standard areas of the disk.
The first few times a backup partition table was used or viewed, it was
similar to the CHS of the disk, except for one cylinder less. That was
wiped
by one of the tools, yet this other one still pops up.
Using multiple tools is going to do this. They have different numbering
systems, use the same terms in slightly different ways, and calculate things
slightly differently. You have to allow for these differences if you use
more than one tool.

Using a disk editor shows a large amount of information stored beyond the
CHS end boundary of the disk, including what appears to be a partition
table
and a boot sector.grin
The 'large amount of information" is probably unused sectors between the
last logical sector and the last physial sector (which occurs due to
rounding to a full track). The other might be a backup (you need BIOS
details and a full history of the disk to answer that, such as special
paritioning software that may have been used). But it might also be the
disk editor is simply wrapping at the end of the disk - a commn 'feature'.

--
Jeff Richards
MS MVP (Windows - Shell/User)

"Jeff Richards" wrote in message
...
| If you are quoting this because you think this describes where the problem
| is coming from, then I think I can see what the problem is
[all the stuff cut]
| --
| Jeff Richards
| MS MVP (Windows - Shell/User)
|
|

Okay, I see you still don't grasp the situation, let's look at it like
this:

If this was the same old NT4 NTFS file system, then who would buy the new
friggin OS.
Almost everything included in XP was available in the NT4 version via third
party programs. Even to the extent of hardening the system, and remote
administration.
The major SELLING point and supposed major REASON for businesses to upgrade
was because of the NEW protection schemes and hardened filing activities
integrated into this NEWEST TECHNOLOGY NTFS.

Now, I appreciate the continued attempts to deal with the disks as if it's
the same old DOS based functioning as MS DOS through NT4, but to do so is
stating and indicating that Microsoft committed fraud and then could be sued
for doing so.... I don't think Microsoft would appreciate that much.
My research shows they did exactly as they said they did or would.
The failure, and potential for suit would be the blatant disregard to
create a tool to completely remove all traces of the OS and return the disk
to it's pristine state, and forewarn of the potential risks involved with
disk reuse or attempted wiping for security reasons.

Moreover, since you did check at least one disk, and FOUND evidence of XPs
protected area traces, one would think you might have done this research
yourself several years ago.

Per one of your other presentations, you stated you were about to send a
disk back to the manufacturer with LARGE amounts of potentially sensitive
data STILL INTACT on the disk. When I performed these types of activities, I
had to CERTIFY that I had securely wiped these disk of any potentially
sensitive data, and that meant EVERY trace.

Might want to think about that.

--
MEB
http://peoplescounsel.orgfree.com/
BLOG http://peoplescounsel.spaces.live.com/ Public Notice or the "real
world"

"Most people, sometime in their lives, stumble across truth.
Most jump up, brush themselves off, and hurry on about their business as if
nothing had happen." Winston Churchill
Or to put it another way:
Morpheus can offer you the two pills;
but only you can choose whether you take the red pill or the blue one.
_______________

"Jeff Richards" wrote in message
...
| Really,, the health huh, when some of these are apparently the files
that
| are created during XPs Restore Point activities.
| When some of these are apparently the files created when XP backs itself
| up.
| When some of these are apparently the protected HIVES.
| When some of these are apparently from the protected system areas.
|
| Some of WHAT? The only missing space that yo have described properly has
| been thoroughly explained by Franc.

No it wasn't.

|
| | Scandisk and Format operate at the file system level, and any bad
sector
| | identification they do is relevant to the file system only. Other
| utilities
| | can make more permanent identification of bad or suspect sectors, and
in
| | some cases these can cause problems. Some procedures for marking
suspect
| | sectors as bad interferes with the inbuilt sector remapping routines
and
| | causes the disk to use up all its reserved sectors trying to restore
| | corrupted sectors that never were really bad in the first place.
This
| is
| | the result of using unsuitable tools, and has nothing to do with XP or
| NTFS.
| | --
| | Jeff Richards
| | MS MVP (Windows - Shell/User)
|
| Sure they're not... best check again
| .
| Not WHAT? Not really bad? That's exactly the point I made - there are
some
| tools that will declare sectors bad when they aren't. If this happens at
| the file system level then the flagging goes away when the file system is
| removed, and the space is recovered. But if the tool is 'smart' and tries
| to make them permanently bad, then (a) it might succeed, and the space is
| lost forever or (b) the drive firmware kicks in and tries to remap a
| reserved sector, and things can go downhill from there. That's why I
| commented that it is possible for these types of uutilities to 'damage' a
| disk. ZAP and Wipe are not that 'smart', but in any case the issue has
| nothing to do with NTFS. It's a problem of using the wrong tool..

Which tool was wrong?
These tools (depending on which one) can completely ignore the BIOS, read
the actual disk, and work from there, were these the wrong tools?
Others can reset the disk from the manufactures own onboard coding and work
from there. So were these the wrong ones?
Then there are those which are highly touted tools designed for only one
purpose, and that's to wipe every sector of the disk, even "bad areas", were
these the wrong tools to use?
And still others can do some commands on disks which could, with improper
use, literally kill the disk, so were these the wrong tools?

Then we have to consider the tools which can read every part of the disk,
every sector, every bit and byte, and the instruction sets, were these the
wrong tools to use on the disk?

So in your honoured opinion which ones were wrong. Need the list again?

|
| I have seen a similar thing happen when the drive was badly configured and
| two utilities got extremely confused about how many sectors, tracks and
| heads there were (sectors was the critical one, IIRC). This can also
happen
| if overlay software is used. That's a configuration problem, and again
| nothing to do with NTFS.

Yep, programs do get confused, change a disk from hard set to AUTO and you
may have a major problem on your hands.
Beyond that, changing CHS effects how the disk is accessed, so the point is?

|
| So you really never did any forensic reviews on the bad sectors created
| after using DOS tools, right.
|
| I have done lots of examination of bad sectors. In many cases I have
| recovered data from bad sectors. I would assume that quite often the data
| was some sort of XP 'special usage' area, such as system restore. But I
| have never known the allocation of disk space to bad sectors to be
| associated with how XP or NTFS was using that particular sector.

So what your saying is that 2+2 didn't equal 4 to you.

|
| Pull those disks out and check them. I'd love to see I'm wrong, and I
may
| be.
| I'm fully prepared to apologize to the world if necessary, are you?
|
| Check them for what? That the disk capacity visible to the file system
| agrees with the physical size of the disk? That's already carefully
checked
| in making sure the disks are fit for use.
|
| Don't take this wrong. I'm not demeaning you, or acting facetious, I am
| truly concerned with this, if this is what is occurring.
|
|
|


So once again, you have some disk available, let's have some fun. Do some
tests.

--
MEB
http://peoplescounsel.orgfree.com/
BLOG http://peoplescounsel.spaces.live.com/ Public Notice or the "real
world"

"Most people, sometime in their lives, stumble across truth.
Most jump up, brush themselves off, and hurry on about their business as if
nothing had happen." Winston Churchill
Or to put it another way:
Morpheus can offer you the two pills;
but only you can choose whether you take the red pill or the blue one.
_______________

So once again, you have some disk available, let's have some fun. Do some
tests.

Set out a test sequence that you believe shows the problem and I will run it
through. Other than that, I'm not interested.

--
Jeff Richards
MS MVP (Windows - Shell/User)