Beep boop - this is a robot. A new show has been posted to TWiT…
What are your thoughts about today’s show? We’d love to hear from you!
The capacity fraud on USB sticks, memory cards and even SSDs has been going on for years, Heise in Germany has been regularly running exposes as far back as I can remember (at least 2005).
Often a 1GB up to 16GB stick or card has its firmware rewritten to report the false size and do an overwrite (I.e. when you get to the end of the storage, it wraps around and starts over, others, like the reported case, just stop writing.
There are several testing utilities that can discover this and it is wise to use one of these.
Last year, they also discovered a company buying up old 2.5” HHDs, scraping off then model and serial numbers and repackaging them as new, higher capacity HHDs for external backups, NAS etc.
1 Like
Here’s the C code for one such tool, that discovers if a drive’s advertised size is fake:
I cobbled together a little command-line utility in Java that fills a disk to maximum capacity with pseudo-randomly named files of pseudo-random data. It then takes a pass to verify all the files can be read back as expected. And finally it cleans up after itself. It seems a little slow in my testing so far, so I need to see what I can do to speed it up, but it’s always going to take some time to fill a drive anyway, and if the most important thing is knowing you can depend on the device, then the time may be worth waiting for.
I have done basic sanity testing of it, but I need some testers and feedback on any problems them find. If you’re interested you can get a copy in the next few days before the file host automatically expires the file.
I did finish an alpha of the tool just tonight. I need to figure out which machine I used the last time I published something here (it was that LastPass breach tool) so I can package it as an executable, but I am sharing a executable JAR now in case anyone wants to test it.
The catbox litterbox only allows for up to 3 days, so get it quick if you want it. [expired link removed]
As ever, there is no warranty of correct function, and I guess, in theory it could somehow wreck your disk… but I can’t imagine how. It determines how much free space it has, and fills it with files. During the cleanup phase it only deletes the files in created. It’s been tested for basic function on my one Windows machine, but that doesn’t guarantee it will work on all machines (but I can’t expect why it won’t work on other platforms so that’s why I need testers.)
Here’s what’s in the included ReadMe.TXT:
What is this?
A tool to help verify that a disk operates like a disk and
actually stores the files you request of it. It’s not yet
well tested, but basic function testing has been completed
on Windows. It’s a Java JAR file… expecting Java 20.
You would invoke it using a command like with three parameters
such as:
java -jar DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar param1 param2 param3
Where param1 is the disk name (for example D: on Windows)
and param2 is one of three modes fill or verify or clean
and param3 is a random (password like value) particular to you
Use the fill mode to fill the disk with test files, then
use the verify mode to make sure they can be read back correctly
and the finally use the clean mode to delete all the created files.
The random value should be the same for all three modes as it seeds
the filename generator as well as the file content generator. It
can be a different value when testing a different disk though.
Below are some example outputs from my testing:
C:\Dev\>java -jar DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar
Usage: param1=drive param2=action(fill|verify|clean) param3=unique(used to seed randomness)
A list of visible drives should follow:
(C:) NTFS
RAMDisk (R:) NTFS RAMDisk
C:\Dev>java -jar DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar r: fill FooBar123
Fill mode active
(C:) NTFS
RAMDisk (R:) NTFS RAMDisk
Executing on r:\
Fill size is 1984737280 bytes
Using one file of size: 1984671744 bytes
Using Disk Fill Path : r:\DF29731
Using Info Log Path : r:\DF29731\IL58531
DF2134053508
Made new directory: r:\DF29731
1984737280
Creating file: r:\DF29731\DF2134053508
*** Fill finished, now invoke verify ***
C:\Dev>java -jar DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar r: verify FooBar123
Verify mode active
(C:) NTFS
RAMDisk (R:) NTFS RAMDisk
Executing on r:\
loggedFillSize: 1984671744
totalNumFiles: 1
firstFileName: DF2134053508
DF2134053508
Verifying the contents of file: r:\DF29731\DF2134053508
*** Verify finished, now invoke clean ***
C:\Dev>java -jar DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar r: clean FooBar123
Clean mode active
(C:) NTFS
RAMDisk (R:) NTFS RAMDisk
Executing on r:\
loggedFillSize: 1984671744
totalNumFiles: 1
firstFileName: DF2134053508
DF2134053508
Deleted: r:\DF29731\DF2134053508
Deleted Info Log: r:\DF29731\IL58531
Removed Disk Fill directory: r:\DF29731
*** Clean finished ***
2 Likes
For Windows, the best tool for checking the capacity, performance and integrity of a USB stick, card or drive is H2Testw from Harald Bögerholz (H2testw - Gratis-Download von heise.de)
For *NIX machines, including Macs, the best tool is F3 (F3 - Gratis-Download | Heise), which is a command line tool.
Here is an example of the reports that Heise/c’t bring out most years:
You’ll probably need to run it through Google Translate.
2 Likes
Thanks all! Looks like Steve is working on one, too!
1 Like
@Leo, just to let you know there is some suspect (low volume/muffled) audio in this episode. Like around 33m.
It does sound like we lost a little high end. I played back the video and audio from the web site. What version were you listening too? Just so I can make sure I’m hearing what you hear.
Same for me. I usually watch/listen live, but I was away for Tuesday’s shows so I am catching up.
I’m just catching up with the public (adverts) audio version. Steve’s voice went a lot quieter and a bit muffled (so both loss of volume and top end) roughly between 30 and 37 mins into the show.
Coming up to the 45 min mark, Leo’s Thinkst Canary advert sounded to me as if his level rose and fell a bit during the advert - like someone varying their distance from the mic, which I know Leo wouldn’t do.
Got to the end of the show without hearing any other level glitches.
An extra thanks to Steve for his investigation of the Windows system time issues - it gave me the clue I needed for some searching that uncovered the solution to a problem that had been bugging me. I have a couple of systems not in active use - an early Framework laptop, and a Lenovo Yoga A940 like Leo uses in the big studio - where the system time would always be wrong, usually a month behind, and the automatic update from a time server would fail for being too far out.
From new, they’ve only been turned on once a month for the Patch Tuesday updates, so my surmise is that maybe they use rechargeable CMOS batteries and are not on long enough for the battery to charge up.
What I discovered, from looking up details of the UtilizeSslTimeData key in the area of the Registry that Steve mentioned, is that the same Registry block also contains limits for the max amount by which the clock can be automatically adjusted forward or back. Studying the page Configure W32Time against huge time offset - Windows Server | Microsoft Learn suggested that the current default setting of 48hrs is of benefit for domain controllers in big networked setups, but setting it to a larger value seemed unlikely to cause issues for standalone workstations, especially as they’re unlikely to be powered up without a working network connection. My auto time adjustment in Settings is set to use a local NTP server rather than the default Windows Time server, so it should be unlikely to suffer from distance-induced time drift mentioned in the article.
Accordingly, I set both neg and pos keys to the original default of all "F"s used in earlier versions of windows, rebooted - and my problems were solved. In each case the system time is now reliably accurate at startup. In case it might help anyone else, here are the changes I made:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config:
-
UtilizeSslTimeData set to zero
-
MaxNegPhaseCorrection set to FFFFFFFF
-
MaxPosPhaseCorrection set to FFFFFFFF.
If anyone can see a reason why this would not be a good idea, do pitch in, but I’m currently happy with systems I don’t have to adjust the clock on every startup.
I appreciate the reports of sound issues. We’re trying to track these down. If you hear it when I’m speaking, too, that’s a big clue. Thanks!
1 Like
I’ve subsequently updated the utility to have a GUI. Here’s the info:
I mentioned this before, it’s taken longer than I would like to get
this to the stage it is at, but here we are.
This is meant to test a USB device by FILLING it with data that it then
verifies can be read back. Do not use this on a device that can’t
withstand the write (flash wear) or that would otherwise lose data you
care about if the flash drive decided to drop old data when new
arrives. I suspect this to be similar to the hwtest app people have
mentioned, but I have not use that app, so I can’t really be certain.
It’s written in Java, but packaged with its own runtime, so all you
need to do is download the 7ZIP file and unzip it to whatever location
you’d like. It arrives as a directory and inside that directory there
is an EXE. (It’s packaged for Windows, it could also be packaged for
Linux, but I don’t have the necessary machine available for that right
now.) The JAR file is also present, so if you’d like to run it on
Linux or MacOS and you do have the proper JRE (20) available, you can
run it from the command line with the “java -jar
DiskFillerAndFakeCatcher-0.0.1-SNAPSHOT.jar” using the jar in the inner
“app” folder.
The file is password protected because the file host views executables
(including Java JAR files) as dangerous. And indeed they could be, so
only use this if you trust my reputation to have provided it.
https://files.catbox.moe/bl7til.7z
Password for 7z file: DiskFillerAndChecker2023Sept16
FYI: The 7z file was originally named “DiskFillerAndFakeCatcher.7z”
before the file host mangled the name.
It should be pretty straightforward to use. In the text field enter
the drive letter (like D:) or on Linux or MacOS you’d need to give a
mount point. (Mount points have NOT been tested, that’s why I’m
releasing it, so feedback any error messages.)
There are three buttons: [Fill] [Verify] [Clean] (they have nicer names
on the buttons, but basically these are the functions.)
Start with Fill and it will write a large file(s) to the device until
it’s full. Unless your USB device is faulty, it won’t harm anything
already present, it uses “random” names that are VERY unlikely to
collide with anything.
Once the fill completes (it could take some time, depending on the size
of your USB and the speed of your PC/USB) you can then run the Verify
button to make sure the files provide back data as expected by what was
written. This should catch any fake drive spoofing content.
When it’s done, clean up the files with the Clean button… and your
test is complete.
If you test this, please let me know how you make out (platform info
please.)
1 Like
I think that was Steve’s mistake. He wrote to random parts of the drive, but not all of it. You need to exceed the capacity of the original storage space in order to see if it is a fake.
Writing to the beginning, middle and end of a drive won’t help. If it has a 2TB drive masquerading as a 10TB drive, you need to write over 2TB, before you can be sure. Likewise, if it is a 256GB drive masquerading as 2TB, you need to write more than 256GB.
Because you don’t know the capacity of the original, you have to essentially fill the whole drive to be sure, which is why you have to test, before you start using the drive, and why all the other tests are destructive.
3 Likes