A surprising way to lose your files on Windows
They logged into Windows 7 (I know, upgrade...) and it looked like their files were missing. In a panic, they opened Explorer and searched for their files. They turned up in the search. They just didn't show up in the usual "My Documents", "My Pictures", "My Videos" paths.
They decided to move the files "to the correct place". And then they shut the computer down.
The next time they started it, the same thing happened. This is where I got called in, because this time, the files didn't show up in a search. I told them to turn the computer off immediately and drop it with me.
Can you guess what happened? Well, check this out:
- Windows couldn't use their user profile because it was corrupted
- So it created a temporary profile in "C:\Users\TEMP". (This wasn't obvious to the user because Explorer hides the 'detail' of the file path and simply shows the username)
- Unwittingly, when they moved the precious files to the "correct" place, they were putting them into a temporary profile.
- On shutdown, Windows promptly deleted the temporary profile, so "C:\Users\TEMP" got wiped along with all of the files.
I was frankly astonished that Windows would drop them into a temporary user profile without dire warnings about its transience. Anyway, now I have to try to recover not only the files, but the directory structures. I'm not even sure it's possible... :(
248 comments
[ 4.8 ms ] story [ 259 ms ] threadSo this is the source of the problem. How does it happen?
You can use autopsy which is open source to carve for files. Or you might be able to use binwalk.
Does it? I recall using Windows 7 and being dropped into ~TEMP sometimes, but I was warned with a notification.
The trick here is how do we solve this other than further user education? When the user profile folder is unusable for some reason you either have to do this or just refuse login. Neither are good answers, but this one is better as long as you understand what the system's doing.
If one doesn't know what the system's doing and isn't interested in trying to understand it I don't see a way to avoid this issue.
As this whole discussion shows, no, this one is worse since it can easily lead to data loss. A third option, however, would be to do this but not erase the profile on logout.
That is apparently false because the user did access their files and dragged them into the temp profile.
Either that should be literally true (a completely broken profile is a bad problem that needs help from a competent tech support person) or the profile should at least be made read-only until the entire profile is deleted. That way they could copy the files into the temp folder, but not lose anything (other than changes) after logging out.
Also, the warning should be more obviously "fatal", not something to be clicked through. E.g. replace the desktop background with a black screen and put the warning text in red on it.
Hell, even just changing the terrible wording on the notification (and putting it somewhere much less ignorable) would be a step forward. E.g. "Your files have been temporarily moved to X. Any files you place in My Documents, My Pictures (etc) will be deleted when you log off or turn off your computer.").
Or the move opration immediately prompted for admin authorisation, and they just clicked through that? (not suggesting that these prompts are in any way useful for the average user)
https://answers.microsoft.com/en-us/windows/forum/all/my-pro...
A notification in the status bar, along with their other 10+ notifications about Skype, printers, antivirus etc. They didn't see it, or if they did see it, didn't understand the implications. The horrible thing is that they didn't want to bother me with it, and tried to solve the issue themselves :(
I also doubt the typical user will know that profile is shorthand for user profile and know what that is.
End result: because of corruption of some data the user likely is only remotely interested in, all data they were interested in got deleted.
https://www.windowsphoneinfo.com/attachments/97fdf7d3-6fca-4...
I remember undeleting was possible in the old FAT days.
However, this also shows why having a "backup" is always a good idea. So when/if you do recover what you can recover, you might also consider setting them up with some form of automatic backup such that should this (or a disk failure, which is always a possibility) occur in the future, recovery can be "restore from latest backup" instead of the task you are now facing.
However, an equally good idea when faced with a sudden and utterly unexpected situation, is to not immediately try to solve the problem without understanding what actually happened. The chances of the "fix" doing more harm than good is high.
That said, consumer software should definitely not assume that the user understands how the internals work. In this case, rather than a popup being shows, it would have been more appropriate to give the user a full-screen walkthrough, similar to the first-launch experience, explaining what has happene, what Windows has done, that the documents and pictures are most likely safe, and finally include warnings about the active profile, and associated documents and pictures, being deleted when logging out.
I would be much better to show an error at login saying there is a problem with the account and a question like "Would you like to log in as a guest user?"
That's basically what I was suggesting.
(My story: just last week I fixed a temporary profile issue for an acquaintance and all that had happened was that Windows messed up the connection with her online Microsoft account. After signing out and in again, it worked fine. I was surprised at first because I wasn't aware that such a thing as temporary profiles existed. It became clear that she was working on a wrong profile once I opened a shell and the path before the prompt said "c:\users\temp".)
There are definitely warnings that any work/data stored in the temporary profile would be lost, so maybe they did not take these seriously enough.
The only thing that I actually find absolutely unforgivable is that the temporary profile is named after her normal account. Creating temporary accounts with ample warnings, yeah, not great. Pretending to be the profile of the user...? Nope
Microsoft seems to have tried to get rid of outmoded notions such as directories, without ever articulating what new model they favoured. So you get these privileged spaces (I have no better word) like "Documents". They've tried to make Explorer as invisible as possible.
But they never finished the job! So you end up with a mess of "AppData" (apparently a normal directory) and "Application Data" (not accessible). I presume the first is an alias for the second; but the second should be invisible if it's not accessible (there are other directories like this).
This started with Win7. Then they stopped abruptly, but never reverted these aborted changes. It's as if there's nobody in charge. My guess is that forcing Microsoft accounts, "instrumentation", and failed attempts to hegemonize mobile were prioritized over fixing what they'd broken.
NT4 was a reasonable OS, with a rotten commandline. Nothing they've shipped since has come close.
And all of these partially duplicate functionality. Want to change the password of a different user account? You can’t do it from the user account panel - you can only do it from the other user account panel.
But yes it is a mess and only as an IT person I can somewhat make sense of when to use the modern Settings - Accounts, Control Panel users, MMC or sysdm.cpl
I've always joked that the gpedit.msc is where the real settings are. It's one of the pieces that has stayed the same (and even been added to) while all the other parts are gradually dumbed-down and becoming less useful.
But it also made the 250 character path limit much easier to reach. When the NT unification happened with 2000/XP the shorter AppData was used with "Application Data" remaining as a junction point to the roaming profile for compatibility.
It is invisible unless you decided to enable "Show hidden and system files".
Should Explorer have three modes, "no hidden", "show hidden and system but only some", "show all files"?
Who would complain if they would make it like that?
Thing is, I don't want any hidden files, ever. I've been driving with "Show hidden and system files" set for 20 years. It's just about the first setting I change on a new Windows system.
What the fuck?!
So it all but forces you to link your identity to something Microsoft owns... just to refuse your identity back and throwing you into some unrecognizable woods if Microsoft fails to validate it.
I guess failing to log you on your own computer because MS is down would enrage people. So it's better to gaslight them until they give up.
And there's this nagging alert that keeps popping up:
"Microsoft account problem."
"We need to fix your microsoft account. (Most likely your password changed). Select here to fix it in shared experiences settings."
Which I assume most novice users would obey.
https://answers.microsoft.com/en-us/windows/forum/all/seeing...
My ritual for family machine setup is locate wifi kill switch-> if not available, announce wireless going down-> kill network->setup to local account->reenable network
It's patently infuriating. Throw in where I've had family members set up Microsoft accounts with only 1 form of auth (Phone number), change that, them get locked out of their Microsft account for a month because of Microsoft's daft policies. Though I didn't set up that machine. It boggles the mind that Microsoft's default flow would run the risk of such an outcome.
[1]: https://www.thurrott.com/windows/windows-11/269419/tip-set-u...
No we won't just add a "No thanks" button, but we'll set up an undocumented magic username to serve the same purpose?
And to make it even more nightmarish, where is that check? Is it extractable from the installer binary? Or is it some weird exception case implemented in server code only viewable by microsoft?
For that matter, who owns thankyou.com? Is there some unfortunate sod with no@thankyou.com as an email address?
There is so much wrong with this. I can't even...
There is nothing about that setup that isn't an exercise in plausibly deniable forced on-ramping. No it isn't a best practice to create the illusion you can't use an operating system without agreeing to a cloud services contract.
That's BS, and should be called out as such. UX is trying to gaslight the non-technical into services they don't need, and I'm willing to go out on a limb that everyone in the market is trying to converge on that exact practice.
And when you did, the next step required you to first select "I don't have internet" then next be shown some embarassing pseudo-technical propaganda appealing to your FOMO, to provide one full page of discouragement before you agree to "continue with limited setup" if you want to actually have a full regular local account. And that illusion is maintained further into the user experience, where status will sometimes be reported as "setup incomplete".
With the Sept 2022 release of W11 you can't even do that any more.
When you reach this point and there is no longer the option to admit your poor soul has no internet, the incantation here is to hit Shift+F10 which opens a command prompt. Click in the CMD window to make it respond to your keyboard then oobe\bypassnro. Reboots and reverts to previous "I don't have internet" option.
Sheesh.
Domain joining from setup is impossible (it requires a reboot, and then group policy could push something that breaks the rest of the out-of-box experience), so it's safe to assume that will always be there (unless they do a major change to how AD joining works, but they probably won't since they consider it "legacy").
https://imgur.com/a/TXNtJHy
Select "Join a domain/corparete" network and there is a button for offline account.
But yes, if you get an script with the set of actions that make local accounts possible, you can replicate them.
Seems like it still holds!
But the current versions of windows 10 only offer that if you don't have internet (no ethernet cable in, and answered "I don't have a wifi" in the screen before user creation), if you connected to the wifi it won't show the option for local account.
https://imgur.com/a/TXNtJHy
Select "Join a domain/corparete" network and there is a button for offline account.
If Microsoft has full access to your hard drive and can make changes to your system and files any time they want they own a lot more than your user profile. It's their system. They are the admin, and you are a peasant and the extent of your access to your(?) data and Microsoft's system can be limited or revoked at any time.
The real-world number of linux admins who inspect all packages before applying them is basically zero. Most Linux/BSD systems likely have unattended upgrades enabled by default.
How do you diagnose a corrupt profile? What causes corruption? Is there some diagnostic I can run that will detect and repair corruption before it's too late?
Not on Windows 7
https://www.cgsecurity.org/wiki/TestDisk
edit: oh, I just read that you mentioned TestDisk. Yes, it can recover the directory structure, at least it did in my case. After using it you will likely see that some files have been duplicated with a suffix added, like document.doc and document.doc.xyz (can't recall the exact suffix) in the same directory. In that case you need to check which file has been fully recovered, and rename it if needed.
You'll do much better with purpose-build Windows undelete utilities. I've had good luck with freeundelete:
https://www.officerecovery.com/freeundelete/
https://www.ccleaner.com/recuva
Photorec is about files only.
They are both vey good at what they are supposed to do, but in your case Testdisk is not suited and Photorec represents a last attempt as recovered files are carved, basically you will lose not only directory structures, but also filenames and metadata.
A good GUI tool (not of the "one click" type, it needs some learning to use it) is DMDE:
https://dmde.com/
When you have a similar problem on Linux, and you home is inaccessible or read-only, you get an almost screen-sized dialog, no desktop background (something hard to replicate on your settings), and the system theme (that is probably different from your DE's default).
https://www.windowsphoneinfo.com/attachments/97fdf7d3-6fca-4...
Windows 7 left extended support two and a half years ago, mainstream support seven years ago, and hasn't been in development for eleven years. (Like commenting from Windows XP in 2001 that Windows 3.11 in 1993 had an unhelpful dialog box).
of course it's always been the unix way to be strong-silent without have a lot of "informative" message clutter, so I'm generally sad to see it friendlied up.
Unix is for silent success. I don't think there is any overall culture about what to do on errors.
But anyway, when you lose your home dir, your computer becomes obviously broken. Your a bit more than mine, but it's the obviousness that is important. (Anyway, I don't gain anything from the extra functionality my system holds when compared to yours. It's always a hardware problem, a X session is useless.)
WTF. That looks really amateurish. Why couldn't they make the notification window larger?
My lessons learned from it.
windows profiles are far too complicated, it was hard to find low level information on how they work or how to diagnose and fix a bad profile.
make sure you have and know another login. preferably the admin one.
Edit: this might be what you meant by the admin one (rather than a)
What is so baffling to me is that it's not being fixed. It would be understandable if it only affected power users or if it was a launch-day issue getting patched in the first week. But it isn't.
See the recent experiences with Notepad or Voice Recorder for examples.
https://www.youtube.com/watch?v=hzClnwGeJUM
In this case the deleted files would have to recovered. I recently use Easus Data Recovery Wizard and was able to restore a lot of files for a client who's photo storage drive crashed. It would be best to pull the drive and do the recovery on another computer. I think the free tool allows up to 2GBs of data recovery but in this case the client had 100's of GBs so he paid for a 30 day license for $65.
And take an image before booting the disk.
Is this a normal thing in Windows?
Not having a backup is a platform agnostic way to lose files, and works pretty reliably given sufficient time.
Windows 8 came out in 2008.
Typo; you meant to say 2012.
Not affiliated in any way, just a sporadic user.
https://www.forensit.com/
But you really have to try and put in some effort here.
Without taking action to have it your way, you'll be left to the default approach which Microsoft has molded for users who may never grasp filing systems at all, which is simultaneously backward-compatible with DOS. And then third-party software which sometimes tries to better cope by having their own layer of obscurity which can help your workflow but the filing can end up even more ugly.
But the filing cabinet never was an office machine anyway, and as far as automatic, I would much rather have a real intelligent secretary file & retrieve my folders than I would a machine.
And without a secretary I figure I'm going to have to put in some effort my dang self, and get the most out of the electronic filing at my fingertips.
The C: volume containing DOS or Windows was subject to the most frequent defects or corruption and could be considered the most risky place to put any of your valuable data if you could help it.
But regular PCs for office or home were supplied with only one partition on their HDD so there was no separate volume that could be dedicated exclusively for storage. The highest-reliablity upgrade was having a second hard drive dedicated for storage, but naturally you had to work from a blank slate and label all the folders yourself just like a secretary would do. Further you need to be very vigilant and direct any valuable data from all Windows or third-party apps to your dedicated folders every time. However nobody actually needed this drive space at the beginning since that's when there's always plenty of room on C:.
But it was worth it.
Be your own secretary.
Yes the C: drive was still a PITA to restore but it was good to maintain readiness to Format C: at any time without losing any valuable data at all.
Then restoring only the system (Windows, installed apps, and settings) quickly from backup, or alternatively reinstalling them in the previously optimized repeatable full recovery procedure. Always install programs to C: right along with Windows. To a newer, faster, or bigger HDD whenever you felt like it. The C: drive was like the office machine substitute, where you need to be able to change some of the things on your desktop or even get a whole new desk from time to time, but you wouldn't want to handle your company files and their filing system as disposably as you would a broken office machine.
And the second HDD volume was the filing cabinet substitute, which could physically be moved or copied and placed into a different office without affecting the desk or desktop that generated the data.
Now this was before the Documents And Settings folder appeared. Which pushed the mainstream in the opposite direction, placing your valuable data by default into folders automatically labeled and tucked away largely within deeply nested trees within Windows' increasing number of self-created folders, and completely downplaying any underlying need to know the filing system at all. Instead of all being in one filing cabinet, your data became a "part" of Windows and it became acceptable for your files to be less accessible to your fingertips alone. Not without Windows and each App in question being there in perfect working order. And with temporary files from the internet growing right there in the same folders, rendering your relatively small number of useful business files more like a needle in a haystack as time goes by.
Is this risk really worth it just to have the simplified option of multiple users in offices where everyone needs their own Personal Computer to begin with?
Anyway they basically call it the Users folder now and you've got a symbolic link from Documents And Settings, so the woods got deeper that time too.
I hate having to knoc...
But before you do that, make sure the issue isn't the hard drive itself. If the hard drive is damaged, you may still be able to recover data, but it'll need to be done by someone who knows what they are doing. If the drive is spinning irregularly, or clicking, or not recognized by another system you might have issues.
Source - I used to work in a digital forensics lab.
https://d3nevzfk7ii3be.cloudfront.net/igi/n4Fo16YKi45Y5JxL.f...
Source: https://www.ifixit.com/Guide/MacBook+Air+11-Inch+Mid+2011+So...
I really really dislike Windows when it hides paths, for example, "My Pictures". Where the heck is that?
This lunacy infects Windows applications, like Quicken and Thunderbird, which spew themselves and their data into gawd knows where hidden directories. Why can't they simply do the obvious? Create a \Quicken directory in the root, and have a \Quicken\program and \Quicken\userdata ? But noooo.
Networked machines. (Aside from the obvious security problems on a multi-user installation.) The "gawd knows where hidden directories" (of which there are two, %APPDATA% and %LOCALAPPDATA%, plus, on a recent Windows version, a number of compatibility links and some sort of integrity level thing) are segregated into the part which should have an authoritative copy on the file server ("roaming") and the part that can or has to stay on the workstation ("local").
And you aren't the only user on pretty much every OS. There are many many many built in "users" for different internal operations.
Your user should be the admin, shouldn't be tied a profile etc. Linux is setup like more like this.
Yeah, it makes it more complex for certain network style things (you have 4 computers and want to sync between them), but it's trivial to work around.
The reason Windows is built like this is they develop first for enterprises and libraries, everyone else gets the shitty castoffs...
Android has been copying Apple in that it now also has automagic folders and file managers. I also remember Google doing some idiotic A/B tests some moons ago of hiding the actual page result URL, it was one of the reasons (on top of search quality) that I stopped using it.
On Android you now have to go through multiple settings to enable explicit paths as if was somehow an 'advanced' feature to show an explicit file path.
I assume it is mostly driven by UX somehow it is now an agreed truth that file manager abstractions are too complex for the average user to grasp therefore must be vigourously hidden at all cost.
Or to people in different age brackets: for a few years now, people have finished uni while doing all their work on phones and tables without a desktop OS involved. No files, no desktop, no actual applications. We should stop copying desktop mentalities into modern workflows.
Or the other way around: nobody has desks or paper folders in file cabinets anymore, nobody needs to physically handle files and organise them, nor do their have boxes for physical mail that comes "in" and goes "out". We should stop copying legacy office concepts into software.
Knowing how to store, organise and retrieve information, regardless of what system is used (including computers, file cabinets and archive rooms) is a skill that doesn't just magically appear in your brain, you have to actually observe, trial or classically learn about it to make use of it. It used to be through day-to-day tasks and a growth in responsibilities when growing up that people got that skill semi-automatically, but that is no longer the case.
Most systems have or used to have tagging functionality as well, but apparently that wasn't used all that much either for information classification and organisation.
Granted, I can't read minds and I don't know what everyone everywhere does, but I do see large scale changes in employee instructions for new hires, university hires and even in cross-org/education projects. For most young/new people, their phone was their first 'computer' and the 'default' environment, and a desktop is the anomaly. People looking up how to do things on ticktock is not a funny meme, it's reality and seriously tainting the knowledge of junior hires on the lower end jobs (like entry level service desk backoffice). Knowing where to find authoritative information, how to verify it, and how to use that as a reference to validate and find other information seems to be a foreign concept to some as well. Even something like a somewhat specific google search or wikipedia article and the references it can supply are not as normal as one might think, and the gap is getting bigger, not smaller.
When they want to find stuff, they just search for it in the file picker. Or hopefully the file picker is just already in the right place. Often this UX is kinda bad, and it's hard to find things. Sometimes they give up, and either do without, or (if applicable) visit a website to just re-download whatever they can't find. Or they ask a technically-adept friend to help them.
Carefully creating a directory hierarchy and actively picking and choosing where to put things when downloading or saving files is just not a thing most people do.
They aren't morons. They don't necessarily understand all the complexities, like aliased folders, etc., but that's because they don't need to and so haven't tried.
But parent is right. Nobody is out here maintaining and pruning their beautiful directory hierarchy, and complaining about how messy applications make their home directory. Because nobody turns on “show hidden files”. Nobody cares.
These are organizational concepts, not software concepts. People, even mobile uni students, have and must master them. As a result they get reinvented constantly in/as apps.
What people are mad about is people are told 'this is how you will organize' rather than just learning to organize. It is absolutely fucking stupid.
So actually the problem is worse. Apple is schizophrenic about folders and leaves their users with the cognitive dissonance.
Windows has an existing convention for this, like Linux's XDG. Imagine if you looked at your Linux filesystem root and saw /boot, /dev, /usr, /tmp, and /Quicken.
You'd probably see a /Quicken/ as well as a file /Quicken-uninstall as a broken symlink to nowhere.
Beyond that, getting a full path is trivial (realpath) and figuring out which block device backs it is also quite straightforward (findmnt).
There are also strange constructions that don't actually correspond to a single folder, or at least there were the last time I used Windows which was quite a few years ago now. A place that looked like a photo directory in Explorer, but actually it was a view of multiple directories and I don't know where stuff ended up if you copied into it.
The Explorer does not browse the file system, it browses a graph of (COM) objects called the “shell namespace”[1], and has been since Windows 95. Notable examples of non-filesystem nodes include My Computer, Recycle Bin, Control Panel, and Network Neighbourhood; a more obscure one is the All Tasks (“god mode”) collection of all settings applets that exists so that the Start menu can search it[2].
I actually think it’s neat how seamlessly this approach has been integrated—we don’t usually think about it even though we actually encounter it all the time. Microsoft’s attempts to extend this to a full-blown object-oriented (Cairo) or relational (Longhorn) filing system were always so muddled I don’t really get what the actual idea was, though.
[1] https://learn.microsoft.com/windows/win32/shell/namespace-in...
[2] http://brandonlive.com/2010/01/04/the-so-called-god-mode/
Contrast to windows where your "Documents" folder might be getting backed up via one drive or roaming profile, or it might just be local disk on that machine.
In my experience for a corporate windows environment its best to map a H: drive to a network home drive share and get users to always save in there - if its not mapped it means you are probably on misconfigured host.
macOS is the same.
The second best is a fully manual directory structure with no “surface” added. But this is not nearly as good for the average user.
By far the worst one is what you get when you (correctly) like Microsoft realize you are in the second category but you’d like to be in the first. The end result is an annoying middle ground that has plagued windows for a long time now. They probably (incorrectly) assumed they could get to the iOS situation eventually.
Separating the application and its binaries isn’t a bad idea though. I don’t want to back up applications, just their data, for example. And I want to be able to have different permissions for the app binaries and elsewhere. So programs\Quicken and data\Quicken would be better than the other way around. And this isn’t far from what windows does. If I create an app I put the user data in %appdata%\myprogram. The folder is annoyingly hard to find though but that’s more an explorer UX issue than a problem with how the files are stored.
• Videos I need to watch for class.
• PDFs I need to read for class.
• Word documents I am writing for class.
• Photos, videos, and audio recordings I will use to write papers for class.
The "iOS model" would have me segment all of this data by file type. The PDF from New York State on K-12 educational standards would be next to my 2021 tax returns. My paper on single-sex schools would be next to the technical documentation for a web app I'm developing. This would be terrible.
Files need to be organized by purpose, not file type. I know what my files are for. My computer does not.
This is also a major reason why I hate my iPhone.
I’m not saying the final solution to data organization is just “all Images in one directory”. The solution is one where you don’t worry about directories and still easily find and group the data you need.
Windows had grand plans for this that were scrapped. Understandably, but also sadly.
Oh, I absolutely agree! But as I said, I know what my files are for, and my computer does not. So until we get some incredible AI that not only understands the context of every file but can also use that context to make perfect organizational decisions, I expect to be the one responsible for grouping my data.
To be clear, I don't think directory hierarchies are necessarily the be-all end-all of digital organization. For example, I like the idea of organizing based on tags, because a lot of documents rightly belong in more than one category. However, I expect I'll need to be the one who sets the tags.
iOS's solution does work better on mobile than on desktop (and iPad), although I do run into problems on my iPhone.
All that said, I do agree with the general premise that files should be organized in reasonable places by default and users should be able to override that should they see the need.
Contrast this with traditional OS X. I can open a file in any app, but the file continues to live in Finder. In a way, you could say that the Mac puts files first in the way that iOS puts apps first. An app is just a tool, and could be used for anything: I write papers for school and documentation for work and poetry for pleasure. A file, on the other hand, has a distinct purpose.
And yes, of course no one should be stopped from having one area where they have a word document, six images and a text file. OF COURSE. But whether that's a directory or not isn't what's interesting. The interesting part is that I can group them together logically, find and use them easily etc.
Also, don't use 14 years old EOL systems.
I helped my wife reorganize many hundreds of images. Finding out where they really were located and where they would actually end up was a tremendous chore, and locating them again has continued to be difficult.
Even doing a backup is stressful. Where are the files really located?
I suppose I could create some automation tools, and I have thought about that.
I'm sure there was some warning. But you know, users don't read messages...