> The only person in the world who leapt to my defense was, of course, Raymond Chen, who is, by the way, the best programmer in the world, so that has to say something, right?
So in my mind I've made the connection that Raymond Chen = best programmer in the world since then haha.
This is an interesting article, because it's a product of its time: modern languages solve a lot of these exact problems. I think this is a resounding success that they have correctly identified a genuine problem that people used to struggle with (safe types, exceptions) and make it standard and correct and ergonomic.
Yes, the same way one could write VBS (Visual Basic Script).
I think Windows 98 already had this ability. Possibly Windows 95 as well. It's a variant of the language called JScript, which is what was used in old versions of IE too.
It was about Windows 98 that Windows Scripting Host ended up prominent.
WSH btw allowed you to run any language that you had interpreter for - they had to support necessary COM interfaces (and to be truly usable, allow you to call COM objects), and register their interpreter class with ActiveScripting (WSH internal) engine.
Then you could use them not just for desktop automation, but also for scripts inside Internet Explorer (essentially, classic IE used WSH engines to implement scripts, iirc)
I've seen WSH (including HTAs) used with Perl, Python, Tcl, Rexx...so long as you install the interpreter with compatible COM service, you could use it.
Been using this for years. Mostly really useful. Sometimes tricky to get right since the available APIs are semi-well documented and it's JScript, which is some sort of old Internet Explorer-ish version of JavaScript.
By the way, there are also HTAs, which are Microsoft HTML Applications. You can create a simple double-clickable GUI with these using only HTML and JScript.
Pretty crazy how Microsoft basically invented the Electron app as HTAs all the way back in 1999. Of course we browsers weren't as capable as they are today, but "I just want a HTML+CSS GUI" had been a solved problem for over ten years when Electron first came out.
Yes, and XULRunner allowed this too, using Gecko, Firefox's web engine, to render HTML-like markup specifically designed to build native-like GUIs.
Apparently XULRunner was first released in 2006, but Thunderbird, which uses (used?) the same technology, was released as early as 2003, and maybe this was existing in the Mozilla Suite even before.
Thunderbird never quite used XULRunner, I think; they always built their own binary (though at some point quite a lot of the shared stuff moved into the XRE stuff). Think of it as they had a fork of Firefox (much like Firefox had a stripped down fork of the SeaMonkey stuff).
Also, I think one of the Start Menus (might have been XP‽) was kind of HTA-ish? Not sure about that part, though.
Notable projects using actual XULRunner included Songbird (a music player) and BlueGriffon, an WYSIWYG HTML editor (a successor of Nvu and KompoZer, themselves succeeding Netscape Composer). Both released after 2006 indeed.
I liked XUL, I strongly believe Mozilla could have dominated the market taken by Electron, had they pushed XULRunner more, and perhaps make it transition to pure HTML, like they did to Firefox's core, because that's what people know and because XUL was a maintenance burden. I think XUL tags made more sense than HTML to build UIs, though, and with XUL, Gecko have had a CSS flex-like mechanism for a long time by the way.
There was an experiment back in the hazy past around that time called Entity that did something similar. It was never complete enough to be a competitor to XULRunner, but it was fascinating for two reasons:
1) You could write event handlers in multiple languages, including C. If you wrote them in C, it spawned gcc and compiled it into a library, and dynamically loaded it... The overall idea of a polyglot runtime like that was fun.
2) #1 is only really weird because this could be done at runtime. One of the demo apps was an editor for the GUI itself, where you could add buttons to the editor, then write that event handler in C, and have it compiled and loaded into the editor itself...
It was a fascinating starting point, though full of heavy duty foot guns, and I'm still sad nobody took it further.
> The overall idea of a polyglot runtime like that was fun.
Active Scripting, which powers scripts in both WSH and old-school IE including HTAs, is polyglot and extensible. It’s why Active{Perl,Python,Tcl} are called that—the original headline feature (IIUC) was that they integrated with it. It’s also why you could write VBS in IE: IE just passed the text of the script along with the language attribute to AS and let it sort things out.
Nobody did ever a C interprerer, though, I think—perhaps because you basically have to speak COM from Active Scripting, and while speaking COM from C is certainly possible it’s nobody’s idea of fun. (An ObjC-like preprocessor/superset could definitely be made and I’ve heard that Microsoft had even entertained the idea at the dawn of time, but instead they went with C++, and I haven’t been able to find any traces of that project.)
That’s not to say AS is perfect or even good—the impossibility of caching DISPIDs[1], in particular, seems like a design-sinking goof. And the AS boundary was also why DOM manipulation in IE was so slow.
the best epub reader for desktop I've ever encountered, was epubreader (pre-WebExtension version), I used to launch it as a standalone app with XULRunner.
For the curious: Here's a completely unfinished guide to how you might start developing such an application: https://marksweb.site/hta/ From HTAs, you have access to the file system, the network, the registry, the shell - everything. It might be a bit different than normal web dev, but it's not too bad either.
Wow, that's so cool! I played around with making HTAs as a kid and never thought those could be that powerful. (I quickly moved on to topics more exciting to a teenage hacker, like making WinForms apps with some PHP RAD IDE.)
Wondering what would it take to port mshta (with all the ActiveX goodies) to other platforms. Maybe it's a little bit late for that, but sounds like it might be a fun project to me.
To be honest, in my ideal world, mshta, Electron and the like would be discontinued and, instead, there'd be a cross-desktop-platform HTML/CSS/JS app-runtime (_not a browser!_). This runtime should support a sensible, large subset of modern Web APIs plus a set of cross-OS and OS-specific APIs so it's easy to work with for developers. To be easy to use for users, it should be installed by default on all major consumer-facing OSes. So yeah, it's probably not gonna happen anytime soon...
You're brave, putting "ActiveX" and "fun" in the same sentence.
Wine Gecko supports ActiveX, supposedly, so if someone implements all the common ActiveX components, that could be a cross-platform method of running HTAs outside of Windows.
That said, I'm afraid the Electron API is the closest thing we have to a cross platform HTML application these days. On Manjaro, several packages are already implemented by installing Electron next to the application specific code, so that would be the closest thing to a modern HTA alternative that I know of.
PWAs work fine if you don't need integration with the system itself other than file prompts, for chat apps for example. They're not really alternatives to HTAs to be honest.
It should be noted that HTAs are a common way to infect computers (because they're executables that aren't usually recognised as such) and they're disabled in many security conscious environments.
It is very common for malware to contain java script payloads that try to obfuscate themselves like like this:
Seemingly_random_code(seemingly_random_string)
The seemingly_random_code decompresses/decodes whatever is in the seemingly_random_string and hands over control to it. Interestingly the decoded code is another version of the same with different code and string. This goes on for ~100 layers deep then at the end it just downloads and executes some file from the net.
> This goes on for ~100 layers deep then at the end it just downloads and executes some file from the net.
I understand doing one layer. I guess I could maybe see two layers. But why would it bother with 100 layers? Either the antivirus or reverse-engineering tool can grab the final product or it can't.
Typically scanning tools have some limit to how much they probe complex formats, to avoid stalling the entire system while they're scanning. It's very much conceivable that a scan tool will try to resolve code like this for 10 layers, and then if the result is not found to be malicious, consider it safe.
This is similar to how compilers will often have recursion limits for things like generics, though in that case it's easier to reject the program if the recursion limit is reached.
Because of potential false positives, and the speed at which files need to be analyzed at runtime (suspend process executing it and then analyze it), having files which take a long time to unpack and identify can cause these to be allowed to run. They get offloaded to a sandbox or other systems to be analyzed while the file is already being executed. The sandboxes are too slow to return a verdict before the main logic of the file will be executed. IF those dynamic systems cannot identify a file, an engineer will manually need to look at it.
In very strict environments or certain systems it might be practical to block all unknown files, but this is uncommon for user systems for example where users are actively using javascript or macro documents etc. (developers, HR, finance etc.) The FP rates are too high and productivity can take a big hit. If all users do 20% less work that's a big loss in revenue (the productivity hit can be much more severe even!). perhaps this impact / loss of revenue ends up being bigger than a malware being executed depending on the rest of the security posture/measures.
technically its possible to identify (nearly?) all malware by tracking p-states/symbolic execution/very clever sanboxing etc.. but this simply takes much too long. Especially if the malware authors are aware of sandboxing techniques and symbolic execution and such things as they can make those processes also take extra long or sometimes even evade them totally with further techniques.
I wish it _was_ possible to do all of the cleverness that malware researchers have invented to detect things, but unfortunately, in practice this cannot happen on like 90+% of environments.
If you run like a DNS server or such things, it's possible to do it as such a system would not be expected (ever?) to have unknown files. (gotta test each update and analyze new versions before deploying to prod). As you can imagine, this is also kind of a bummer process but imho for such 'static' systems its worth it.
Malware delivered as an email with a link to a zip file containing a .js file is one of the most common methods of delivery, right behind word macros. The "map the .js extension to notepad.exe" is a common security trick with a measurable, immediate drop in malware in large orgs. You can deploy it via GPO or InTune.
Personal promotion, I built this as a better alternative:
> can't you write [a self-deleting executable] in C?
The point of the exercise is that, on Windows, you can’t, because Windows won’t let anyone delete executables that are currently in use (try it, you won’t be able to delete one either). Upgrading shared DLLs in the face of this fact is why installers for Windows programs often have to have you reboot the system (and in more civilized times asked you to close other programs before installation to reduce the probability of hitting a locked DLL). It’s also why there’s a registry key[1] containing a list of rename and delete actions to be performed on next reboot (usually accessed via the MOVEFILE_DELAY_UNTIL_REBOOT flag to MoveFileEx).
You can’t (straightforwardly[2]) make a self-deleting batch script, either, because the command interpreter parses a command at a time and so wants the batch file to exist. The Windows Scripting Host, on the other hand, will parse the whole file at once, close it, and then forget about it, so you can write self-deleting WSH scripts.
The workaround used by the uninstaller under discussion is instead for the executable to inject some code into the Windows Explorer (on the assumption that it’s always running and the user has to have access permissions for it) that accomplishes the deletion through return-oriented programming, so that the stack it’s executing from can then disappear into the wind (apparently? I’m not seeing how they plan to clean that up).
On a POSIX system you are explicitly allowed to delete any open file—including an executing one—making it languish in a kind of system-managed limbo (and take up disk space, invisibly) until it’s closed. The tradeoff is then that it’s impossible to ensure you’ve opened the same file as somebody else when all you have is its name. (I think you can at least check for success, provided you also have the device and inode numbers for it.)
The key is that unsigned scripts are opt-in, not opt-out. Chen is not going to suggest a solution that requires all users of the software to configure their computer to be less secure.
It's not really a security measure in that sense. It's a "safety feature" that prevents accidentally running such a script. Anything can trivially disable the protection using a bat script (or anything else) to bootstrap.
E.g. `powershell.exe -ExecutionPolicy Unrestricted`
Installing / uninstalling / updating software should be a service provided by the OS. Letting vendors do it themselves just gives them an opportunity to mess it up, and they frequently do.
This makes me think of the recent changes to pip on Arch Linux that recommend installing modules with pacman.
Pacman is ok, but I recently destroyed the Ruby installation on a computer while trying to use Vagrant and other Ruby packages. I still haven't figured out how to fix it. Lesson learned, stick to AUR and pacman repo.
Virtualenvs are your friend. I use pipx for Python utils not in my OS repos, never had a problem with that. (`pipx install yt-dlp` creates a venv in ~/.local/pipx/venvs, installs yt-dlp there, then symlinks the executables to ~/.local/bin.)
For per-project dependencies, Poetry is pretty good, although there are other options too.
For Ruby, I think Bundler is the venv/Poetry counterpart. Not sure what you could use for installing global tools though.
I can't speak for whether Apple gets it right, but my experiences with the various package managers on Linux have not been any better than my experiences with installers on Windows. I've settled for avoiding system packages for anything I can build from source since system packages are always outdated and often Strange, and I try to avoid third party package sources and weird stuff like Snap or Flatpak since it's also historically been a source of problems for me. Maybe life is better outside of the Debian sphere though, since I've only dealt with Ubuntu and Debian.
I think part of it is that Debian and Ubuntu try to ship a good, usable default config, which is not necessarily the upstream default. I think it makes it hard to debug config issues.
I don't quite get this, I've been using Debian-esque Linux since like 1997 in various forms and have had problems with apt-get/apt maybe five times since then in total and it's always fixable with a little work. I've seen this a lot and I've haven't really understood the problems.
I saw Linus from LTT brick his installation (in Pop_OS! I think?) but that was a clear user error.
YMMV. I just bricked a Debian install last month by using the package manager to install display drivers then trying to uninstall it to try a different driver. Had to go editing files in the console on a 4k display and just cross my fingers.
A user error that can happen to any user who isn't Linux savvy and just wants to paly games, and not learn how a package manager works, and that it can uninstall your desktop environment if you aren't proficient with Linux, is no user error but OS error.
How many MacOS or Windows users expect that going through the installation steps of Steam, it can uninstall your desktop environment? For that user demographic, that is a clear OS issue.
There was a Steam package error, the error warned it might be temporary, the installer said it wouldn't continue because that would remove "popos-desktop" amongst other things.
So, he opened a console (like any user?), then used apt-get ... which had a WARNING ... "This should NOT be done unless you know exactly what you are doing!". He then had to type "Yes, do as I say!" in order to "do something potentially harmful" ... and then, what a surprise it did something harmful!
That's a user error that will only happen to people who are cocky, people who are idiots, or people trying to firm up their long held stance that 'Linux isn't for gaming'.
I saw the LTT video the following day to release, loaded a VM up, installed Pop_OS (my first time) and installed Steam, no issues. Very simple. All button clicking.
>going through the installation steps of Steam //
That's mis-characterisation, he went through the Steam install steps (click install by the Steam icon), the OS told him there was an error and refused to do it. End.
Then he went sudo-ing and ignoring warnings. "You can delete System32 this OS isn't ready for users!".
You'll tell me now installs never fail on Windows, presumably, despite having experienced them myself.
That would happen to any other user trying linux for the first time because not every user knows they should first update their packages before trying to install something because why doesn't the OS do it automatically like any other OS?
It's 100% an OS UX error you're trying to spin into an user error.
The problem isn't "apt" so much as "the system apt packages do weird things and as a result I can't build this open source package" or "I updated Ubuntu and now varnish and znc don't work even though I was using the system packages" or "the system apt package for mono is just plain broken and it conflicts with one I build from source, so I have to uninstall it"
mundane gripe: Uninstalling an apt package is too complicated and it is beyond me why it isn't a single command in 2023. I have to skim stackoverflow answers every time I need to do it (multiple times a year).
Windows provides various installer systems (.msi and the later iterations for UWP applications). It's unfortunate that MSI files aren't used more often, because they're more reliable than their uninstall.exe counterparts. They're not unlike .deb/.rpm files, except they integrate with the system GUI better when it comes to prompts and variables.
I believe MSI files were originally supposed to be almost atomic in use (software is either installed or uninstalled), providing transactional operations for installing software and limited UI options. I remember (but can't find) an old quote from someone on the team that designed the file format, basically saying something among the lines of "giving developers the option to execute random commands from MSI installers was the worst decision we've made".
historically all windows software has either been unzipped into a folder or installed using an installer created by the vendor, so as a result the vendor has to provide their own uninstaller too. uninstallers are complex enough that the OS can't completely take their place, though Windows has shipped with an install/uninstall framework called MSI for a long time.
What they never shipped is understandable docs and tools for devs to create .msi installers, which caused the deluge of custom installers such as the one being discussed.
Whilst winget looks like progress (presumably there'll be a non-CLI interface soon?) it seems like it's a tool for a limited set of applications that are allowed into a Microsoft list, not even applicable to .msi[x] in general and definitely not a way to uninstall (nor even just list) files installed by applications in general.
Ah that hadn't clicked for me until your example. So basically it's something 'popular' but doesn't mean it's the right way to do it, it's just abusing a capability.
...or maybe Windows should just offer an API for marking a file for deletion once it's not in use anymore (I understand unlink semantics may not be possible, but that's not what my suggestion above is saying)
> I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour?
How do you defragment/move files that are unreachable on the file system? How do you shrink volumes when you can't move files that need to be moved?
Edit: Actually, hmm... as I type this, I suddenly recall you can also open a file by its ID on NTFS. And you can enumerate its streams as well. So presumably this could work on NTFS if you loop through all file IDs? Though then that would make these files still accessible, not truly unreachable.
Huh? That API requires a file handle. Which you get by opening a file. Which you can't do because you can't find it on the filesystem when it's not there.
You're suggesting opening every single FD of every single process (which might not even point to a file, let alone a file on that volume) and querying it just to do this? I mean, sure, I guess that's usually not physically impossible (unless e.g. /proc is unavailable/unmounted)... but it's clearly a hack.
In fact, I think it's not just a (slow!) hack, but a buggy one too. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
No I'm just saying it's possible. I can count on the fingers of 0 hands the number of times I've needed to do this to edit a deleted file out from under a process that has the only reference to an unlinked file open so at least in my experience it's merely acadamic knowledge!
You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later. Worst case you defragment on boot.
> You don't? Those are either free space, or held by handle by a running process, so you just leave them be and assume they will be released sooner or later.
Well that's what I was getting at, it would suck to not be able to move around file blocks just because a process is using the file. That "sooner or later" might well be "until the next reboot". The current strategy makes it possible to live-shrink and live-defragment volumes on Windows - ironically, saving you a reboot in those cases compared to Linux.
But actually, maybe not - see the edit in my original comment.
I'm yet to want to defrag my computer and worrying about still open deleted files.
I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often, and annoys me more. (or being unable to replace a running service binary of a service being developed/tested, needing to stop the service, replace it, and start again. Or failing log rotations because a logfile is open in a Notepad. Or...)
Also see my link for a solution on unix, where you can indeed fix this problem, or simply kill the process holding the file. I didn't need to defrag my computer in the last 20 years, neither on Linux, nor on Windows, but hey, it makes me happy that my daily work is hindered for this hypothetical possibility. (which could and is solved in other OSs with appropriate APIs for the job)
Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
I wasn't just talking about defragging. I was also talking about live volume shrinking.
> Also see my link for a solution on unix, where you can indeed fix this problem
Looping through every FD of every process just to find ones that reside in your volume of interest is... a hack. From the user's perspective, sure, it might work when you don't have something better. From the vendor's perspective, it's not the kind of solution you design for & tell people to use.
In fact, I think that "solution" is buggy. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
> or simply kill the process holding the file.
That process might be a long-running process you want to keep running, or a system process. At that point you might as well not support live volume shrinking or defrags, and just tell people to reboot.
> Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
This seems pretty irrelevant to the point? It's not like they would design the kernel to say "we'll let you do this if you promise you're an installer".
> I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often [...]
Yes, I agree it's frustrating. But have you considered the UX issues here? The user has C:\blah\foo.txt open, and you delete C:\blah\. The user saves foo.txt happily, then reopens it and... their data is gone? You: "Yeah, because I deleted it." User: "Wait but I was still using it??!"
I have considered it. Never had any serious problem about it during 15 years of desktop linux use as a developer machine. Grandma would not have more problems than unplugging the pendrive where the file was opened from, and trying to save it, for example... Modern operating systems have far worse and more user hostile patterns.
And for the live volume shrinking: the kernel can solve this problem, it there is a need for this, there is no need for this invariant for this feature, as it is not only possible to do it via the same APIs offered for ordinary basic file manipulation gruntwork. On unix basically a filename is disassociated from the inode, but afaik the inode holding the blocklist still exists, will be cleaned up later, thus it can be updated if its blocks are moved under the high level filesystem APIs.
> Never had any serious problem about it during 15 years of desktop linux use as a developer machine.
You're not the typical customer of Windows.
> Grandma would not have more problems than unplugging the pendrive where the file was opened from, and trying to save it, for example
Actually she would, because in that case writing to the same file handle would error, not happily write into the ether.
Also, you have one tech-savvy grandma. I don't think mine even knows what a "pendrive" is (though she's seen one), let alone try to open a file on one, let alone try to save her files on it, let alone use pen drives on any regular basis.
> You just made a strawman you are sticking to.
The only strawman I see here is your grandma using pen drives to save files.
What I'm pointing at are real issues for some people or in some situations. Some of them you might be able to solve differently at a higher investment/cost, or with hacks. Some of them (like the UX issue) are just trade-offs that don't automatically make sense for every other user just because they make sense for you. Right now Windows supports some things Linux doesn't, and vice-versa. Could they be doing something better? Perhaps with more effort maybe they could both support a common superset of what they support, but it's not without costs.
it's a little more general, I think, since one common use case for it is to use it on your own process in order to intercept calls to stdlib/OS code from libraries you don't control.
For example, in the bad old days I used detours to virtualize the windows registry so that I could do "fake installs" of COM components inside of a VB6 app, allowing it to run without an install and without administrator permissions. This worked by detouring all the Win32 registry APIs, which was sufficient to intercept registry accesses performed by the COM infrastructure.
Detours is a library for instrumenting arbitrary Win32 functions Windows-compatible processors. Detours intercepts Win32 functions by re-writing the in-memory code for target functions. The Detours package also contains utilities to attach arbitrary DLLs and data segments (called payloads) to any Win32 binary.
Detours preserves the un-instrumented target function (callable through a trampoline) as a subroutine for use by the instrumentation. Our trampoline design enables a large class of innovative extensions to existing binary software.
> it's a little more general, I think, since one common use case for it is to use it on your own process in order to intercept calls to stdlib/OS code from libraries you don't control.
This capability is intrinsic to how ELF linking works. The main application or even any library can interpose a libc function just by defining and exporting a function with the same name, and that definition will be preferentially linked in both the main application and all subsequently loaded dynamic libraries and modules. Your definition can then use dlsym(RTLD_NEXT, "foo") to obtain a function pointer to the next definition, which would normally be libc itself but may be from another library. A running application could actually have several implementations of a function, all proxying calls onward until the terminal (usually libc) implementation.
Basically, the way ELF linking works bydefault is that the first definition loaded is the preferred global symbol used to satisfy any symbol dependency with that name. It follows that there's normally a singular global symbol table. Though there are features and extensions that can be explicitly used to get different behaviors.
There's nothing magical about LD_PRELOAD within the context of ELF linking. LD_PRELOAD support in the linker (which is the first bit of code the kernel loads on exec(2)) is very simple; all the linker does is load the specified libraries first, even before the main application, so symbols exported therein become the initial and therefore default definition for satisfying subsequent symbol dependencies, including in the main application binary, and even if the main application binary also defines and exports those symbols.
All of this is basically the exact opposite behavior of how PE linking works on Windows, for better and worse--depending on your disposition and problems at hand.
Also note that all of this is different than so-called "weak" symbols, which is a mechanism for achieving one of the same behaviors--overriding another definition--when statically linking. Otherwise, when statically linking, multiple definitions are either an error or it's difficult (i.e. confusing, especially in complex builds) to control when and where one definition is chosen over another.
[1] Though main application symbols aren't usually exported by default, so you need to explicitly mark a definition for export or build the entire main binary with a compiler flag like `-rdynamic`, which is the main binary analog to the `-shared` flag used for building shared libraries. The Python and Perl interpreters, for example, are built with -rdynamic as the interpreter binary itself exports all the implementation symbols required by binary modules, rather than defining them in a separate shared library against which modules explicitly link themselves against at compile time. (This is also why when building Perl, Python, and similar language modules you have to tell the compile-time linker to ignore unresolved symbols.)
> Whether they follow the licensing terms for that code I do not know.
I'm guessing they didn't ship the binary with a link pointing back to this page?
These's also another codeproject example that uses a bat file, which is fairly similar to the recommendation in the post. I guess that's the better example.
At least the author seems to agree with Raymond Chan on the similarities between his approach and malware...
> shellcode is the technical term (in security circles) for binary machine code that is typically used in exploits as the payload. Here's a quick and dirty way of generating the shellcode from the obj file generated when you compile your source files. In our case, we are interested in whipping the shellcode up for the remote_thread routine. Here's what you've got to do:
The whole article has the vibes of some questionable DIY blog along the lines of "Your house is infested by vermin? Here is an easy way to get rid of them using a small, homebuilt neutron bomb!"
Funnily enough, real malware does this correctly. Usually by just ShellExecing "ping -n 3 127.0.0.1 >nul" followed by "del" - no temporary file needed.
Snark aside, when it comes to nix boxes, I've personally basically given up - for all of the "system" software I trust apt or other package managers, whereas in the case of any "services" I want to run (mail servers, web servers, backup services, databases, APIs) everything is run in containers with custom bind mount directories, to not pollute the host file system with crap that might get left over when removing a service or putting it on another node.
So something like "/var/lib/postgresql/data" in the container becomes something like "/app/my-postgres-service-12/var/lib/postgresql/data" on the host.
I have just one directory to backup, I can also move it to different nodes entirely and OS upgrades don't break anything either due to the running software being decoupled from the OS somewhat.
But my Linux and Windows desktops? It's absolute Wild West over there and I just reinstall the entire OS every few years - they're beyond saving.
Uhm, for uninstallers? How about Windows Installer?
If you mean in other contexts... I think the point is you're not intended to be able to do this? Outside of uninstallers, running code that only exists in RAM is... the type of thing malware typically wants to do more than anything else.
But in terms of what's physically possible, I suppose there's the command prompt, PowerShell, and scheduled tasks? I'm not sure if all of those can be disabled.
The latter is not "javascript" in general, it's just a Node.js (a specific v8 Javascript runtime + native integration and libs like fs bundle) API.
And the former windows version is JScript which is an implementation Javascript just using an different name for trademark reasons, but tied to an old version of the JavaScript standard (which is confusingly called "ecmascript" officially), plus some Windows-specific integrations, like ActiveXObject and COM/OLE support.
(There's also a later .NET version of JScript to add to the confusion).
Yes and no. Windows has a very fine grained permissions system, including at the admin level. The problem is that it was designed for multi-user systems in the 90s, so the permission and security systems are mostly concerned about keeping users safe from each other, and having administrator roles for managing those users and doing system-wide tasks. Preventing a process from injecting code into another process by the same user running in the same session just isn't in the original threat model, since it's just the user screwing with their own stuff.
The shift towards protections from malware happened mostly as a consequence of Windows XP. There are now better controls, like assigning low-trust processes like a browser's renderer a low integrity level to prevent them from doing that. But it's also late enough that it's hard to rock the boat too much without breaking existing applications. Microsoft tried to make a clean break and offer more sandboxed applications with a user-friendly package manager (called the Microsoft Store) but this wasn't well received by app developers: most didn't use it at all, and those that did often opted out of the sandboxing.
And this has been the case since Windows NT in 1993 according to a presentation I saw from Sami Laiho where he strongly argues that you don't need and should be a admin account as default.
From microsoft's documentation [1]
> Administrator-to-kernel is not a security boundary.
Everything is allowed to do that. You're right that it's not good security-wise which is why Apple blocked that sort of thing years ago. On Windows unfortunately the whole Win32 ecosystem is very dependent on programs injecting things into other processes, the API makes it quite easy and there's lots of sample code for it. It's a major source of stability and crash bugs there.
For example, antivirus products do this all the time, as do many video drivers and other system utilities.
Also, Explorer has various plugin interfaces where it'll load third party code and run it in-process since the very first version.
Uninstaller likely running with admin privileges or as system. Despite it's huge surface area, explorer is not a super privileged binary on windows and is essentially a userspace program that has tons of open ports to plug in for many many apps. Locking it down like this would likely cause a lot of apps to break. I know for a fact microsoft keeps very close eyes on crash reports of their core apps such as explorer, and if your app causes a fault in explorer and you're registered on windows developer portal you'll get notified of this when they roll out beta insider preview releases.
I can only imagine the Win32 API team meeting prior to this...
A: So, people are resorting to injecting code in Explorer to delete in-use files in such numbers that it shows up in our top-100 crash report reasons
B: Well, maybe we should add a public API to Windows to support this incredibly common functionality that apparently has been missing so far?
A: Nah, let's just write a mildly condescending blog post that recommends using an unreliable workaround that is pretty much guaranteed to trigger any client-side intrusion detection software, that will set them straight
B: Right on!
(Meanwhile, somewhere, a third-party developer is gearing up to ship a kernel-mode driver to directly manipulate file system structures from their uninstaller, since their old solution kept crashing: can't wait to read the postmortem once the crash dumps from that one hit the Microsoft servers!)
Well, they kinda had to rush that meeting because they spent too much time on the previous meeting listing all the reasons why the microsoft store is a functioning package system and not at all a din of inequity.
> There are already many services and APIs for doing this
I... don't think so? The particular problem here, is that an uninstaller executable needs to delete itself from disk after doing its main job.
Other than using MoveFileEx with a NULL destination file and the MOVEFILE_DELAY_UNTIL_REBOOT flag, then suggesting/forcing a reboot, I can't think of a straightforward solution.
And that solution instantly lights up your JIRA with 2 tickets:
#1: We MUST not suggest/force a reboot! Users hate it!
#2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
So, then you try things like 'create a Task Scheduler job to delete the file', which then adds:
#3: PRIO 1: uninstaller.exe crashes if Windows Task Scheduler disabled
#4: SHOWSTOPPER: uninstaller.exe still not always deleted after uninstall. Why is this so hard?
Et cetera, ad absurdum. This then escalates to code injection (as described in the linked article), and (you heard it here first) kernel-mode drivers. So, if you're aware of a reliable solution, feel free to share in a comment here, for the betterment of the world!
>Other than using MoveFileEx with a NULL destination file and the MOVEFILE_DELAY_UNTIL_REBOOT flag, then suggesting/forcing a reboot, I can't think of a straightforward solution.
One of the comments on the article gives one example:
> When our product’s uninstaller sees an undeletable file (possibly a DLL loaded in another process), it uses the MOVEFILE_DELAY_UNTIL_REBOOT flag to mark it for deletion, and warns the user “Please reboot as soon as possible to remove the remaining files.”
> However, some user uninstall our product, just to be able to reinstall it later, to the same location. And of course they ignored the warning. Once they reinstalled it, everything works, until a reboot.
That solution instantly lights up your JIRA with 2 tickets:
#1: We MUST not suggest/force a reboot! Users hate it!
#2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
This. The deep dive is fascinating, but really feels like a distraction from a product management failure. Microsoft has been 3 decades in the OS business. Surely someone must have noticed that package management should be a core OS feature.
At this point the Windows package manager omnimisery has metastasized and now idiot developers are insisting on writing their own installers even for platforms where it's totally un-necessary (harmful in fact!)
Please don't inflict more Microsoft package managers on us. All that needs fixing is there should be a supported way to delete "in-use" files just like you always could on Linux. People may make theoretical arguments about the virtues of locking files but the Linux behavior is so clearly the right thing in practice. Over the years I've encountered approximately zero problems with that behavior on Linux and dozens of problems with the Windows behavior, both as a user and as a developer.
I'm sure it would be hard to add such a thing to Windows while minimizing compatibility problems but I'm equally sure that it's possible and that it's worth the effort.
Raymond Chen writes great stuff but gives a very one-sided picture of compatibility -- he doesn't mention all of the times that it was the other way around, with Windows doing something lame and application authors having to work around it.
Like, for instance, the time that they decided that LaunchAdvancedAssociationUI(), the previous officially recommended way to show UI to allow the user to associate file types with a program, just wouldn't work anymore in Windows 10. Instead of opening up the Default Programs UI in Settings, it just displays a dialog telling the _user_ to go there -- which is even modal so they can't even refer to it while doing so. No compatibility shim or grandfathering for old programs, they just broke all applications that used this like they originally said good programs should do for Windows 8.
Or the case of Dark Mode in Windows, which for some reason they've dragged their heels on implementing barely any Win32 support at all for -- even just simply a call to query whether it is enabled. The current silly recommendation is to obtain the foreground color through WinRT and do a dot product on it to compute luma and determine if it is a dark or light color:
https://learn.microsoft.com/en-us/windows/apps/desktop/moder...
Or the fact that the official way of reporting bugs on the Windows APIs is the Feedback Hub, which is completely unsuitable for task.
I don't have sympathy for the Windows team anymore. Their lack of developer support is partially responsible for all of the hacks that applications have to do to ship.
I keep forgetting Windows-isms that won't allow you to delete the executable file of a running process. I guess that's also why the arcane .dll upgrade process / WoW is so necessary.
Any file that was opened without specifying FILE_SHARE_DELETE in the call to CreateFile[1] (the Win32 equivalent of open(2)). Unfortunately, most language runtimes that wrap CreateFile tend not to pass that flag.
Apparently some parts of this are quite recent, huh[1]:
> jstarks commented on Jun 18, 2019:
> [I]n the most recent version of Windows, we updated DeleteFile (on NTFS) to perform a "POSIX" delete, where the file is removed from the namespace immediately instead of waiting for all open handles to the file to be closed.
While full Unix-like behaviour is only available on Windows 10 for the past five or so years, you can still have the old win32 behaviour on older systems (delete once the last file handle is closed).
But there must be an API to unlock files that programs like "The Unlocker" use? Or do they just enumerate the other processes handles to that file and close them?
Their Teams uninstaller isn't quite as good - it just doesn't uninstall all the crap that the Teams malware has left behind. I still have a stray MS Teams audio device left on my macOS machine.
I actually believe in filing as few bug reports to Microsoft as possible, that way they don't have any signals about what to research. Hopefully it'll waste some amount of their resources, no matter how few.
551 comments
[ 3.6 ms ] story [ 382 ms ] threadI remember very distinctly this quote about him:
> The only person in the world who leapt to my defense was, of course, Raymond Chen, who is, by the way, the best programmer in the world, so that has to say something, right?
So in my mind I've made the connection that Raymond Chen = best programmer in the world since then haha.
I think Windows 98 already had this ability. Possibly Windows 95 as well. It's a variant of the language called JScript, which is what was used in old versions of IE too.
WSH btw allowed you to run any language that you had interpreter for - they had to support necessary COM interfaces (and to be truly usable, allow you to call COM objects), and register their interpreter class with ActiveScripting (WSH internal) engine.
Then you could use them not just for desktop automation, but also for scripts inside Internet Explorer (essentially, classic IE used WSH engines to implement scripts, iirc)
I've seen WSH (including HTAs) used with Perl, Python, Tcl, Rexx...so long as you install the interpreter with compatible COM service, you could use it.
By the way, there are also HTAs, which are Microsoft HTML Applications. You can create a simple double-clickable GUI with these using only HTML and JScript.
Apparently XULRunner was first released in 2006, but Thunderbird, which uses (used?) the same technology, was released as early as 2003, and maybe this was existing in the Mozilla Suite even before.
Also, I think one of the Start Menus (might have been XP‽) was kind of HTA-ish? Not sure about that part, though.
> Think of it as they had a fork of Firefox
Yep indeed, you are right.
Notable projects using actual XULRunner included Songbird (a music player) and BlueGriffon, an WYSIWYG HTML editor (a successor of Nvu and KompoZer, themselves succeeding Netscape Composer). Both released after 2006 indeed.
I liked XUL, I strongly believe Mozilla could have dominated the market taken by Electron, had they pushed XULRunner more, and perhaps make it transition to pure HTML, like they did to Firefox's core, because that's what people know and because XUL was a maintenance burden. I think XUL tags made more sense than HTML to build UIs, though, and with XUL, Gecko have had a CSS flex-like mechanism for a long time by the way.
[1] https://en.wikipedia.org/wiki/Songbird_%28software%29
[2] https://en.wikipedia.org/wiki/BlueGriffon
1) You could write event handlers in multiple languages, including C. If you wrote them in C, it spawned gcc and compiled it into a library, and dynamically loaded it... The overall idea of a polyglot runtime like that was fun.
2) #1 is only really weird because this could be done at runtime. One of the demo apps was an editor for the GUI itself, where you could add buttons to the editor, then write that event handler in C, and have it compiled and loaded into the editor itself...
It was a fascinating starting point, though full of heavy duty foot guns, and I'm still sad nobody took it further.
Active Scripting, which powers scripts in both WSH and old-school IE including HTAs, is polyglot and extensible. It’s why Active{Perl,Python,Tcl} are called that—the original headline feature (IIUC) was that they integrated with it. It’s also why you could write VBS in IE: IE just passed the text of the script along with the language attribute to AS and let it sort things out.
Nobody did ever a C interprerer, though, I think—perhaps because you basically have to speak COM from Active Scripting, and while speaking COM from C is certainly possible it’s nobody’s idea of fun. (An ObjC-like preprocessor/superset could definitely be made and I’ve heard that Microsoft had even entertained the idea at the dawn of time, but instead they went with C++, and I haven’t been able to find any traces of that project.)
That’s not to say AS is perfect or even good—the impossibility of caching DISPIDs[1], in particular, seems like a design-sinking goof. And the AS boundary was also why DOM manipulation in IE was so slow.
[1] https://ericlippert.com/2003/09/16/why-do-the-script-engines...
Wondering what would it take to port mshta (with all the ActiveX goodies) to other platforms. Maybe it's a little bit late for that, but sounds like it might be a fun project to me.
https://web.dev/progressive-web-apps/
Wine Gecko supports ActiveX, supposedly, so if someone implements all the common ActiveX components, that could be a cross-platform method of running HTAs outside of Windows.
That said, I'm afraid the Electron API is the closest thing we have to a cross platform HTML application these days. On Manjaro, several packages are already implemented by installing Electron next to the application specific code, so that would be the closest thing to a modern HTA alternative that I know of.
PWAs work fine if you don't need integration with the system itself other than file prompts, for chat apps for example. They're not really alternatives to HTAs to be honest.
It should be noted that HTAs are a common way to infect computers (because they're executables that aren't usually recognised as such) and they're disabled in many security conscious environments.
Seemingly_random_code(seemingly_random_string)
The seemingly_random_code decompresses/decodes whatever is in the seemingly_random_string and hands over control to it. Interestingly the decoded code is another version of the same with different code and string. This goes on for ~100 layers deep then at the end it just downloads and executes some file from the net.
I understand doing one layer. I guess I could maybe see two layers. But why would it bother with 100 layers? Either the antivirus or reverse-engineering tool can grab the final product or it can't.
This is similar to how compilers will often have recursion limits for things like generics, though in that case it's easier to reject the program if the recursion limit is reached.
In very strict environments or certain systems it might be practical to block all unknown files, but this is uncommon for user systems for example where users are actively using javascript or macro documents etc. (developers, HR, finance etc.) The FP rates are too high and productivity can take a big hit. If all users do 20% less work that's a big loss in revenue (the productivity hit can be much more severe even!). perhaps this impact / loss of revenue ends up being bigger than a malware being executed depending on the rest of the security posture/measures.
technically its possible to identify (nearly?) all malware by tracking p-states/symbolic execution/very clever sanboxing etc.. but this simply takes much too long. Especially if the malware authors are aware of sandboxing techniques and symbolic execution and such things as they can make those processes also take extra long or sometimes even evade them totally with further techniques.
I wish it _was_ possible to do all of the cleverness that malware researchers have invented to detect things, but unfortunately, in practice this cannot happen on like 90+% of environments.
If you run like a DNS server or such things, it's possible to do it as such a system would not be expected (ever?) to have unknown files. (gotta test each update and analyze new versions before deploying to prod). As you can imagine, this is also kind of a bummer process but imho for such 'static' systems its worth it.
Personal promotion, I built this as a better alternative:
https://github.com/technion/open_safety
Note the built in .js parser hasn't basically ever updated, if you're writing for this you're writing like you're targetting IE5.
Huh, neat!
like, really? can't you write that in C? I don't think most Win32 apps use JavaScript for their installers.
The point of the exercise is that, on Windows, you can’t, because Windows won’t let anyone delete executables that are currently in use (try it, you won’t be able to delete one either). Upgrading shared DLLs in the face of this fact is why installers for Windows programs often have to have you reboot the system (and in more civilized times asked you to close other programs before installation to reduce the probability of hitting a locked DLL). It’s also why there’s a registry key[1] containing a list of rename and delete actions to be performed on next reboot (usually accessed via the MOVEFILE_DELAY_UNTIL_REBOOT flag to MoveFileEx).
You can’t (straightforwardly[2]) make a self-deleting batch script, either, because the command interpreter parses a command at a time and so wants the batch file to exist. The Windows Scripting Host, on the other hand, will parse the whole file at once, close it, and then forget about it, so you can write self-deleting WSH scripts.
The workaround used by the uninstaller under discussion is instead for the executable to inject some code into the Windows Explorer (on the assumption that it’s always running and the user has to have access permissions for it) that accomplishes the deletion through return-oriented programming, so that the stack it’s executing from can then disappear into the wind (apparently? I’m not seeing how they plan to clean that up).
On a POSIX system you are explicitly allowed to delete any open file—including an executing one—making it languish in a kind of system-managed limbo (and take up disk space, invisibly) until it’s closed. The tradeoff is then that it’s impossible to ensure you’ve opened the same file as somebody else when all you have is its name. (I think you can at least check for success, provided you also have the device and inode numbers for it.)
[1] https://superuser.com/questions/58479/is-there-a-registry-ke...
[2] https://stackoverflow.com/questions/20329355/how-to-make-a-b...
E.g. `powershell.exe -ExecutionPolicy Unrestricted`
My concern is more than Raymond Chen suggest that using it is still the recommended way. So much malware came through WScript.
Pacman is ok, but I recently destroyed the Ruby installation on a computer while trying to use Vagrant and other Ruby packages. I still haven't figured out how to fix it. Lesson learned, stick to AUR and pacman repo.
For per-project dependencies, Poetry is pretty good, although there are other options too.
For Ruby, I think Bundler is the venv/Poetry counterpart. Not sure what you could use for installing global tools though.
Windows does have installer/uninstaller infrastructure called MSI (https://learn.microsoft.com/en-us/windows/win32/msi/windows-...), but ultimately it's up to developers to choose to use it.
I saw Linus from LTT brick his installation (in Pop_OS! I think?) but that was a clear user error.
A user error that can happen to any user who isn't Linux savvy and just wants to paly games, and not learn how a package manager works, and that it can uninstall your desktop environment if you aren't proficient with Linux, is no user error but OS error.
How many MacOS or Windows users expect that going through the installation steps of Steam, it can uninstall your desktop environment? For that user demographic, that is a clear OS issue.
So, he opened a console (like any user?), then used apt-get ... which had a WARNING ... "This should NOT be done unless you know exactly what you are doing!". He then had to type "Yes, do as I say!" in order to "do something potentially harmful" ... and then, what a surprise it did something harmful!
That's a user error that will only happen to people who are cocky, people who are idiots, or people trying to firm up their long held stance that 'Linux isn't for gaming'.
I saw the LTT video the following day to release, loaded a VM up, installed Pop_OS (my first time) and installed Steam, no issues. Very simple. All button clicking.
>going through the installation steps of Steam //
That's mis-characterisation, he went through the Steam install steps (click install by the Steam icon), the OS told him there was an error and refused to do it. End.
Then he went sudo-ing and ignoring warnings. "You can delete System32 this OS isn't ready for users!".
You'll tell me now installs never fail on Windows, presumably, despite having experienced them myself.
It's 100% an OS UX error you're trying to spin into an user error.
mundane gripe: Uninstalling an apt package is too complicated and it is beyond me why it isn't a single command in 2023. I have to skim stackoverflow answers every time I need to do it (multiple times a year).
I believe MSI files were originally supposed to be almost atomic in use (software is either installed or uninstalled), providing transactional operations for installing software and limited UI options. I remember (but can't find) an old quote from someone on the team that designed the file format, basically saying something among the lines of "giving developers the option to execute random commands from MSI installers was the worst decision we've made".
I used to bundle an INF file in a CAB archive (converted to self-extract executable) to distribute software. Using only Microsoft's own tools.
It doesn’t rely on it. It’s just something that’s possible.
Where do I look in the OS for the manifest of all installed files from an installer?
Thanks, I'm mostly a Linux user and I've sorely missed a `dpkg -L` on MS Windows for ages.
https://learn.microsoft.com/en-us/windows/package-manager/
I never digged into the question, but why is it used, what benefits did it provide over the UNIX unlink behaviour?
How do you defragment/move files that are unreachable on the file system? How do you shrink volumes when you can't move files that need to be moved?
Edit: Actually, hmm... as I type this, I suddenly recall you can also open a file by its ID on NTFS. And you can enumerate its streams as well. So presumably this could work on NTFS if you loop through all file IDs? Though then that would make these files still accessible, not truly unreachable.
Edit: Actually, hmm... see edit above.
In fact, I think it's not just a (slow!) hack, but a buggy one too. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
https://unix.stackexchange.com/questions/68523/find-and-remo...
This is how it works on UNIX. Generally better then apps randomly failing because a file(name) is held open somewhere by something.
Well that's what I was getting at, it would suck to not be able to move around file blocks just because a process is using the file. That "sooner or later" might well be "until the next reboot". The current strategy makes it possible to live-shrink and live-defragment volumes on Windows - ironically, saving you a reboot in those cases compared to Linux.
But actually, maybe not - see the edit in my original comment.
I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often, and annoys me more. (or being unable to replace a running service binary of a service being developed/tested, needing to stop the service, replace it, and start again. Or failing log rotations because a logfile is open in a Notepad. Or...)
Also see my link for a solution on unix, where you can indeed fix this problem, or simply kill the process holding the file. I didn't need to defrag my computer in the last 20 years, neither on Linux, nor on Windows, but hey, it makes me happy that my daily work is hindered for this hypothetical possibility. (which could and is solved in other OSs with appropriate APIs for the job)
Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
> Also see my link for a solution on unix, where you can indeed fix this problem
Looping through every FD of every process just to find ones that reside in your volume of interest is... a hack. From the user's perspective, sure, it might work when you don't have something better. From the vendor's perspective, it's not the kind of solution you design for & tell people to use.
In fact, I think that "solution" is buggy. Every time you open a an object that doesn't belong to you, you extend its lifetime. I think that can break stuff. Like imagine you open a socket in some server, then the server closes it. Then that server (or another one) starts up again and tries to bind to the same port. But you're still holding it open, so now it can't do that, and it errors out.
> or simply kill the process holding the file.
That process might be a long-running process you want to keep running, or a system process. At that point you might as well not support live volume shrinking or defrags, and just tell people to reboot.
> Also the original post is about Windows installers... don't get me started on the topic (or windows services), please.
This seems pretty irrelevant to the point? It's not like they would design the kernel to say "we'll let you do this if you promise you're an installer".
> I face builds failing because I have a terminal open in a build output directory or a textfile in an editor open is far more often [...]
Yes, I agree it's frustrating. But have you considered the UX issues here? The user has C:\blah\foo.txt open, and you delete C:\blah\. The user saves foo.txt happily, then reopens it and... their data is gone? You: "Yeah, because I deleted it." User: "Wait but I was still using it??!"
And for the live volume shrinking: the kernel can solve this problem, it there is a need for this, there is no need for this invariant for this feature, as it is not only possible to do it via the same APIs offered for ordinary basic file manipulation gruntwork. On unix basically a filename is disassociated from the inode, but afaik the inode holding the blocklist still exists, will be cleaned up later, thus it can be updated if its blocks are moved under the high level filesystem APIs.
You just made a strawman you are sticking to.
You're not the typical customer of Windows.
> Grandma would not have more problems than unplugging the pendrive where the file was opened from, and trying to save it, for example
Actually she would, because in that case writing to the same file handle would error, not happily write into the ether.
Also, you have one tech-savvy grandma. I don't think mine even knows what a "pendrive" is (though she's seen one), let alone try to open a file on one, let alone try to save her files on it, let alone use pen drives on any regular basis.
> You just made a strawman you are sticking to.
The only strawman I see here is your grandma using pen drives to save files.
What I'm pointing at are real issues for some people or in some situations. Some of them you might be able to solve differently at a higher investment/cost, or with hacks. Some of them (like the UX issue) are just trade-offs that don't automatically make sense for every other user just because they make sense for you. Right now Windows supports some things Linux doesn't, and vice-versa. Could they be doing something better? Perhaps with more effort maybe they could both support a common superset of what they support, but it's not without costs.
And with it, less data corruption issues.
For example, in the bad old days I used detours to virtualize the windows registry so that I could do "fake installs" of COM components inside of a VB6 app, allowing it to run without an install and without administrator permissions. This worked by detouring all the Win32 registry APIs, which was sufficient to intercept registry accesses performed by the COM infrastructure.
Detours preserves the un-instrumented target function (callable through a trampoline) as a subroutine for use by the instrumentation. Our trampoline design enables a large class of innovative extensions to existing binary software.
https://www.microsoft.com/en-us/research/project/detours/
[0] https://github.com/Zeex/subhook
https://easyhook.github.io/
https://easyhook.github.io/#features
https://github.com/EasyHook/EasyHook
I’ve created a PowerShell module that wraps this library to make it easier to hook functions on the fly for testing https://github.com/jborean93/PSDetour. For example I used it to capture TLS session data for decryption https://gist.github.com/jborean93/6c1f1b3130f2675f1618da5663... as well as create an strace like functionality for various Win32 APIs (still expanding as I find more use cases) https://github.com/jborean93/PSDetour-Hooks
Yes please. Thank you for this
This capability is intrinsic to how ELF linking works. The main application or even any library can interpose a libc function just by defining and exporting a function with the same name, and that definition will be preferentially linked in both the main application and all subsequently loaded dynamic libraries and modules. Your definition can then use dlsym(RTLD_NEXT, "foo") to obtain a function pointer to the next definition, which would normally be libc itself but may be from another library. A running application could actually have several implementations of a function, all proxying calls onward until the terminal (usually libc) implementation.
Basically, the way ELF linking works by default is that the first definition loaded is the preferred global symbol used to satisfy any symbol dependency with that name. It follows that there's normally a singular global symbol table. Though there are features and extensions that can be explicitly used to get different behaviors.
There's nothing magical about LD_PRELOAD within the context of ELF linking. LD_PRELOAD support in the linker (which is the first bit of code the kernel loads on exec(2)) is very simple; all the linker does is load the specified libraries first, even before the main application, so symbols exported therein become the initial and therefore default definition for satisfying subsequent symbol dependencies, including in the main application binary, and even if the main application binary also defines and exports those symbols.
All of this is basically the exact opposite behavior of how PE linking works on Windows, for better and worse--depending on your disposition and problems at hand.
Also note that all of this is different than so-called "weak" symbols, which is a mechanism for achieving one of the same behaviors--overriding another definition--when statically linking. Otherwise, when statically linking, multiple definitions are either an error or it's difficult (i.e. confusing, especially in complex builds) to control when and where one definition is chosen over another.
[1] Though main application symbols aren't usually exported by default, so you need to explicitly mark a definition for export or build the entire main binary with a compiler flag like `-rdynamic`, which is the main binary analog to the `-shared` flag used for building shared libraries. The Python and Perl interpreters, for example, are built with -rdynamic as the interpreter binary itself exports all the implementation symbols required by binary modules, rather than defining them in a separate shared library against which modules explicitly link themselves against at compile time. (This is also why when building Perl, Python, and similar language modules you have to tell the compile-time linker to ignore unresolved symbols.)
but honestly, please Windows Dev's.. use MSI's please
it make me love you and i have so much love to give.
Alright cowboy, so where are you storing preferences and settings?
https://www.codeproject.com/Articles/17052/Self-Deleting-Exe...
> Whether they follow the licensing terms for that code I do not know.
I'm guessing they didn't ship the binary with a link pointing back to this page?
These's also another codeproject example that uses a bat file, which is fairly similar to the recommendation in the post. I guess that's the better example.
https://www.codeproject.com/Articles/4027/Writing-a-self-des...
> shellcode is the technical term (in security circles) for binary machine code that is typically used in exploits as the payload. Here's a quick and dirty way of generating the shellcode from the obj file generated when you compile your source files. In our case, we are interested in whipping the shellcode up for the remote_thread routine. Here's what you've got to do:
The whole article has the vibes of some questionable DIY blog along the lines of "Your house is infested by vermin? Here is an easy way to get rid of them using a small, homebuilt neutron bomb!"
Imagine having a package system, instead of that crap.
Imagine a file explorer that doesn't simply prompt for admin rights when you need them, instead silently failing.
Imagine using a file explorer you don't like.
So something like "/var/lib/postgresql/data" in the container becomes something like "/app/my-postgres-service-12/var/lib/postgresql/data" on the host.
I have just one directory to backup, I can also move it to different nodes entirely and OS upgrades don't break anything either due to the running software being decoupled from the OS somewhat.
But my Linux and Windows desktops? It's absolute Wild West over there and I just reinstall the entire OS every few years - they're beyond saving.
If you mean in other contexts... I think the point is you're not intended to be able to do this? Outside of uninstallers, running code that only exists in RAM is... the type of thing malware typically wants to do more than anything else.
But in terms of what's physically possible, I suppose there's the command prompt, PowerShell, and scheduled tasks? I'm not sure if all of those can be disabled.
(It's weird that this comment was upvoted twice, now down voted twice)
This is the quote I'm referring to: https://en.m.wikipedia.org/wiki/Clarke%27s_three_laws
It's the third of his 3 laws
And the former windows version is JScript which is an implementation Javascript just using an different name for trademark reasons, but tied to an old version of the JavaScript standard (which is confusingly called "ecmascript" officially), plus some Windows-specific integrations, like ActiveXObject and COM/OLE support.
(There's also a later .NET version of JScript to add to the confusion).
The OS isn't to blame when people give root access left and right.
Actually this is why macOS got SIP.
The shift towards protections from malware happened mostly as a consequence of Windows XP. There are now better controls, like assigning low-trust processes like a browser's renderer a low integrity level to prevent them from doing that. But it's also late enough that it's hard to rock the boat too much without breaking existing applications. Microsoft tried to make a clean break and offer more sandboxed applications with a user-friendly package manager (called the Microsoft Store) but this wasn't well received by app developers: most didn't use it at all, and those that did often opted out of the sandboxing.
From microsoft's documentation [1] > Administrator-to-kernel is not a security boundary.
I recommend the talk https://www.youtube.com/watch?v=Y09nAxZFKzc
[1] https://www.microsoft.com/en-us/msrc/windows-security-servic...
For example, antivirus products do this all the time, as do many video drivers and other system utilities.
Also, Explorer has various plugin interfaces where it'll load third party code and run it in-process since the very first version.
A: So, people are resorting to injecting code in Explorer to delete in-use files in such numbers that it shows up in our top-100 crash report reasons
B: Well, maybe we should add a public API to Windows to support this incredibly common functionality that apparently has been missing so far?
A: Nah, let's just write a mildly condescending blog post that recommends using an unreliable workaround that is pretty much guaranteed to trigger any client-side intrusion detection software, that will set them straight
B: Right on!
(Meanwhile, somewhere, a third-party developer is gearing up to ship a kernel-mode driver to directly manipulate file system structures from their uninstaller, since their old solution kept crashing: can't wait to read the postmortem once the crash dumps from that one hit the Microsoft servers!)
A: There are a dozen ways to do this correctly. Which right way should we pick?
B: I don’t know, I found this code on the internet. Should I just use it?
A: Sure, if it’s on the internet it must be the right way.
I... don't think so? The particular problem here, is that an uninstaller executable needs to delete itself from disk after doing its main job.
Other than using MoveFileEx with a NULL destination file and the MOVEFILE_DELAY_UNTIL_REBOOT flag, then suggesting/forcing a reboot, I can't think of a straightforward solution.
And that solution instantly lights up your JIRA with 2 tickets:
#1: We MUST not suggest/force a reboot! Users hate it!
#2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
So, then you try things like 'create a Task Scheduler job to delete the file', which then adds:
#3: PRIO 1: uninstaller.exe crashes if Windows Task Scheduler disabled
#4: SHOWSTOPPER: uninstaller.exe still not always deleted after uninstall. Why is this so hard?
Et cetera, ad absurdum. This then escalates to code injection (as described in the linked article), and (you heard it here first) kernel-mode drivers. So, if you're aware of a reliable solution, feel free to share in a comment here, for the betterment of the world!
And what's the problem with this?
> When our product’s uninstaller sees an undeletable file (possibly a DLL loaded in another process), it uses the MOVEFILE_DELAY_UNTIL_REBOOT flag to mark it for deletion, and warns the user “Please reboot as soon as possible to remove the remaining files.”
> However, some user uninstall our product, just to be able to reinstall it later, to the same location. And of course they ignored the warning. Once they reinstalled it, everything works, until a reboot.
That solution instantly lights up your JIRA with 2 tickets: #1: We MUST not suggest/force a reboot! Users hate it! #2: CRITICAL BUG: uninstaller.exe still present after uninstalling product
I'm sure it would be hard to add such a thing to Windows while minimizing compatibility problems but I'm equally sure that it's possible and that it's worth the effort.
Like, for instance, the time that they decided that LaunchAdvancedAssociationUI(), the previous officially recommended way to show UI to allow the user to associate file types with a program, just wouldn't work anymore in Windows 10. Instead of opening up the Default Programs UI in Settings, it just displays a dialog telling the _user_ to go there -- which is even modal so they can't even refer to it while doing so. No compatibility shim or grandfathering for old programs, they just broke all applications that used this like they originally said good programs should do for Windows 8.
Or the case of Dark Mode in Windows, which for some reason they've dragged their heels on implementing barely any Win32 support at all for -- even just simply a call to query whether it is enabled. The current silly recommendation is to obtain the foreground color through WinRT and do a dot product on it to compute luma and determine if it is a dark or light color: https://learn.microsoft.com/en-us/windows/apps/desktop/moder...
Or the fact that the official way of reporting bugs on the Windows APIs is the Feedback Hub, which is completely unsuitable for task.
I don't have sympathy for the Windows team anymore. Their lack of developer support is partially responsible for all of the hacks that applications have to do to ship.
Programmers would hesitate to use it, because they can't be sure the user of the uninstaller will be on a fully updated windows install.
The JScript workaround might seem like a bit of a hack, but it has an advantage of working on every version of windows all the way back to Windows 98.
Any open file
Any file that was opened without specifying FILE_SHARE_DELETE in the call to CreateFile[1] (the Win32 equivalent of open(2)). Unfortunately, most language runtimes that wrap CreateFile tend not to pass that flag.
[1] https://learn.microsoft.com/en-us/windows/win32/api/fileapi/...
[0]: https://github.com/golang/go/issues/32088#issuecomment-53759...
[1]: https://bugs.openjdk.org/browse/JDK-6607535
So to me it's just not there...
> jstarks commented on Jun 18, 2019:
> [I]n the most recent version of Windows, we updated DeleteFile (on NTFS) to perform a "POSIX" delete, where the file is removed from the namespace immediately instead of waiting for all open handles to the file to be closed.
[1] https://github.com/golang/go/issues/32088#issuecomment-50285...
While full Unix-like behaviour is only available on Windows 10 for the past five or so years, you can still have the old win32 behaviour on older systems (delete once the last file handle is closed).
https://stackoverflow.com/q/60424732/1362755
AFAIK, that's exactly what they do. And it can cause problems, see for instance https://learn.microsoft.com/en-us/previous-versions/technet-...