But where are the tools? Of course I can always write a shell script, but the vast majority of GUI tools is not programmable.
I would love to automate my image capturing workflows, my tax filing, my invoicing, my calendaring, my contacts, and SO many other things. But most tools are not scriptable. They are sealed, proprietary binaries that I can't interact with.
Conversely, I love tools like the shell, Emacs, and programming tools in general, because I can automate them. I can automate testing, compiling, generating reports... And if something is not built-in, these tools provide hooks and APIs to extend their functionality for my particular use cases.
In fact, we can already see the impact of smartphone-only students on their ability to grasp and automate complex workflows. The fact that mobile apps are largely non-composable and non-programmable seems to seriously limit students' abilities in general digital problem solving.
I always remember being impressed by the Amiga and the ability to automate GUI applications with Arexx - I know OSX has Automator but apps don't seem to integrate as deeply (if at all) and Apple script feels nasty.
As someone with some potential admin work coming up, can anyone recommend some good cli tools for office work? I'd love to be able to take a word/libreoffice template and generate a document by supplying command line parameters for example.
> In fact, we can already see the impact of smartphone-only students on their ability to grasp and automate complex workflows. The fact that mobile apps are largely non-composable and non-programmable seems to seriously limit students' abilities in general digital problem solving.
Computers became so easy to use that no one learns how to use them. It's no wonder grandma always opens that attachment with an exe extension, file extensions have been hidden for as long as she's used computers, she doesn't know what the convey.
Teaching kids to code is the latest fad but I think the majority of them would be much better served learning some basic bash scripting and related unix tools.
Pandoc can generate and convert between various document types, including Office docs based on a template (although I haven't used the option myself, I just convert Markdown to PDF/HTML most of the time).
You use a scripting language - most languages have somewhat usable libraries for manipulating e.g. MS Office documents directly, not through the Office applications.
> I'd love to be able to take a word/libreoffice template and generate a document by supplying command line parameters for example.
Python with PyWin32 is great for this, the Word doc part anyway. I've got several scripts the create modify or read Excel and Word docs for a number of different purposes: tracking my mileage, automating creation of new users (now done with Powershell though), converting and uploading documents to a web server. VBA/VBScript can do some of these things too, but Python is a much more capable and sane language to use. Powershell is another great automation choice too.
python-docx allows you to generate Word files even on non-Windows platforms. Word 2016 says something about "converting" when it opens them, but they look OK. Apparently it will also let you modify them.
Someone else spoke about Python for Windows. That I have used with Excel but not Word. I imagine it would work pretty well, though.
On Apple and Windows we have Apple Script, COM, WPF Automation and quite a few other third party tools when the applications don't expose any kind of API.
In some respects I think the apparent lack of automation in GUI comes down to the methods by which it's accomplished feeling too crude to take seriously. Even ordinary "power user" apps like Excel are so in-depth already and support so many workflows that it's hard to figure out the efficient path for what you want to do.
OTOH, almost all major OSes also have tools that let you automate stuff at the input layer, so you can automate any GUI tool. The difficulty comes with race conditions on inputs.
Looks useful. I'd really like Siri integration. Actually, I'd probably want to say "Hammerspoon", then a menu would appear and I would say a keyword or a number.
A lot of polished produce lack this feature. You end up turning away a 90% solution for a 70% solution because the former isn't programmable. Emacs, for example, can easily become a maintenance nightmare.
Doesn't Emacs itself have calendaring options? If you're already using it and like it for its scriptability, maybe you're a classic candidate to bloat up your Emacs a bit.
My coworker has that XKCD comic on the wall. He spends so much time trying to figure out what is worth his time to automate that all potential time savings are negated.
I think the comic undersells the value of automation, there are documentation and accuracy benefits as well, not to mention they can often remove a lot of mental context switching.
First: suppose a task takes an hour and I do it once a month. If I can automate it so that it takes a few seconds, I don't just save an hour a month, I also have the option of doing it much more frequently.
Second: once a task has been automated it can become a building block for other tasks which I previously wouldn't have considered.
Most of the time automation is not only about time saving (and XKCD is just for fun).
It's also about: decreasing errors, allowing for stuff to happen when operators are not around, ensuring testability and reproducible behaviors, scaling up processes.
But those things ultimately translate as time: time fixing errors, time operators would spend if the task was not automated, time manually testing, time monitoring and scaling manually processes.
Sometimes being lazy is a lot of work. Usually I automate not to save time, but because manually repeating the same monotonous task is so annoying that I'd rather spend my time automating it even if I end up taking longer.
I also have it on my wall. One issue I see with automation is that human and computer processes often differ. Humans can easily deal with weird edge cases, but those can prove difficult to automate. So sometimes when you automate things, you have adapt the process first.
While Lightshot Screenshot is great, it cannot upload screenshots to other services than prntscr.com and that makes it unsuitable for sensitive information.
So far I've only found Greenshot that is cross-platform and let you choose where to save the screenshot (It's also open source).
Have you tried Monosnap? It's my favorite for macOS and Windows. Unsure if it had a Linux app, but it has a great quick editor, and support for a bunch of image hosts, including S3 uploads.
This is the whole promise of UNIX. Do something once? Write a shell on-liner at the console. Do it a second time? Save your one-liner to ~/bin or as a shell function in your .${SHELL}RC. Need something more complex than a one-liner? Grab Ruby or another "loose" language that adds some sanity -- but importantly not too much.
When you build tools, try to add options to make my tools more flexible and amenable to scripting. Options like -z, to separate records on STDOUT with NULL, or having a trailing -- read input from STDIN, make tools much easier to script.
Pipes are nothing more than function composition, already available in LISP systems since the late 50's.
Using a UNIX shell is like a poor man's REPL without access to the OS internals.
OS prior to UNIX already explored similar ideas by producing temporary files into cards, paper tape, tape and magnetic cores.
Using a multi pass Algol compiler was an exercise in patience with the card data being fed into the multiple stages, effectively doing pipes via paper.
Also UNIX pipes were influenced by the way pipes were designed in Multics.
Streams, meaning dealing everything as bytes were just innovative in that UNIX provided no alternative, many of the previous OSes used to provide pure binary access (streams) and structured file access (records).
That may be the official story but I can tell you as a onetime Multics programmer, Multics definitely had pipes and as we all know Ken T worked on Multics before going back to Bell Labs.
Everything doesn't index file contents, which makes the problem of searching significantly simpler. It does require the human to remember filenames. Offloading the problem to the user isn't the greatest idea.
You may be underestimating how bad Windows Search is - finding files by name is something it is inordinately slow at, and Everything is a good solution.
I used Everything attached to several launchers (Keypirinha(py), Wox(c#), Cerebro(js), Farr, Launchy, Zazu) before finding Listary, a very light launcher (30K on Win7) that does also fast file search (at least with a reasonable number of files).
Other good options are Keypirinha (with a python powered calculator) or Wox + Everything. Although Wox solution uses easily 150K memory.
This is gonna sound funny, but once way to train yourself to do this is playing Factorio[1]. Virtually every new thing you can build starts off as "that's a pain, I'll just make it manually" and rapidly graduates to "oh I need 5000 of them." You learn to just build at least a minimal production line for everything.
At least from version 0.15 on you don't need the enemies anymore to complete all research as the "alien relics" have been removed and the science system has gotten a major overhaul...
imacros [1] for firefox has been great for automating long tasks and can also be used on sites, that generate html elements with dynamic ID (for example typo3 backend). Together with csv files, it's excellent. I spent two days to find it, so you don't have to :))
The problem for me is in properly estimating the time it will take to automate a particular task. I often underestimate the time it takes to do it well enough to cover all the cases I would handle intuitively.
Usually though I never regret it - once it is done. But at the time putting a day or two dent in my schedule makes me pretty cross.
The first and second times I do something, I do them manually. The third time, I create a repeatable procedure that I execute manually. The third time I use that, I start writing scripts to take bites out of the time it takes.
Around the 30th or so time I have to do it, there's no point to automating any further. But I've learned that automating to the point where you can put it on a cron job is usually just not worth it. You haven't yet squeezed out all the failure cases. If you could have completely scripted it and put it on a cron job you would have done it at the beginning.
Usually when I find myself doing this, I'm doing it in response to emails that have been sent with spreadsheets. It's a huge time pit to automate that the first time you get one. You just don't know what they're going to keep sending you each month, and you feel like such an asshole nagging them about it. Just make a note in your procedure, and use an error to detect the case when you finally script it.
This is what Smalltalk and Lisp workstations promised and went some way into accomplishing, but they have never been built into the current generation of desktops like Linux and Windows.
We even have Emacs where everything is configured with elisp. The current problems come from building on the wrong foundations. The whole desktop paradigm and OS paradigm needs to be rebooted, and sure as hell the impetus will not come from Microsoft, Apple or Google, who seem to be entrenching sub par practices.
The author mentioned starting a static web server in any folder. I'd be interested in Windows solutions for this. I usually just run Apache and vhosts on my Windows laptop.
77 comments
[ 7.4 ms ] story [ 182 ms ] threadI would love to automate my image capturing workflows, my tax filing, my invoicing, my calendaring, my contacts, and SO many other things. But most tools are not scriptable. They are sealed, proprietary binaries that I can't interact with.
Conversely, I love tools like the shell, Emacs, and programming tools in general, because I can automate them. I can automate testing, compiling, generating reports... And if something is not built-in, these tools provide hooks and APIs to extend their functionality for my particular use cases.
In fact, we can already see the impact of smartphone-only students on their ability to grasp and automate complex workflows. The fact that mobile apps are largely non-composable and non-programmable seems to seriously limit students' abilities in general digital problem solving.
> In fact, we can already see the impact of smartphone-only students on their ability to grasp and automate complex workflows. The fact that mobile apps are largely non-composable and non-programmable seems to seriously limit students' abilities in general digital problem solving.
Computers became so easy to use that no one learns how to use them. It's no wonder grandma always opens that attachment with an exe extension, file extensions have been hidden for as long as she's used computers, she doesn't know what the convey.
Teaching kids to code is the latest fad but I think the majority of them would be much better served learning some basic bash scripting and related unix tools.
http://stackoverflow.com/questions/2050505/way-to-run-excel-...
http://stackoverflow.com/questions/27613878/list-of-perl-com...
Python with PyWin32 is great for this, the Word doc part anyway. I've got several scripts the create modify or read Excel and Word docs for a number of different purposes: tracking my mileage, automating creation of new users (now done with Powershell though), converting and uploading documents to a web server. VBA/VBScript can do some of these things too, but Python is a much more capable and sane language to use. Powershell is another great automation choice too.
Someone else spoke about Python for Windows. That I have used with Excel but not Word. I imagine it would work pretty well, though.
In some respects I think the apparent lack of automation in GUI comes down to the methods by which it's accomplished feeling too crude to take seriously. Even ordinary "power user" apps like Excel are so in-depth already and support so many workflows that it's hard to figure out the efficient path for what you want to do.
(Things like GUIs try to put a top on our data. Like owning the customer. And we always want to do more than the person who sealed it shut thought of)
http://www.mikadosoftware.com/articles/topless_computing
Start factoring "is this thing automatable?" in to your tooling decisions. The problem will resolve itself.
[0] http://www.hammerspoon.org/go/
Making Emacs a Lisp interpreter turned out to be a great idea. Too bad they didn't use Scheme or Common lisp.
It self referencing and you can add your own changes to it to automate you specific tasks.
Two other effects which are often large:
First: suppose a task takes an hour and I do it once a month. If I can automate it so that it takes a few seconds, I don't just save an hour a month, I also have the option of doing it much more frequently.
Second: once a task has been automated it can become a building block for other tasks which I previously wouldn't have considered.
It's also about: decreasing errors, allowing for stuff to happen when operators are not around, ensuring testability and reproducible behaviors, scaling up processes.
So far I've only found Greenshot that is cross-platform and let you choose where to save the screenshot (It's also open source).
It does have a really sexy UI though, almost as good as Capto!
When you build tools, try to add options to make my tools more flexible and amenable to scripting. Options like -z, to separate records on STDOUT with NULL, or having a trailing -- read input from STDIN, make tools much easier to script.
Using a UNIX shell is like a poor man's REPL without access to the OS internals.
OS prior to UNIX already explored similar ideas by producing temporary files into cards, paper tape, tape and magnetic cores.
Using a multi pass Algol compiler was an exercise in patience with the card data being fed into the multiple stages, effectively doing pipes via paper.
Also UNIX pipes were influenced by the way pipes were designed in Multics.
Streams, meaning dealing everything as bytes were just innovative in that UNIX provided no alternative, many of the previous OSes used to provide pure binary access (streams) and structured file access (records).
It's searching files with an ease that you start question yourself why windows search is so fucking retarded
I have 1.3 million files on my C: drive, and it can find any of them (or any matching subset) as fast as I can type.
It has a ton of options, but I have most of them on the defaults. Here are the ones I like to customize in Tools/Options:
In General, turn on "Check for updates at startup". ("Start Everything at system startup" should be on by default.)
In View, turn on "Show row mouseover".
In Keyboard, set "New window Hotkey" to Win+Alt+F (or any other key combo you like, but this one is pretty convenient).
And related to the current topic, Everything does have a CLI and an SDK.
Other good options are Keypirinha (with a python powered calculator) or Wox + Everything. Although Wox solution uses easily 150K memory.
Now I use osx and sadly finding files seems to be an unsolved problem...
[1] https://www.factorio.com/
/me goes off to read the change logs
[1] https://addons.mozilla.org/en-US/firefox/addon/imacros-for-f...
Usually though I never regret it - once it is done. But at the time putting a day or two dent in my schedule makes me pretty cross.
Around the 30th or so time I have to do it, there's no point to automating any further. But I've learned that automating to the point where you can put it on a cron job is usually just not worth it. You haven't yet squeezed out all the failure cases. If you could have completely scripted it and put it on a cron job you would have done it at the beginning.
Usually when I find myself doing this, I'm doing it in response to emails that have been sent with spreadsheets. It's a huge time pit to automate that the first time you get one. You just don't know what they're going to keep sending you each month, and you feel like such an asshole nagging them about it. Just make a note in your procedure, and use an error to detect the case when you finally script it.
I already automate to extreme.
We even have Emacs where everything is configured with elisp. The current problems come from building on the wrong foundations. The whole desktop paradigm and OS paradigm needs to be rebooted, and sure as hell the impetus will not come from Microsoft, Apple or Google, who seem to be entrenching sub par practices.