I'd love to have a service like this with wavenet-quality TTS. These are a bit jarring. Although, like low quality audio files, I guess you get used to it pretty quick?
It uses AMZN Polly for the backend, switching to Wavenet types would cause an 4x price increase (based on current offerings by the majors clouds). It is a bit difficult to justify because that would exceed the cost of an off-the-shelf audiobook from e.g. Audible. I do however plan to add it as an option.
If you have any ideas etc. drop me an email! (See profile)
Lights in the garden/living room come on automatically based on sunset time and weather. Fans ventilate the house/shed based on moisture and CO2. I'm in the process of automating the sunscreens on our windows.
Heating comes on at 4.30am in cooler months, turns off when we leave to go to work (along with the bedroom, hallways and kitchen lights which come on at 6am)
Heating comes back on at 2pm if outside air temperature is below a certain threshold.
Bathroom floor heating and towel rail switch on and off at various times during the morning, afternoon and evening depending on the day.
Does this mean you constantly have light there even when you're not around? Any particular reason for that (maybe the switches for it are placed awkwardly or so?)
Assuming you mean security nor for yourself (as in, to see where you're walking) but for burglars, thinking about this for a bit I'm not sure if this always works as intended. Would be interesting to see stats. But it's definitely not as simple as 'light on = scare away intruders'. Suppose your hallway light goes on every single day, even if you add some randomess to the timing, a burglar which scouts the neighbourhood a couple of days will quickly figure out the light being on doesn't mean anything with respect to you being at home or not. Moreover it would be convenient for him/her once inside the house since there's more visibility. Unless of course that also means the neighbours can then clearly see someone being in the house. Then again, if your neighbours don't know you're not at home that matters less.
My cat is often underfoot or very close by, without the automation I’d be turning the lights on and off manually many times a night. They only come on at 1% brightness.
Funny enough, possibly because I work on embedded software for automating random processes, and also from home, I don't feel the need to automate anything not work related.
When you spend most of your day stuck in front of a monitor, it feels good to get up and turn lights on/off.
I've often commented that I'm glad I have to water the garden because it's an excuse to just stand around looking at plants.
I don't like watering grass because grass is boring and requires a ton of water, so rather than install irrigation, I ripped out all the grass and planted more interesting plants. (Drought-tolerant xeriscaping is common in my area, so this isn't unusual.)
I don't think most of them are truly native to the area, but they're mostly cuttings of abuse-tolerant species that were growing well at friends' and neighbors' houses in the area.
When we bought a house with a pool, I did not keep the existing pool maintenance guy who'd come in every week for $100/month, and neither did I install a better pool vacuum robot.
I love the 30min when I come home after work where I scoop leaves and various kinds of debris out of the water. Very relaxing.
Hm, I had a pool once. Never again -- I felt like that's all I did was clean it, vacuum it, add chlorine tabs and clean the DE filter and destroy my lungs.
I also don't really like plants, or the outdoors. I'd much rather be inside in the A/C. Working on my own coding projects doesn't bother me, even after coding all day long.
Perhaps it's because I've been spending less time coding at work and more time managing...
Same here. Only part of my work involves automation etc but indeed for some reason that took away the need/desire to automate more at home. Likewise when setting up the electricity in our house I specifically selected something based on teleruptors only without any programming/smart stuff. Just works, still some versatility since it's wired in a star, and I also just happen to like hardware switches and buttons vs touch screens.
Me too, at work (PhD student) I take enormous pleasure in writing end to end fully automated pipelines. Nothing that needs to be done more than once isnt automated pretty much. At home, I would never buy or use an Alexa/Google assistant/use Siri or any 'smart' device, I simply don't want to bother with it.
Exactly. Anything that needs even very minimal human interaction like pressing a "y" key is not automated at all. It is very common to see in the corporate environment that scripts needs some editing in b/w to be done during its running with editor which can be easily automated using sed.
An interesting effect I have noted for me when doing research is that even if it only takes one extra click or changing one parameter to generate a plot, or run some extra analysis, I will only do it selectively and start rationalising it to myself that I know when a certain plot or analysis will help and when it won't.
This becomes especially egregious over month to year long experiments where I run the same experiment every day on end.
There was really no reason not to auto generate every possible plot, every possible analysis every time (and I cannot use ipython notebooks or things like that because it's many distributed things chained together with lots of scheduling).
The productivity gains have been enormous and are hard to overstate. I don't dread any experiment any more because even in a large complicated distributed setup, everything from initialising kerberos tickets to tons of config files, restarting services, running multiple experiments dependent on each other, and generating plots and summaries and committing them to a repo is one command. Anything that's analysed once is evaluated always.
I now almost look forward to setting up new experiments because of the pleasure I get from just chaining together calls from my control utilities.
All I have to do is pull on my laptop and I download a filter with all results pre-generated paper ready. I think a lot of people do this in experiments where everything is on a single machine, but I haven't seen it as excessive from other phd students doing complicated distributed stuff. There is always a lot of manual command line args passing, manually changing some config while instead of just creating dedicated scripts, etc.
I can relate to this. I would like some remote control features though but then I always consider what a hacker could do and what's the worst that could happen if the system malfunctions.
Quite often the worst-case scenario is a fire, so the little benefit is IMHO not worth the risk.
I'm talking about self-made hacks and cheap Chinese hardware here.
Or false positives that desensitize you. Cry wolf too many times and people tune out.
I got my parents a Nest smoke alarm (I can see the alarms too) but a year later it was being triggered by steam.
The folks at Nest actually sent a free new model that's better at not triggering for steam. But I have to wonder if my elderly parents trust it anymore.
It enforces no internet before sleep rule. Personally I find it hard to stick to such habits just by my own will. Too often I find some excuse to turn on my laptop just to check one thing, which then turns into one hour wasted browsing random junk.
Yes, same here except it turns off at quarter past midnight. I reprogrammed a Sonoff switch so that when it turns the main router off it turns on a wifi hotspot which allows me to turn the switch on again - but only for an extra 15 or 30 minutes. In terms of stopping me wasting time on the internet until late at night while still allowing access if I really need it, it seems to work well:
Automatic driveway lights via motion sensor. Automatic house fan (when in saves me money on heating or cooling). Light assisted alarm, makes the bedroom lights 10% brighter a minute before the alarm goes off.
Pretty amazing how cheap things are these days, an arduino or Pi 0w and some relays can do quite a bit. Just bought a opengarage.io and considering the opensprinkler.
Daily backups (ARM board in my parents' garage plus rsync and cron, and vice-versa). I think the original script is over 10 years old now.
Reminders to do cleaning and maintenance, just cron emails to Trello.
Music alarm clock, this has been local control of Amarok (KDE) or a Python script for a Chromecast.
I intended to automate control of the central heating, which for some reason has a commercial grade control system in my apartment with an ethernet port. But I found the building is so well insulated, it hardly runs anyway. (Which I found by monitoring the state of the heating and making a graph.)
Most automation doesn't work for me. I don't keep to a set 9-5 schedule, so anything that assumes a routine fails me. Here's what I do use.
Tado - a smart thermostat, sets the temperature while we're in the house. Heats up water only if we're in. Uses an app to geolocate us.
Lifx - lightbulbs which automatically dimm at 2300 (good reminder to go to bed) and auto switch off a couple of hours after sunrise.
Everything else is either always on (security cameras, smoke detector) or run as needed (Roomba, car charger, electric blanket) or autonomous (Moixa solar battery).
> I don't keep to a set 9-5 schedule, so anything that assumes a routine fails me. Here's what I do use.
This has been a pet bug of mine for a while. My shifts are never the same 2 weeks in a row, and I feed it all into my calendar. But homekit etc won't consume this.
I've botched together a few scripts to set my heating based on the events in my 'work' calendar, so it sets before I get home, not after, and it does seem to work much better than any geofencing or 'learning'.
I built an accelerometer-based washing machine sensor which sends a notification to my phone when the washing is done. This is useful because the machine itself has no indication when it's done: http://www.instructables.com/id/Washing-Machine-Notification...
I did a similar thing, via an esp8266 device and a vibration sensor!
I also automated the display of tram-departures from the tram-stop next to my house with similar hardware.
Otherwise I setup some "wireless buttons" to trigger alerts/actions on my Linux desktop system, and record local temperature/humidity into a time-series database for tracking purposes.
I was wondering about that as well but I only found the Amazon ones, which either need tricky operations to be usable (the brand-oriented dash ones) or expensive (the Amazon IoT (eom dash button) for 25 eur).
For the buttons I first of all wired a button to an ESP8266 device, made it connect to wifi and publish to MQTT and handled it that way.
But when it came to time to setup a lot more of them I realized that would get expensive quickly. SO I cheated. I bought a bunch of 433Mhz-based radio-buttons:
Then I hooked up an SDR receiver to my PC, and handle it that way. I have a 433Mhz receiver for the arduino, which could also decode those transmissions, but I never used it for anything except to prove I could.
There's a pointer to the software and some overview here:
What kind of display did you use for the tram-stop times? I wanted to do something similar but worked out that the cost to update the display every minute was too much in terms of battery life even with an epaper display. Also, it was too inconvenient / ugly to use mains power.
I do have a couple of epaper devices, and they would be ideal but for their terribly slow update-time. I like the clock showing Time & Date (or Time & Temperature), and having epaper update every second just isn't possible. Even if it were then I suspect updating so frequently would negate the potential power-saving of using epaper in the first place.
I do have a nice epaper device for showing weather & weather-forecasts, that works on batteries and updates every 20 minutes. But for trams I update departures every 2 minutes.
Not OP but did a similar project, the display method I used was simply to use 9 led lined up, each representing a stop before mine, to show the current position of the tram.
My machine has a time remaining feature which is great in theory but it's completely inaccurate. It might tell you there's 45 minutes left in a cycle, you'll come back an hour later and it'll say 17 minutes. I guess I know where the windows file-copy dialog author went to work after Microsoft
I think modern washers weigh the clothes and change their cycles based off that and other information. Mine is similar but seems to always overestimate the time instead, which is probably the better failure state.
From what I understand, it has to do with 1) the time displayed is for an "optimal" load (not too full, normal soilage, balanced, etc.) instead of the actual load, and 2) the machines may pause and restart or lengthen parts of the cycle if the weight and/or balance are slightly off.
In any case, the washers at my apartment complex are always off, but it's at least consistent so that I can add about 6 minutes to whatever it shows at the beginning and plan around that.
The semi-old one in my building weighs it when you start the load and gives an accurate minute timer. The odd thing is it is always different, almost no correlation to weight, sometimes 50, sometimes 70.
Some industrial laundries do this, but I have heard this over and over again to describe residential HE laundry machines but never actually seen one with a way to measure this. The usual method is to tumble the clothes until the water level in the tub stops dropping. Now they're saturated with water.
What you are probably getting when you start your washer is an estimate considering the steps of the cycle and loaded to capacity. Some machines have turbidity sensors - mine doesn't - this could deliver information that might shorten the cycle. Also the time may go up if it has to deal with suds or stops spinning to rearrange the load if it shakes.
Most of the variation in how long mine takes is during the spin cycle, which seems like an easy thing to get a sensor on. Drain the water then spin until you stop getting more water to drain.
We bought an LG washer and drying last year that tries to be smart and it drives me nuts. When you start it, it takes several minutes moving the tub in very small increments. It starts and stops and reverses over-and-over and I assume it's sensing the load. Then it starts and occasionally the balance is off and I does the thump-thump-thump thing. So I stop it, rearrange the load and restart. Instead of just resuming, it goes through the sensing dance again and I have to stand there three or four minutes before it actually starts to spin again.
If I could turn all the load sensing stuff off, I would. All I really want is two sliders that give me 0-100% for load size and temperature and a switch for delicate.
A plug-through device (or cable mounted) could probably do this based on current draw, similar to the killawatt but with different purpose. Might be able to also do error messaging, or warnings of exceptional current draw.
My solution for this was an ESP8266-based Sonoff POW (wireless smart switch and power monitor), running Tasmota, sending data through MQTT to Home Assistant, scripted to send notifications through Telegram.
Interesting, I didn't know that Sonoff did a power consumption monitor. That makes sense for something like a washing machine. Presumably though you never actually use the relay/off-switch capability of the Sonoff, right?
The only time so far I used both the switch and the power monitor was on the Christmas tree, because I was curious how much it cost to run ($3 or so overall iirc).
My trick is to set a timer on my phone when I start a load. Even if the machine doesn't tell you how long it's going to take on a display, these things tend to be pretty consistent. Low tech but it works.
Screen scraping photos of my kids from a kindergarten website. Recognizing good photos that feature my kids and deleting everything else is manual, though.
Alerts to my phone if the fridge or freezer temperature exceeds a limit or the humidity under my dishwasher starts rising.
Granted, neither are fully automatic and require manual intervention.
I wouldn’t be that pleased with a childcare facility taking photos of my kid then putting them on the internet. I suspect I’m in a small minority though.
It's behind a login and they've asked for parents' consent for each child separately. Everyone with access has agreed not to share any of the photos.
I actually haven't shared my scraper script just because I think someone would then share the pictures (perhaps accidentally).
EDIT: I'll just add that there are no public photos of my kids and I ask anyone who takes photos not to share them outside family. So I do agree with your point.
The question was directed at lostlogin, not you. I’m interested to know why people object to have their kids pictures on the sites of the daycare providers.
Kids don't need to be exposed on the internet until they are able to decide themselves if they are okay with it. When that is is up to the parent. I always ask my 8-year old even before I share photos with her grandparents.
It’s more that I don’t know the reason it’s being taken. We have had photos taken by tourists that have sidled up and taken photos and had companies take photos then ask for permission for use in promotion. We felt pressure to accept these after the event, though didn’t.
I'm using a ready-made solution from http://wirelesstag.net . While I think I could build suitable hardware myself, I wouldn't have the skills to make them run on a battery for two years.
I write my all own tools, currently private. To get the data I use a combination of screen scraping and Quicken OFX gateways. Everything else is basic coding and accounting. I have a CPA that advises me on tax rule updates yearly and signs my return.
I used DSP to recognize the commercials in the radio broadcast on my stereo receiver and turn down the volume automatically ("adblock for the radio broadcast"). I described it here: http://blog.rekawek.eu/2016/02/24/radio-adblock/
I automated handling DNS updates via simple "git pushes" - Lets you revert from bad changes, and gives you a good history of changes over time - https://dns-api.com/
Great work! I have been working for a while on a similar adblock, but that does not need jingles to detect ads. I'm on my way to open source it. A radio player with adblocking included is available at https://www.adblockradio.com
> The commercial block starts and finishes with a jingle, so the potential software should recognize these specific sounds and turn off the volume between them.
I suppose that many stations don't have such markers though.
First is to analyze the other signal features of the commercials (eg. increased volume), although it may be tricky.
The other option is a crowd-sourced solution - pretty much as for the browser adblock - where users can mark samples recognized as ads. Since the publishers often buy campaigns for many stations in the same country or state, it may be a shared database.
On the other hand, the described project only scratches my own itch. I wouldn't try to productise an app that takes away the main source of income for the radio stations.
Yes, or perhaps a combination of techniques. E.g. shared database to train an ML system to detect ads. Of course, the downside is that the ad industry will then tweak the ads until they pass the ML test.
> I wouldn't try to productise an app that takes away the main source of income for the radio stations.
I wouldn't think of it as taking away a source of income, but rather as forcing them to find a source that doesn't bother their customers so much. Ad blockers seem to be getting more accepted.
Good point. Perhaps use only parts of the material (not the entire clips). This should be covered by "fair use".
And perhaps using the clips for other purposes than "viewing" may in fact be fair use. Especially since you are trying to find a method for not viewing them.
IANAL as well, but this notion of "fair use" seems slippy. I had a look in the French article that lists some exceptions that allow sharing copyrighted material, and did not find any obvious match with the potential shared database.
Yeah, but if you lose enough info such that it doesn't play like the original or can't be reconstituted into the original it's still large enough of a change that it should steer clear from copyright issues. No one's really tested exactly where the distinction falls.
How about a third option, ads are by definition short and repeat a lot.
No fancy ML needed, after a couple of times the filter gets one of these repeating fragments it should be able to block it. Fairness bonus: you get to hear each new ad a couple of times.
The problem you have here is that in order to continue listening to songs more than just a couple of times you'd have to able to classify the song versus an ad. Otherwise once the system heard the song a few times it would start blocking it too.
it's radio. the length is infinite. there are pauses in music. there are pauses in speech. Also, he didn't mention anything about delayed listening. The implication was that it happens in real-time so you don't know how long it will be when you need to start dropping the levels.
That's not necessarily a bad thing. The decline in commercial music radio has been, in large part, because people quickly grow tired of hearing the same songs over and over again.
Ads repeat several times an hour, songs very seldom repeat within an hour and even then usually only once. It should be possible (although not sure how practical) to just autocorrelate the current audio with audio over the past hour.
Maybe radio spots are a little different because they're cheaper and usually more low-quality than TV ads, but it doesn't really work for TV ads - they often have small variations, e.g. 10sec identical, 5sec different, 10sec identical (easy example). Also depending on your method of analyzing the audio it's sometimes broadcast with an unhearable fingerprint that distorts the waveform (let's say like MP3 versus WAV, but worse).
So yes, you can find some patterns - but the commercial breaks are highly mixed up and you wouldn't believe how many distinct commercials per channel are there, even if you think you hear the same ones all the time :)
Easier still... in the US, many radio stations transmit the artist and song title that's currently being played, via a low-bitrate subcarrier encoding. (The proper name is "Radio Data System", IIRC.)
There were previously some FM to MP3 "ripping" tools that would use the RDS information to tag the resulting recordings -- I'm not sure of the status of them. But it could provide a good way to detect commercials, since most radio stations change to a generic station identification message when they break for commercials / banter. (Whether you'd also want to turn down for banter is another question.)
It certainly works for some stations, but not for many other (from personal experience in France), and it would be trivial for radios to change their RDS / metadata system to circumvent ad blocking.
As long as it's only individuals doing this I don't see anybody reacting on a wide scale. Also there's no feedback of listener numbers depending on volume. So there's no way to detect people "adblocking" on radio, so there's no impact on revenue.
> analyze the other signal features of the commercials (eg. increased volume), although it may be tricky
My home theater receiver does this (Marantz). It works pretty well. It doesn't cancel out the TV commercials though, it just normalizes the volume so it matches the show. But, I assume you could make it work for muting too.
I wonder, if you accepted a little delay, say 20 seconds, could you use Shazam to identify what song is currently playing? I have been amazed at the range of music Shazam can identify, if you could do regularly queries and separately listen out for gaps in music, it seems plausible to be able to identify when a non-song (from Shazam's perspective) is playing.
I once made a radio adblocker for streams hosted on http://radioplayer.co.uk/ to be run in a terminal. There is also code to increase/decrease the volume on a Mac. It's not perfect but was a fun little project.
For some reason, the data and audio are out of sync but once calibrated it works quite well.
This Is great. I wanted to do something similar with a TV. Mic detects commercial signature and sends mute command via IR interface. I'm just an idea man though who never followed through.
I would like something like this, but to cut out all non-music (i.e. talking, interviews, etc). Would tensorflow be able to be trained to detect just speech vs music? It would obviously fail if there is speech with background music, but that isn't too common outside of Jamaica (and reggae stations/shows)
Monthly credit card payments. So money flows from my employer to my checking account to my credit cards. Also of course to my 401K, IRA, Mortgage account, Saving account etc. Which means I don't encounter money much and I like it this way (I generally make sure I live within my means and my money does flow into my investments).
Downsides:
-I'm not actively thinking about money and hence improved investments
-I'm not monitoring credit card errors
-Comcast gradually increased increased my bill from $44/mo to $103/mo without any resistance from my side
My credit card bill is paid every month by deducting from my checking, and the bank does this by default. I don't know why everyone wouldn't want this. Unless they're spending money they don't have. Interest is a sucker's game.
I do this for many years now. My one check is that when the credit bill comes (electronically) I sort it by size and check that at least all the big charges make sense. Also I have alerts on my cards for any charge over a few dollars.
Money-related matters are my primary automation as well. Especially with the fair bit of traveling I do, it largely eliminates something I'd have to keep an eye on and probably get a late fee from every now and then.
Like you, it does mean I need to make mental notes to check into things every now and then so that there aren't charges I don't understand or investments that really seem out of whack.
Pretty much the only other thing of note I do, besides have a lawn service, is some automated plant watering because of the aforementioned traveling. For this and other reasons my schedule is variable enough that none of the usual SmartHome stuff really does anything for me even if it solved problems I felt I had, which I don't other than some remote monitoring.
I started this right after college after reading "Automatic Millionaire" that advocates for "paying yourself first" and automating your finances. Money goes into checking then automated to online savings accounts, one account per purpose (xmas, vacation, kids college, kids sports, big ticket purchase), money goes to 401k/IRA, money goes to stock account. Only bill I need to remember to pay is credit card (although minimum payment is at least made so no late fees), everything else is automated. Only decisions I make are how to rebalance things if I overspend the money left over, which stocks to invest, and periodic rebalancing to make sure things are on track to reach long-term goals. The real savings is "saving me from myself" :)
I made a sensor board that has light, motion, temperature and humidity sensors and IR LEDs and put one in every room. I put Sonoffs on the lights and various other devices and wrote a small managing program to coordinate the lot, so lights turn on when you walk into a dark room and turn off if there's no motion or if you turn on another light. The IR LEDs also control the TV/AC/etc.
I've also added various other things to it like Wifi presence notification, so it sends me a notification if a guest arrives, and hooked it up to Kodi and an Amazon Echo so I can say things like "play Iron Man" and it will turn on the TV, put on the movie and turn off the lights.
I used an ESP8266 with cheapo sensors, the whole board costs like $5 or something. The software is pretty ad-hoc but is not more than 100 lines of code.
I have several scripts that make my keyboard behave better, which is "automating" in a sense. I basically have vim-style keys anywhere on the computer, as long as I'm holding caps lock. (So e.g. caps+j is the same as down arrow). This includes some more complex motions like "select current word", and even things like "open two quotes and but the cursor between them."
I also have a script which scrapes my podcast player's website, to give me a list of podcast episodes that I already played. I save this list every day in an excel file, but the script makes this much easier - I run the script, then paste the results into the excel.
Ooh also I automate a bunch of news downloads via calibre and Amazon's email to kindle service. So ive got Sunday papers for the cities care about, the Economist, etc all ready and synced when I fly or take a train each week. And updates are easy to just enable my phones Hotspot and go for it
Nice! Where do you host your Huginn server? I've had trouble getting Huginn running on my RPi, I've to try again soon to get it running. Any chance you'd be willing to share some of your recipies? I know someone who scrapes a certain deals website in my country, and gets Telegram texts for items matching certain parameters, which is cool.
Just on a cheapie $5 a month VPS. Dockerizing makes it very easy to get going.
I've put some recipes on the wiki off Github. Most of them are quite teasy to string together once you just get the right extraction pattern, then they're simple WebsiteAgent -> Dedupe Agent -> (Email/Digest/Pushover Agent)
The most useful thing I have is a little script that logs into my bank, downloads a CSV and sends me the latest transaction via email daily. Saves me time to login to online banking, and also serves as a useful way to search for previous transactions inside my mailbox from time to time.
Besides that I use Home Assistant[0] with a few simple automations. For example, when the lights are on in the bathroom, and it's after dark, increase the heating/towel warmer (using Tado[1]).
Tado has its own home/away detection, plus some schedule for the heaters. It's supposed to save on energy costs.
When we're on holiday, I switch on/off various lights (with some random offset) at night... Or some times I might leave a radio on timer / connected to an IoT power switch. (I don't use any burglar alarms, cameras etc, but I try to deter them with those simple fake occupancy tools)
Philips Hue has a nice way to slowly turn the lights on. Useful to wake up during the dark winter months.
I've wanted to do something like this for so long but it just seems so painful. How do you do this with just a little script when there's so much Javascript involved in the page, as well as random interjections like random extra security verification steps? Do you have a particularly scrape-friendly bank?
Try to watch network requests in developer console of your browser. Sometimes there's a simple REST interface behind that JavaScript and all you need is to login and issue one request. Extra security verification steps probably won't let you do that, though, if implemented correctly.
In theory, yeah. I actually did try Selenium and Chrome several months or maybe a ~year ago. I don't remember exactly what the pain points were but I remember it was really painful to figure out how to get it to work. Everything from figuring out how to inject JS into a page to figuring out how to wait for events, etc. it was just a massive pain. I don't think I ever figured it out entirely.
For your banking script, are you able to use an API with a limited access token? Or does it use your normal credentials. I don't save my various banking credentials anywhere, not even 1password, as they're so sensitive, curious how you do this.
Your banking password isn't that sensitive, because banks are used by regular people who don't have any conception of good security practices. The worst case scenario of your bank login credentials leaking are really not that bad.
Your personal email account is probably a million times more sensitive. It contains enough personal information to get your identity stolen a thousand times over, and once your email has been compromised there is no way to ascertain the damage.
Couldn't agree more. See my comment above. My online bank only provides readonly access with a password. Anything else requires a one-time-password anyway.
I'd be much more worried about my email getting hacked though.
Your banking password isn't that sensitive, because banks are used by regular people who don't have any conception of good security practices. The worst case scenario of your bank login credentials leaking are really not that bad.
Huh? The worst case scenario of your bank login details is... someone depositing all your money to their account, surely.
Nope, the login in (some? many?) banks is just a first step, to actually perform transactions they require a second stage auth. Most they can do is transfer money to some of my family members (I've disabled the second stage for a few specific destinations).
I don't hear often about banks providing API access. Only big companies on special contracts may get something like that, but it is usually very limited - talking from my personal experience, but what I learnt from the internet - it's the same everywhere.
But... In Europe thanks to just another EU laws we should have every bank offering an API access for everyone starting from September 2018 IIRC. That would help a lot with automation, I hope generating few api keys with different access permissions would be possible with that.
PSD2 is not what you think it is. It has API access, but not for the end user. Only 'integrators' will have access to these APIs. You will need to request permission to write/deploy software using the APIs. This involves massive bureaucratic overhead and in some cases permission from your country's national bank.
There is a system for integrating with larg(er) corporate banking software, but here it seems to require a smartcard, likely with a PIN, and no way to restrict it to be readonly.
It's horrible XML protocol stuff though.
My bank gives only readonly access by default with the username/password. Any transaction (transfer, operation) requires a one-time pin (TAN) anyway. So I'm not particularly worried about leaking my password.
I wish they had an API but I just use a selenium script essentially. Luckily(?) logging in to this particular account only requires user/password and no fancy codes etc. I guess when the system is built with readonly default and OTP for anything else, then the login can be kept simple.
I really want to like Home Assistant, but at least on Windows there's some gnarly concurrency issues with it the last time I looked: it launches two Python processes and one of them fails every time.
Also, there was no auto-discovery of LIFX lights and I was told I needed to re-implement the LIFX protocol from scratch because no extra dependencies were allowed in the discovery module.
You missed a big obvious one there, you've automated your grocery shopping. I'm still driving, parking, grabbing, paying, packing, and driving like a sucker.
Yes, they are often incorrect and appear to be manually entered and not updated to reflect price changes as the system does not seem to have a field for item weight.
Yes, and normalize it to price per kg. The product weight needs to be extracted from the item title or description in many possible formats (100G, 100 g, 100 grams), the price location and format however is uniform.
Ooh, I've been looking for something like this. Could I ask how you parse your bills? Did you just hack together your own script or did you find a parser online? I tried doing this several times but it seems rather painful to get right since the formats can differ and since copying text from PDFs can sometimes ignore the table layouts and whatnot.
It actually gets a lot better at categorization the more you use it. It has at least some primitive machine learning (possibly rules engine?). Categorize a transaction once and it usually gets it right subsequently.
There are no general solutions, unfortunately. Every PDF-generating system will do things differently, so parsing them must be custom for each type. Generated PDFs are usually very systematic, though, and PDF commands are text (compressed). You can uncompress with PDFTK:
pdftk input.pdf output output.pdf uncompress
Then try grepping or using whatever tools you like (there will be binary parts of the file still, like embedded fonts and bitmaps).
I just tried it on a bank statement and the only text I see is the PDF commands (stuff like "/MediaBox [0 0 612 792]", "/F1 125 Tf", and whatnot). There's no actual PDF content text that I can find, whether it's my name, the bank name, or anything else.
There are a few possibilities. The main PDF commands for text are "Tj" and "TJ", so you can try searching for them. Text commands must also be between "BT" and "ET". Maybe a bit harder to search for are the other two text commands: ' and ".
TJ is "Show text, allowing individual glyph positioning" which means strings are not necessarily in the PDF as the strings themselves, but have glyph positioning info mixed into them. This is the most common issue I've come across.
Another possibility is that the fonts have been combined/compressed/butchered, so that they're not using ASCII/Unicode (this can sometimes save space by combining bits of multiple fonts. Searching for Tj/TJ/'/" should tell you if that's the case. If it is, technically there should be a translation table somewhere in the PDF file (so that clipboard operations work in PDF viewers). Look for CustomEncoding, I think.
Particularly bad PDF creation libraries can put characters in one by one.
Characters (alone or in strings) can also be in there as octal or hex.
Finally, particularly obnoxious libraries can do it without text commands at all, but convert everything to vector drawing commands, or even worse, bitmaps.
Interesting, thank you for the explanation! I am indeed seeing a lot of Tj's and BT/ET's, but not any TJ's.
Immediately preceding the Tj's are a lot of non-text characters that don't look like anything human-readable. Immediately following them are other other commands (Td, Tf, rg, k, etc.) which are preceded by numbers. There are no CustomEncodings. So I guess this means I can't really grep anything?
Sounds like it. The T commands are text related and rg and k are setting colour in RGB and CMYK. You may be able to figure it out by changing a colour, then looking at what is suddenly bright red or whatever you set. Or by changing the string preceding Tj. When editing, make sure you don't change the length of any objects (use space to pad) as there are hardcoded offsets in a table, usually at the end.
The PDF specification, while long, is pretty readable for this kind of thing. You can download the 1.7 version free from Adobe. Appendix A has a table of commands.
The lights in the most reasonably trafficked areas of my apartment are automated. The kitchen and bathroom mainly.
If I'm in the kitchen I'm actively cooking or using the sink and pacing around, so the motion sensor keeps the lights on. After 1 minute of no motion, they turn off. Important to set timers though as the lights are no longer a reminder if something is in the oven or on the stovetop.
For the bathroom, I have a sensor placed precisely to view both inside and out of the shower with the curtain closed. This means the lights won't turn off while someone is in the shower unless they stand really still for 2 minutes. If someone wants to take a bath and won't move much, I can override the sensor to a longer shutoff time.
At 6:30pm I have a light that turns on to simulate that we're home. Looking to move this to a "sunset minus 30 minutes" model soon so it appears more organic.
At 2am, all of the lights turn off in case we forgot, or as a friendly reminder to go to bed. It hasn't worked today. :)
For the bath, couldn't you add a sensor that detects water level in the bath and use it to override the lights? Even something as simple as measuring the weight of the tub would work, and would simultaneously cover the shower cases too.
I use https://www.home-assistant.io/ to do a coupple of automations like switching everything which uses power off when I leave home to preserve energy and switching a CCTV cam on at the same time for safety reasons.
But one of the more usefull automations is a text2speach script which fires in the morning between 6 and 11, during workdays, when I'm not asleep but still at home, always 7 minutes before the next train to work leaves. This way I miss the train faar less often.
Seconding Home Assistant, it's really the glue that allows for the leveraging of other systems.
I was able to set up something I had wanted for a long time which was to turn off my TV/Stereo/Kodi after idling. I built the conditions which caused the the whole thing to shut off via Harmony hub after 10 minutes of idle after being played.
The one downside is the project moves really fast so you have to keep up with breaking changes sometimes.
I use home-assistant with LIRC to give my Chromecast a dedicated play/pause button on the universal remote. It also pauses the Chromecast when I power off my AV receiver.
Then there's a cron job that beeps on the hour (a 4-bit number from 1-12), so I can hear if my computer and watch are in sync.
746 comments
[ 4.2 ms ] story [ 333 ms ] threadhttps://Auditus.cc
I'd love to have a service like this with wavenet-quality TTS. These are a bit jarring. Although, like low quality audio files, I guess you get used to it pretty quick?
If you have any ideas etc. drop me an email! (See profile)
(The files persist until the conversion is fully complete i.e.uploaded to S3 and sent to user)
So, mostly domotica stuff.
(If indoor, overkill.)
It works perfectly (watering the plants 1-2 times a day depending on the setting I set); I like its simplicity and easy of use!
Heating comes on at 4.30am in cooler months, turns off when we leave to go to work (along with the bedroom, hallways and kitchen lights which come on at 6am)
Heating comes back on at 2pm if outside air temperature is below a certain threshold.
Bathroom floor heating and towel rail switch on and off at various times during the morning, afternoon and evening depending on the day.
Does this mean you constantly have light there even when you're not around? Any particular reason for that (maybe the switches for it are placed awkwardly or so?)
edit seems it's indeed a point of discussion https://skeptics.stackexchange.com/questions/8956/is-it-safe...
When you spend most of your day stuck in front of a monitor, it feels good to get up and turn lights on/off.
I don't like watering grass because grass is boring and requires a ton of water, so rather than install irrigation, I ripped out all the grass and planted more interesting plants. (Drought-tolerant xeriscaping is common in my area, so this isn't unusual.)
Nice. A lot of local plants too?
Hard to overstate how good this is for one's mental health.
I love the 30min when I come home after work where I scoop leaves and various kinds of debris out of the water. Very relaxing.
I also don't really like plants, or the outdoors. I'd much rather be inside in the A/C. Working on my own coding projects doesn't bother me, even after coding all day long.
Perhaps it's because I've been spending less time coding at work and more time managing...
This becomes especially egregious over month to year long experiments where I run the same experiment every day on end.
There was really no reason not to auto generate every possible plot, every possible analysis every time (and I cannot use ipython notebooks or things like that because it's many distributed things chained together with lots of scheduling).
The productivity gains have been enormous and are hard to overstate. I don't dread any experiment any more because even in a large complicated distributed setup, everything from initialising kerberos tickets to tons of config files, restarting services, running multiple experiments dependent on each other, and generating plots and summaries and committing them to a repo is one command. Anything that's analysed once is evaluated always.
I now almost look forward to setting up new experiments because of the pleasure I get from just chaining together calls from my control utilities.
All I have to do is pull on my laptop and I download a filter with all results pre-generated paper ready. I think a lot of people do this in experiments where everything is on a single machine, but I haven't seen it as excessive from other phd students doing complicated distributed stuff. There is always a lot of manual command line args passing, manually changing some config while instead of just creating dedicated scripts, etc.
Quite often the worst-case scenario is a fire, so the little benefit is IMHO not worth the risk.
I'm talking about self-made hacks and cheap Chinese hardware here.
I got my parents a Nest smoke alarm (I can see the alarms too) but a year later it was being triggered by steam.
The folks at Nest actually sent a free new model that's better at not triggering for steam. But I have to wonder if my elderly parents trust it anymore.
https://mikrotik.com/product/RB951-2n
https://github.com/alexspurling/sonoff
I have a vague plan to extend Espurna to have it built in.
Pretty amazing how cheap things are these days, an arduino or Pi 0w and some relays can do quite a bit. Just bought a opengarage.io and considering the opensprinkler.
Reminders to do cleaning and maintenance, just cron emails to Trello.
Music alarm clock, this has been local control of Amarok (KDE) or a Python script for a Chromecast.
I intended to automate control of the central heating, which for some reason has a commercial grade control system in my apartment with an ethernet port. But I found the building is so well insulated, it hardly runs anyway. (Which I found by monitoring the state of the heating and making a graph.)
Tado - a smart thermostat, sets the temperature while we're in the house. Heats up water only if we're in. Uses an app to geolocate us.
Lifx - lightbulbs which automatically dimm at 2300 (good reminder to go to bed) and auto switch off a couple of hours after sunrise.
Everything else is either always on (security cameras, smoke detector) or run as needed (Roomba, car charger, electric blanket) or autonomous (Moixa solar battery).
This has been a pet bug of mine for a while. My shifts are never the same 2 weeks in a row, and I feed it all into my calendar. But homekit etc won't consume this.
I've botched together a few scripts to set my heating based on the events in my 'work' calendar, so it sets before I get home, not after, and it does seem to work much better than any geofencing or 'learning'.
I also automated the display of tram-departures from the tram-stop next to my house with similar hardware.
Otherwise I setup some "wireless buttons" to trigger alerts/actions on my Linux desktop system, and record local temperature/humidity into a time-series database for tracking purposes.
I was wondering about that as well but I only found the Amazon ones, which either need tricky operations to be usable (the brand-oriented dash ones) or expensive (the Amazon IoT (eom dash button) for 25 eur).
Surprisingly I did not find one on AliExpress
But when it came to time to setup a lot more of them I realized that would get expensive quickly. SO I cheated. I bought a bunch of 433Mhz-based radio-buttons:
https://www.aliexpress.com/item/OWSOO-433MHz-Wireless-SOS-Bu...
Then I hooked up an SDR receiver to my PC, and handle it that way. I have a 433Mhz receiver for the arduino, which could also decode those transmissions, but I never used it for anything except to prove I could.
There's a pointer to the software and some overview here:
https://blog.steve.fi/decoding_433mhz_transmissions_with_sof...
I will spend more time on that during summer when the kids are away
https://steve.fi/Hardware/helsinki-tram-times/
I do have a couple of epaper devices, and they would be ideal but for their terribly slow update-time. I like the clock showing Time & Date (or Time & Temperature), and having epaper update every second just isn't possible. Even if it were then I suspect updating so frequently would negate the potential power-saving of using epaper in the first place.
I do have a nice epaper device for showing weather & weather-forecasts, that works on batteries and updates every 20 minutes. But for trams I update departures every 2 minutes.
My idea was based on the top part of this display https://pbs.twimg.com/media/DXGxPtlXkAA0inj.jpg
In any case, the washers at my apartment complex are always off, but it's at least consistent so that I can add about 6 minutes to whatever it shows at the beginning and plan around that.
What you are probably getting when you start your washer is an estimate considering the steps of the cycle and loaded to capacity. Some machines have turbidity sensors - mine doesn't - this could deliver information that might shorten the cycle. Also the time may go up if it has to deal with suds or stops spinning to rearrange the load if it shakes.
If I could turn all the load sensing stuff off, I would. All I really want is two sliders that give me 0-100% for load size and temperature and a switch for delicate.
[0]: https://vimeo.com/87522764
https://www.itead.cc/sonoff-pow.html https://github.com/arendst/Sonoff-Tasmota https://www.home-assistant.io
The only time so far I used both the switch and the power monitor was on the Christmas tree, because I was curious how much it cost to run ($3 or so overall iirc).
Alerts to my phone if the fridge or freezer temperature exceeds a limit or the humidity under my dishwasher starts rising.
Granted, neither are fully automatic and require manual intervention.
I actually haven't shared my scraper script just because I think someone would then share the pictures (perhaps accidentally).
EDIT: I'll just add that there are no public photos of my kids and I ask anyone who takes photos not to share them outside family. So I do agree with your point.
Why do you ask?
And then there is privacy.
http://github.com/hery/accounting
I automated handling DNS updates via simple "git pushes" - Lets you revert from bad changes, and gives you a good history of changes over time - https://dns-api.com/
https://github.com/erikkaashoek/Comskip
It seems the tuning of the algorithm is complex though. There's a dedicated forum for it: http://www.kaashoek.com/comskip/viewforum.php?f=2&sid=effa4b...
Not promising anything, because each station requires time to tune and money for computational resources.
I suppose that many stations don't have such markers though.
First is to analyze the other signal features of the commercials (eg. increased volume), although it may be tricky.
The other option is a crowd-sourced solution - pretty much as for the browser adblock - where users can mark samples recognized as ads. Since the publishers often buy campaigns for many stations in the same country or state, it may be a shared database.
On the other hand, the described project only scratches my own itch. I wouldn't try to productise an app that takes away the main source of income for the radio stations.
Yes, or perhaps a combination of techniques. E.g. shared database to train an ML system to detect ads. Of course, the downside is that the ad industry will then tweak the ads until they pass the ML test.
> I wouldn't try to productise an app that takes away the main source of income for the radio stations.
I wouldn't think of it as taking away a source of income, but rather as forcing them to find a source that doesn't bother their customers so much. Ad blockers seem to be getting more accepted.
And perhaps using the clips for other purposes than "viewing" may in fact be fair use. Especially since you are trying to find a method for not viewing them.
(IANAL)
https://www.legifrance.gouv.fr/affichCodeArticle.do?cidTexte...
Nothing like a little competition to motivate the improvement of ML systems :)
No fancy ML needed, after a couple of times the filter gets one of these repeating fragments it should be able to block it. Fairness bonus: you get to hear each new ad a couple of times.
Maybe radio spots are a little different because they're cheaper and usually more low-quality than TV ads, but it doesn't really work for TV ads - they often have small variations, e.g. 10sec identical, 5sec different, 10sec identical (easy example). Also depending on your method of analyzing the audio it's sometimes broadcast with an unhearable fingerprint that distorts the waveform (let's say like MP3 versus WAV, but worse).
So yes, you can find some patterns - but the commercial breaks are highly mixed up and you wouldn't believe how many distinct commercials per channel are there, even if you think you hear the same ones all the time :)
There were previously some FM to MP3 "ripping" tools that would use the RDS information to tag the resulting recordings -- I'm not sure of the status of them. But it could provide a good way to detect commercials, since most radio stations change to a generic station identification message when they break for commercials / banter. (Whether you'd also want to turn down for banter is another question.)
Do you have similar objections to things like self-driving vehicle technology that will take away the main source of income for truck drivers?
My home theater receiver does this (Marantz). It works pretty well. It doesn't cancel out the TV commercials though, it just normalizes the volume so it matches the show. But, I assume you could make it work for muting too.
I've often toyed with the same idea of the parent poster
For some reason, the data and audio are out of sync but once calibrated it works quite well.
https://bitbucket.org/timwiffen89/radio_adblocker/src/master...
Downsides:
-I'm not actively thinking about money and hence improved investments
-I'm not monitoring credit card errors
-Comcast gradually increased increased my bill from $44/mo to $103/mo without any resistance from my side
Like you, it does mean I need to make mental notes to check into things every now and then so that there aren't charges I don't understand or investments that really seem out of whack.
Pretty much the only other thing of note I do, besides have a lawn service, is some automated plant watering because of the aforementioned traveling. For this and other reasons my schedule is variable enough that none of the usual SmartHome stuff really does anything for me even if it solved problems I felt I had, which I don't other than some remote monitoring.
I've also added various other things to it like Wifi presence notification, so it sends me a notification if a guest arrives, and hooked it up to Kodi and an Amazon Echo so I can say things like "play Iron Man" and it will turn on the TV, put on the movie and turn off the lights.
Not super useful stuff, but it was fun.
Do you share the design?
https://www.stavros.io/posts/building-cheap-home-sensorcontr...
I used an ESP8266 with cheapo sensors, the whole board costs like $5 or something. The software is pretty ad-hoc but is not more than 100 lines of code.
I also have a script which scrapes my podcast player's website, to give me a list of podcast episodes that I already played. I save this list every day in an excel file, but the script makes this much easier - I run the script, then paste the results into the excel.
-- news sites via rss with heavy keyword filtering for news that's important to me like hometown news etc (Google alerts style)
-- firmware updates from various devices that wouldn't otherwise notify me like kindle
-- posts up voted over X times in a period on smaller, specialized subreddits of interest
- daily digests of message boards I follow
- flight price alerts from persistent searches on routes I follow
- account updates via polling over websites like my frequent flyer accounts etc
I also love keyboard macros built into ios and have dozens for commonly used things I type.
[1] https://github.com/muesli/beehive
I've put some recipes on the wiki off Github. Most of them are quite teasy to string together once you just get the right extraction pattern, then they're simple WebsiteAgent -> Dedupe Agent -> (Email/Digest/Pushover Agent)
Besides that I use Home Assistant[0] with a few simple automations. For example, when the lights are on in the bathroom, and it's after dark, increase the heating/towel warmer (using Tado[1]).
Tado has its own home/away detection, plus some schedule for the heaters. It's supposed to save on energy costs.
When we're on holiday, I switch on/off various lights (with some random offset) at night... Or some times I might leave a radio on timer / connected to an IoT power switch. (I don't use any burglar alarms, cameras etc, but I try to deter them with those simple fake occupancy tools)
Philips Hue has a nice way to slowly turn the lights on. Useful to wake up during the dark winter months.
[0] https://www.home-assistant.io/
[1] https://www.tado.com/
I've wanted to do something like this for so long but it just seems so painful. How do you do this with just a little script when there's so much Javascript involved in the page, as well as random interjections like random extra security verification steps? Do you have a particularly scrape-friendly bank?
Your personal email account is probably a million times more sensitive. It contains enough personal information to get your identity stolen a thousand times over, and once your email has been compromised there is no way to ascertain the damage.
I'd be much more worried about my email getting hacked though.
Huh? The worst case scenario of your bank login details is... someone depositing all your money to their account, surely.
But... In Europe thanks to just another EU laws we should have every bank offering an API access for everyone starting from September 2018 IIRC. That would help a lot with automation, I hope generating few api keys with different access permissions would be possible with that.
I'm fully expecting banks to throw extreme security hurdles and probably registration fees in the way if they can.
Hopefully I will be wrong.
I wish they had an API but I just use a selenium script essentially. Luckily(?) logging in to this particular account only requires user/password and no fancy codes etc. I guess when the system is built with readonly default and OTP for anything else, then the login can be kept simple.
Also, there was no auto-discovery of LIFX lights and I was told I needed to re-implement the LIFX protocol from scratch because no extra dependencies were allowed in the discovery module.
Yes, and normalize it to price per kg. The product weight needs to be extracted from the item title or description in many possible formats (100G, 100 g, 100 grams), the price location and format however is uniform.
TJ is "Show text, allowing individual glyph positioning" which means strings are not necessarily in the PDF as the strings themselves, but have glyph positioning info mixed into them. This is the most common issue I've come across.
Another possibility is that the fonts have been combined/compressed/butchered, so that they're not using ASCII/Unicode (this can sometimes save space by combining bits of multiple fonts. Searching for Tj/TJ/'/" should tell you if that's the case. If it is, technically there should be a translation table somewhere in the PDF file (so that clipboard operations work in PDF viewers). Look for CustomEncoding, I think.
Particularly bad PDF creation libraries can put characters in one by one.
Characters (alone or in strings) can also be in there as octal or hex.
Finally, particularly obnoxious libraries can do it without text commands at all, but convert everything to vector drawing commands, or even worse, bitmaps.
I couldn't remember CustomEncoding, and this came up when I searched. Seems helpful: https://stackoverflow.com/questions/29467539/encoding-of-pdf...
Immediately preceding the Tj's are a lot of non-text characters that don't look like anything human-readable. Immediately following them are other other commands (Td, Tf, rg, k, etc.) which are preceded by numbers. There are no CustomEncodings. So I guess this means I can't really grep anything?
The PDF specification, while long, is pretty readable for this kind of thing. You can download the 1.7 version free from Adobe. Appendix A has a table of commands.
Then I match keywords in the strings to categorize the expenses. These days, this would be called AI :)
Edit: I use pdftotext, which has a mode that keeps the spatial structure of tables. Works for my bank.
If I'm in the kitchen I'm actively cooking or using the sink and pacing around, so the motion sensor keeps the lights on. After 1 minute of no motion, they turn off. Important to set timers though as the lights are no longer a reminder if something is in the oven or on the stovetop.
For the bathroom, I have a sensor placed precisely to view both inside and out of the shower with the curtain closed. This means the lights won't turn off while someone is in the shower unless they stand really still for 2 minutes. If someone wants to take a bath and won't move much, I can override the sensor to a longer shutoff time.
At 6:30pm I have a light that turns on to simulate that we're home. Looking to move this to a "sunset minus 30 minutes" model soon so it appears more organic.
At 2am, all of the lights turn off in case we forgot, or as a friendly reminder to go to bed. It hasn't worked today. :)
Makes walking around late at night less painful. (Children's toys end up everywhere!)
But one of the more usefull automations is a text2speach script which fires in the morning between 6 and 11, during workdays, when I'm not asleep but still at home, always 7 minutes before the next train to work leaves. This way I miss the train faar less often.
I was able to set up something I had wanted for a long time which was to turn off my TV/Stereo/Kodi after idling. I built the conditions which caused the the whole thing to shut off via Harmony hub after 10 minutes of idle after being played.
The one downside is the project moves really fast so you have to keep up with breaking changes sometimes.
Then there's a cron job that beeps on the hour (a 4-bit number from 1-12), so I can hear if my computer and watch are in sync.