I have a script that logs into my bank's web interface, and extracts my current balance. I subtract out my upcoming bills/savings transactions to tell me how much money I can spend until my next paycheck. My next step is to have it email the amount spendable every morning.
I'm using python 3, selenium, and SQLAlchemy connected to a home MySQL server.
This is a good idea and it's on my very long nice-to-have list. How are you driving the browser? Some sort of UI testing library? My bank has no API so it'd have to screen scrape.
EDIT: Sorry, I can see that it says it right there in your post: Selenium. Leaving up my stupid question as a marker of shame.
Same here, though I only store them in a plain text file.
My tip for others trying the same is to look into your bank's mobile site, if they have one. Mine was much simpler to parse, I only needed a single XPath expression to extract the number.
Also, if your bank's site is AJAX-y, look into the network tab of your browser, chances are you can find an URL that sends you a nice JSON back with the data you need.
A year ago I was lecturing mathematics and came up with a system to generate randomised coursework assignments (with corresponding markschemes, including graphs) for large classes of students (about 70 in my case). At the core of it was a python script that automated a slightly tricky latex compilation process.
not a huge automation task, but i wrote a python script which plays songs from terminal with filters like title,genre and artist etc.
https://github.com/zuck007/xplay
My brother in law has a "habit" of going to jail. All. The. Time. My wife wants to know when it happens, so I setup a script to poll the local jail inmate rosters, and send me an email when his name shows up. It works pretty well.
The company I work for ships a lot of packages each day. Shipments that are late, even ground shipments, are eligible for a refund if it is the carrier's fault. I have a python application that tracks shipments and let's me know which might be eligible for a refund. Saves us thousands each year.
There was one for a while that did this specifically. You had to give them access to your online accounts with UPS and FedEx. They would then get the refunds for you and keep/bill for 50% of the total refunded. A nice no-risk proposal.
Now, the carriers require that the shipper themselves (or whoever paid for the shipment to begin with) must claim the refund. So third party refund collections are a violation of terms of service.
There are still some very large logistics providers that claim to offer this, but I am not sure how they do this. They might actually route all shipments through them, making them the actual shipper, allowing them to collect refunds. But you have to commit to them for all logistics/shipping.
Unless you monitor it closely yourself, there is no way to get information about late shipments from the carriers. So they make it hard to get them. And not all late shipments are eligible (only if fault of carrier).
So there is value alone in being aware of what to seek refunds on. I've thought about rolling this out into a service. I'm still working on tweaking my setup.
I work for a botanical garden and spend a lot of my time working in the gift shop. We have a huge pile of t-shirts, dozens of designs each in several sizes. Most of the employees spend at least an hour going through the whole pile to figure out which shirts need to be restocked. I wrote a python script that takes an xls sales report generated by our POS system and outputs a list of all the shirts we've sold, by design, size and color. It takes me 5 minutes to go grab those shirts and put them out.
Me and my friends often share articles on our xmpp chatroom, some of them are behind paywalls and we all of use don't have subscriptions to the same websites.
So I have a parser that checks for url of paywall articles being posted. When it happens, the python script logs to the corresponding website and rehost the content of the article on my server. Finally it posts a new link — with no paywall this time — in the chatroom.
This required asking my friends for their login information on the websites where I don't have a subscription, but it made the whole process of sharing articles way easier.
Once upon a time I was writing an Android app. I wrote a simple Python script to monitor the APK file and copy it to my Dropbox when it changed so that I could easily install and test it on my phone. Uses the Win32 API, so not portable.
I use Python to control the GPIO pins on a Raspberry Pi to run my sprinklers. It just runs out of cron. I haven't polished it yet for publication, but here it is:
While I do not contribute code, I do contribute documentation and support what I can via issue triage & community support for Home Assistant (https://home-assistant.io/) which is a Python 3 open source Home Automation webapp with over 500 components.
I use Python to suck structured data out of our sales CRM's REST API and transform it into a nicely formatted narrative biweekly report in Excel for exec management. Pleasingly, it's reduced the aggregate time the sales team spends "writing about work" from about 80 hours a month to under 20. Subsequently I've extended that script to also automate the process around reminding relevant people about expiring legal agreements and the like.
Using Python 2.7, BeautifulSoup, XLSXWriter, Fuzzy Fuzzy. As a Python newbie, the hardest thing was working out how to even pick a version and an environment to start building on.
My last company used Atlassian Stash and switched from a single project and monorepo to lots of projects and repos. I wrote a python script that uses the Stash API (via the stashy library) to update or clone selected (or all, including new) repositories.
I used it fairly regularly to keep my local repos up to date, and I know one developer who used it to generate gource visualizations.
I've automated my build-test-commit workflow. My build script commits my changes at the beginning of the build. If the build or any of the tests fail, then the commit is rolled back, otherwise it is squashed together with other edits ready to be pushed. I automatically get a complete record of builds (in the reflog), and a fairly neat (if somewhat coarse-grained) history of changes with perfect traceability in my commit messages.
I would love to see what it looks like, even in a rough format. I have been meaning to do something similar, in particular for running tests + docs and pushing to a repo.
I automated my search for a houseshare by scraping http://www.flatmates.com.au and searched each description for a series of keywords. Made a potentially awful task much more pleasent.
I am a solo Sysadmin at a decently sized game server host and we have ~60 different dedicated servers. I wrote Platypus as a way of keeping track of it all. It's still definitely a work in progress (bugs are plentiful), but it now uses websockets so theres that. https://github.com/gmemstr/platypus
In my country you can apply for social housing online - but you never know when new listings will be published. So I built a scraper that used a telephony api to give me a call when a new listing was published. On a Friday night I got my automated call, looked at the listing and immediately booked a place on the queue. I was second. But the first person must have said no to the flat, so I got it. Paid very low rent for the few years that I had it. They added a captcha soon after.
Not in Python (yet), but I recently built a tiny Bash script to extract some useful information from Java Flight Recorder logs (by essentially converting them to XML and doing some XPath queries).
However, covering all corner cases in Bash is such a pain that I might just migrate it all to Python someday.
We, meaning 3 of us, have automated a greenhouse using Python code running on a raspberry pi -- this includes getting data from sensors within the greenhouse (temperature, humidity, soil moisture, soil temperature, carbon dioxide), from an accurite 5 in 1 weather station outside, and then running fans (intake and exhaust), water pumps and sprinklers, camera, etc... and pump data to web, so that we do not need to be present to see what is happening inside the greenhouse. No code is production ready, but we have had 2 good vegetables crops from the greenhouse already. The problem we working on is presented here : https://medium.com/@pravenj/a-bucket-a-day-a-hack-in-agricul...
Is Python a requirement? I mostly do my automation in bash. I use Python a lot for throwaway scripting for example if I want to go through a file and e.g. calculate/count/remove/add various points of it. It's a shortform of automation, I guess.
European Govt Bond market making including order management on inter dealer broker electronic markets, risk driven automated hedging with Eurex futures and ax driven quoting on dealer to client markets. Ten years ago. Can't share the code I'm afraid as it's proprietary.
47 comments
[ 6.2 ms ] story [ 84.2 ms ] threadI'm using python 3, selenium, and SQLAlchemy connected to a home MySQL server.
EDIT: Sorry, I can see that it says it right there in your post: Selenium. Leaving up my stupid question as a marker of shame.
My tip for others trying the same is to look into your bank's mobile site, if they have one. Mine was much simpler to parse, I only needed a single XPath expression to extract the number.
Also, if your bank's site is AJAX-y, look into the network tab of your browser, chances are you can find an URL that sends you a nice JSON back with the data you need.
You can read more about it and see the python code here: http://tug.org/TUGboat/tb37-1/tb115alsafi.pdf
My brother in law has a "habit" of going to jail. All. The. Time. My wife wants to know when it happens, so I setup a script to poll the local jail inmate rosters, and send me an email when his name shows up. It works pretty well.
https://www.reddit.com/r/Python/comments/308ucq/how_do_you_u...
If not, is anyone interested in building it with me?
I know there's a company that does something like this for parking tickets in NYC, and another one that does it for airline tickets.
Now, the carriers require that the shipper themselves (or whoever paid for the shipment to begin with) must claim the refund. So third party refund collections are a violation of terms of service.
There are still some very large logistics providers that claim to offer this, but I am not sure how they do this. They might actually route all shipments through them, making them the actual shipper, allowing them to collect refunds. But you have to commit to them for all logistics/shipping.
Unless you monitor it closely yourself, there is no way to get information about late shipments from the carriers. So they make it hard to get them. And not all late shipments are eligible (only if fault of carrier).
So there is value alone in being aware of what to seek refunds on. I've thought about rolling this out into a service. I'm still working on tweaking my setup.
So I have a parser that checks for url of paywall articles being posted. When it happens, the python script logs to the corresponding website and rehost the content of the article on my server. Finally it posts a new link — with no paywall this time — in the chatroom.
This required asking my friends for their login information on the websites where I don't have a subscription, but it made the whole process of sharing articles way easier.
It's here: https://github.com/gusmd/vigil.py
https://github.com/pjungwir/raspi-sprinklers
You can check out my automations, configuration files & docs at GitHub https://github.com/brianjking/homeassistant-config.
Using Python 2.7, BeautifulSoup, XLSXWriter, Fuzzy Fuzzy. As a Python newbie, the hardest thing was working out how to even pick a version and an environment to start building on.
Built for a friend who has to pull it manually all day at work.
https://github.com/marcelo-mason/cccbdb-geometry-parser
I used it fairly regularly to keep my local repos up to date, and I know one developer who used it to generate gource visualizations.
https://github.com/wtpayne/hiai/blob/master/a3_src/h70_inter...
There are lots of managed services to do this of course but I figured it would be a useful learning exercise. And it was
http://myarxiv.org/
https://github.com/siquick/findahouseshare
Python + BeautifulSoup
I am a solo Sysadmin at a decently sized game server host and we have ~60 different dedicated servers. I wrote Platypus as a way of keeping track of it all. It's still definitely a work in progress (bugs are plentiful), but it now uses websockets so theres that. https://github.com/gmemstr/platypus
However, covering all corner cases in Bash is such a pain that I might just migrate it all to Python someday.