What an offensive, restrictive, and anti-free software license! Too bad - this might have otherwise been a useful piece of software. The current license renders it toxic.
The modifications (available to paying licensees) are designed to let this project be useful to those running scraping SaaS's. It lets them hook this into their proprietary infrastructures without GPL'ing their whole stack. Revisions/additions to the project itself need to be licensed as AGPL but the rest can remain proprietary.
The AGPLv3 license stands on its own without the modifications (they're optional). It's essentially a dual licensing option.
There are a limited number of ways to do open source and make a living at it. And given that I am in business independently with no investors, I have not many other options. The one path I've ruled out, at least for now, is BSD-style licensing, as that just allows SaaS operators to leverage my work, deny users freedom, and also not pay me for my time to help their commercial projects.
The GNU licenses are arguably the definition of free software. More permissive licensing just gives you the freedom to make software non-free. That's also offered for this project, but as a paid privilege.
Using neural nets and tuned heuristics, ScreenSlicer is able to intelligently find a search box, enter a query, extract the results, and page forward in the results.
This sounds like it might be usable for making a universal program to go through the idiotic, annoying, hated-by-everyone web-based sign on screens required by many free Wi-Fi hot spots.
As in, something that doesn't have a rigid template database of what UI it can deal with so the users have to beg "please support such and such hot spot in the next release".
There's nothing proprietary in the entire stack and there's no SaaS being offered here. The API is separated from the core logic to allow for clustering. You can put both API and core servers on the same machine.
Specifically, the neural nets are used in finding the parent node of the result set (or one of the parents). Heuristics (basically just generic strategies) are used for finding search boxes. There's also a similar function for finding authentication forms. Most of the functions in this project are static so it should be easy to use isolated components as needed.
The first thing I tried it on was http://hcpdirectory.cigna.com/web/public/providers, which is a typical instance of a site I might like to scrape, but still one of the simpler ones. It was not able to extract meaningful information.
That works in my test locally. The cloud servers are rather overloaded at the moment and also sometimes Tor is just flaky. Here's the request/response: http://pastebin.com/raw.php?i=KncMgbfB
edit: note that there's a couple different search forms on that page. If you run this on your own machines you can use the Form Query API which lets you specify a form ID and then set specific values for each form field (e.g., for a provider search with City/State and Name). And you can use any non-Tor proxies or a direct connection.
18 comments
[ 6.4 ms ] story [ 67.1 ms ] threadThe AGPLv3 license stands on its own without the modifications (they're optional). It's essentially a dual licensing option.
There are a limited number of ways to do open source and make a living at it. And given that I am in business independently with no investors, I have not many other options. The one path I've ruled out, at least for now, is BSD-style licensing, as that just allows SaaS operators to leverage my work, deny users freedom, and also not pay me for my time to help their commercial projects.
The more I think about it the more I like this dual licencing setup which seems a realistic way to ensure those working on OS get paid.
It lets everyone experimenting enjoy the code, whether they are a destitute student or an aspiring startup.
Meanwhile, if you are someone who wants to actually make money from this AND want to hide your own code, you've got to pay the piper.
Otherwise you end up in BSD land (or is it MIT land) where everyone takes from your project and give nothing back.
This sounds like it might be usable for making a universal program to go through the idiotic, annoying, hated-by-everyone web-based sign on screens required by many free Wi-Fi hot spots.
As in, something that doesn't have a rigid template database of what UI it can deal with so the users have to beg "please support such and such hot spot in the next release".
edit: note that there's a couple different search forms on that page. If you run this on your own machines you can use the Form Query API which lets you specify a form ID and then set specific values for each form field (e.g., for a provider search with City/State and Name). And you can use any non-Tor proxies or a direct connection.