One counter argument: national taxes shouldn't be used to produce code that could benefit other countries, unless that is recognised as a kind of international aid.
(Not my opinion, but one I can imagine from certain political quarters)
There are a lot of international cooperators, ESA, JAXA etc. For example, the JWST is a joint project between NASA, ESA and CSA. So it's would be remiss of them not to publish publicly (and perhaps break agreements)
The canonical counter-counter argument is that of course this needs to be established in other countries as well.
For example, this is pretty popular political demand here in Germany, as well as in the European Union. (Closely related, almost a special case: that scientific results, especially from public universities, should be open access. And all sources, code as well as collected data, should be openly available.)
It is essentially the same issue as with trading agreements, just a different solution: Instead of waiting for bilateral agreements, here single countries can make a start without having a huge disadvantage.
Care to elaborate on reasoning? National taxes IMO should be used to (produce code that) benefit citizens of that nation. Who else benefits as a consequence is irrelevant.
Doing this efficiently, especially in software with its low replication costs, often means opening your systems for the world to use (fix, improve, buy), but even if not, considering the benefit of others as a detriment to you is almost never optimal -- SW engineering is not a zero sum game.
I have heard from people that military strength (including force projection) and intelligence gathering benefits a nation.
Releasing the code (and thus methods) related to that strength would decrease some of those benefits.
I am not trying to comment on the validity of using military and intelligence gathering as tools in national benefit. I do, however, consider the US federal expenditures on military and intelligence gathering as a partial proxy for others' thoughts.
Obviously there's tension between releasing code and protecting security interests.
Maybe it'd be better to say that public money should produce public code by default and we'd put the onus on agencies and elected officials to exempt projects from the requirement.
Should the VA publish their record keeping software? Seems sensible. Should we publish the microcontroller code for ICBM guidance systems? I think the status quo of classifying that information is working out well enough.
Talking for what happens in EU right now, not everything gets released as open source. Some public funded software has restrictions in regards to which geographies it can be provided.
What matters is that most orgs should make an effort to release as much code as they possibly can.
For example, the tools to access a public API should have an official repository with tools so they are the most complete and up to date.
A positive side-effect of these public projects is finding talent. I've seen it happen that contributors to an open source project will end up working on a EU space agency because they already met other developers and tech being used there.
I hope this campaign gets some traction. Closed source is wasteful, and even more so to fund it with public money.
Software development today is quite inefficient. It mainly suffers from two problems: duplicated effort, when a solution already exists, but it is inaccessible, and misdirected effort, when a problem is solved in a way that doesn't facilitate preparing other, possibly hardly-related, solutions in the future.
The cause of duplicated effort is that closed source carries no social stigma; in other words greed is not penalized. The cause of misdirected effort is lack of understanding of how to ergonomically assemble sub-systems into systems, which is simple enough and is described in the Unix philosophy [0], but nonetheless is very rare.
> misdirected effort, when a problem is solved in a way that doesn't facilitate preparing other, possibly hardly-related, solutions in the future
Would you be able to clarify what you mean by this? In my experience, trying to write all-encompassing software quickly leads to issues with actually ever getting anything out the door in a working state - better to solve your particular problem and generalise later as needed.
In my experience misdirected effort comes more from a lack of planning or full-fat spec being produced before the work begins rather than an attempt to cram in every feature under the sun.
Instead of knowing exactly what features to put in you get a request along the line of "We need something that does x" and no time to plan how to do x, so you end up with something that does it then the feature creep starts, making the original x look like a code version of Seth Brundle after his teleportation mishap in The Fly.
The age old "Weeks of programming can save hours of planning"
Totally agree. A few pages of good spec / design docs can save so much wasted effort. Indeed I'd struggle to think of many non-trivial past projects where committing the design to paper hasn't revealed some hidden flaw that would've bitten hard later on.
> Seth Brundle after his teleportation mishap in The Fly
I think the underlying wisdom is "don't create frameworks for your solution before you have it" not "don't anticipate your first and second degree direction changes".
I meant the opposite of all-encompassing software. What I mean is largely summed up by the linked Wikipedia article, but I'll give a go at explaining it in my own words.
Given a problem, a solution can be found that explicitly composes simple routines in simple ways, or a solution that may do the same thing implicitly, but only exposes an in and an out. The first solution can be said to be following the Unix philosophy; the second follows the monolithic design [0] paradigm. The first solution is superior, in that it is "simple, short, clear, modular, and extensible code that can be easily maintained and repurposed by developers other than its creators" (quote from Wikipedia).
Generally, creating something new in the Unix style goes like this: I have `X`, I want to arrive at `Y`. The process can be summed up as a sequence of these transformations: `f1`, `f2`, `f3`. So the solution can be said to look like this: `Y = X | f1 | f2 | f3`. `f1` and `f3` are already available, possibly created by someone else while solving an entirely different problem, but the same subproblem). That means implementing `f2`. To implement this missing subroutine, the process is the same.
So the benefit of this philosophy is that it leads to pooling of problem solving resources (i.e. people), by allowing multiple problem solvers to arrive at their unique solutions by solving common subproblems (writting common subroutines).
This would specially be interesting to enable easy external auditing. Specially since governments rely a lot in large consulting companies, and these often deliver software of questionable quality. Also easier to evaluate would be the high costs usually associated with contracts to develop software sponsored by tax money.
Also those who review the open sourced code could find themselves getting job offers potentially, or find it easier to apply for jobs with those systems. The only exception I see to the whole "Public Money should produce Public Code" is in the defense sector depending on the project components. We wouldn't want to share code that is used in war with the entire world I don't think?
I've seen quite some code which was paid for by public money. If this would actually happen it would mostly just expose a tremendous shitheap of "enterprise" code created by consulting firms whose entire business model is based on delivering and "maintaining" overengineered crap with huge maintenance costs.
I work at a public institution. Our department is paid by many other institutions around the country (networks). Our code is scary and terrifying. It's 95% perl5. We run bare metal on nearly everything. We have only the barest notions of a testbed.
Put it this way: if we were required to release our code, nobody would use our services. Frankly, I'm surprised it runs at all.
(BTW, other horror stories.. $DB_password in line 200 or so. No primary keys in our DB - or foreign for that matter. We have no idea of a schema... And on and on and on.)
I work in public sector too. Some of my code could definitely released (hell, we know about tests, we have thousands of them :-))
For example, we double check a lot of persons' "status" in our programs.
We do these checks to ensure that when someone requests help (for example, financial help) for these persons, they
are entitled to receive that help.
Now that "someone" is a bunch of other institutions as well as a bunch of
private sector operators. They would benefit to see what sort of checks we do.
That would help them to understand why we sometimes refuse to help some persons, they could learn from our stuff.
That would also help other organisations (e.g. activits/NGO) to criticize the quality of our code (because what we do is quite connected to interpreting the law, thus a second opinion would definitely be useful).
Although I'm all for those two benefits, one has to realize that they are politically incorrect.
The first one because, by keeping knowledge you just make your institution more important.
The second one because a public institution must give the impression that it is perfect (because you run on public's money...).
edit: (oh and since "it" has happened in the past : this my own opinion, not reflecting the one of my employer)
Good points. I also think that PR departments would cry out loud if the bugs in publicly founded software were too easy to find. The believe in security by obscurity is just too strong there...
Their mindset usually is:
"better keep it hidden, even if we leak citizens private informations all over the internet."
"and at least our funding won't be removed because some shitty paper stumbles on those leaks and makes a big deal about it".
> If this would actually happen it would mostly just expose a tremendous shitheap of "enterprise" code created by consulting firms whose entire business model is based on delivering and "maintaining" overengineered crap with huge maintenance costs.
But that's a good thing to expose too. Might even provide a deterrent to those that are in the habit of producing bad and over-engineered work as it'll be public and potentially affect their reputation. And even then, we can probably still use that code and learn from it. Just because bad code is written with public money doesn't mean it should be kept private.
Agreed, however I think it is unavoidable simply because demand for cheap code is many times higher than demand for good code. And even if it was the other way around, I don't think the market would be able to provide it.
The code may be bad but I am not sure if in many situations it's even possible to get to a much better codebase. At least in my company the processes are often so convoluted that every attempt to automate them will result in a big mess.
I think the message should be that the business process also needs to adapt to the needs of a computer system.
> If this would actually happen it would mostly just expose a tremendous shitheap of "enterprise" code created by consulting firms whose entire business model is based on delivering and "maintaining" overengineered crap with huge maintenance costs.
They deliver the same code over and over again to different gov agencies, charge them and leach from tax payers money each and every freaking time. The problem is that politicians know this (heck they even approve it), one would need to eliminate corruption to get it solved unfortunately that's easier said than done.
The intent is good and I hope we find a good way to implement this.
I do see pitfalls though: I would separate ownership (i.e., one cannot patent the code developed with public money -- IMO a good idea) and visibility (showing the code on request -- IMO a very dangerous one).
My big concern is formalizing things that should be left informal. For example, in applied research, we usually build a number of throwaway prototypes before the real system. Those quick prototypes and proofs of concept are often ugly in many ways. If I have to expose those to the world I have to smooth sharp edges (HR -- "we cannot get this outside"), which means instead of building 10 throwaways I can only build 2-3.
> in applied research, we usually build a number of throwaway prototypes before the real system. Those quick prototypes and proofs of concept are often ugly in many ways.
So basically you just do normal development? In the commercial world, one of those prototypes usually becomes the production code.
I'd love to see this be adopted in the US, but I'm not holding my breath.
There's an example of this in my field that upsets me quite a bit. A set of tools were developed using public funds through federal research grants and other similar mechanisms, with the express purpose of making free alternatives to private tools available. After the project was successfully completed, the leaders of the project decided to start charging hundreds of dollars for "maintenance" of the toolset, which would only run on Apple devices no less. (For some tools, the cost is substantially less than private alternatives; for others it's more expensive or about the same.)
My concern about this isn't so much that they started charging money to maintain the project per se, after federal grants ran dry. My concern is that the product development costs were already paid for using taxpayer dollars, and that if it were open-sourced, it would be almost as usable, if not more usable (if it were ported to other non-Apple platforms), and free.
The other area of this that I find upsetting are these "small business grants" that basically use taxpayer funds for VC software development, with no public return. Government pays for software development, and then the developer gets to charge for it later.
Wholeheartedly agree. There is a big push do this inside the US federal government right now. I work for the United States Digital Service, and at the Department of Veterans' Affairs, almost all application code we write is open source:
How has your experience with the USDS been? From a selfish point of view, it seems like a great way to make contacts in the industry while serving your country.
I'm coming up on having been here for two years, and both of those things have been true. It's been a great experience to work alongside top-notch folks from all over the tech industry on problems that deeply impact so many lives.
It's also the best civics lesson you'll ever have. You get to see up close how the federal government runs, good and bad. There's a lot of nuance on how policy, legislation, and technology all interact that I didn't appreciate before I worked for the USDS.
It's unfortunate that large public sector orgs in the UK are going more in for AWS and their proprietry pieces such as DynamoDB though. Also more crazy when they (AWS) pay very little corporation tax and one of the orgs is in charge of collecting tax </rant>
Unfortunately this doesn't work in public education where it is most needed. All the companies pushing content to the kids, e.g. i-ready, st-math, while they may be "non-profit" do not share their code or source materials. What's worse is often their licenses only last for 1 year and have to be renewed again.
At the US federal level this already occurs. Any code written by government employees exclusively gets released to the public. Just the other day I came across the NIST biometrics matching stuff which was made open for exactly this reason.
Example:
Public of Andorra gives money to Andorra Programmers to make some cool program that all people need.
Why US Public is allowed to use that code? Andorra's budget is not that big so why people of Andorra should pay for people in States? That's a very good way to not produce Public Code at all.
There are other benefits, like US public submitting bug fixes, Andorra's code becoming a de-facto standard making it easier for them to hire contractors and support, and the fact that Andorra can use lots of open source from the US already. It's not a zero sum game.
The obvious problem with proprietary code is the questionable quality of consulting companies, etc., but I think the biggest advantage to free software is interoperability.
Right now, there are thousands of data entry jobs, whose sole purpose is to translate data from one system to another, often within the same organization.
A friend of mine who has recently learned simple Visual Basic has told me about what he calls "huge" savings simply from writing scripts to copy and transform data rather than pay people to read and type it.
54 comments
[ 3.1 ms ] story [ 29.0 ms ] thread(Not my opinion, but one I can imagine from certain political quarters)
Stop playing these silly games. If you don't pay tax, the money is guaranteed to remain unabused and it will be well spent.
For example, this is pretty popular political demand here in Germany, as well as in the European Union. (Closely related, almost a special case: that scientific results, especially from public universities, should be open access. And all sources, code as well as collected data, should be openly available.)
It is essentially the same issue as with trading agreements, just a different solution: Instead of waiting for bilateral agreements, here single countries can make a start without having a huge disadvantage.
Doing this efficiently, especially in software with its low replication costs, often means opening your systems for the world to use (fix, improve, buy), but even if not, considering the benefit of others as a detriment to you is almost never optimal -- SW engineering is not a zero sum game.
No, but some stakeholders will see it as such.
Releasing the code (and thus methods) related to that strength would decrease some of those benefits.
I am not trying to comment on the validity of using military and intelligence gathering as tools in national benefit. I do, however, consider the US federal expenditures on military and intelligence gathering as a partial proxy for others' thoughts.
Maybe it'd be better to say that public money should produce public code by default and we'd put the onus on agencies and elected officials to exempt projects from the requirement.
Should the VA publish their record keeping software? Seems sensible. Should we publish the microcontroller code for ICBM guidance systems? I think the status quo of classifying that information is working out well enough.
What matters is that most orgs should make an effort to release as much code as they possibly can.
For example, the tools to access a public API should have an official repository with tools so they are the most complete and up to date.
A positive side-effect of these public projects is finding talent. I've seen it happen that contributors to an open source project will end up working on a EU space agency because they already met other developers and tech being used there.
Make taxation voluntary or otherwise stop stealing money from the people.
Software development today is quite inefficient. It mainly suffers from two problems: duplicated effort, when a solution already exists, but it is inaccessible, and misdirected effort, when a problem is solved in a way that doesn't facilitate preparing other, possibly hardly-related, solutions in the future.
The cause of duplicated effort is that closed source carries no social stigma; in other words greed is not penalized. The cause of misdirected effort is lack of understanding of how to ergonomically assemble sub-systems into systems, which is simple enough and is described in the Unix philosophy [0], but nonetheless is very rare.
[0] https://en.wikipedia.org/wiki/Unix_philosophy
Would you be able to clarify what you mean by this? In my experience, trying to write all-encompassing software quickly leads to issues with actually ever getting anything out the door in a working state - better to solve your particular problem and generalise later as needed.
Instead of knowing exactly what features to put in you get a request along the line of "We need something that does x" and no time to plan how to do x, so you end up with something that does it then the feature creep starts, making the original x look like a code version of Seth Brundle after his teleportation mishap in The Fly.
The age old "Weeks of programming can save hours of planning"
> Seth Brundle after his teleportation mishap in The Fly
For anyone who didn't get this reference like me ... http://i43.tinypic.com/2ngr969.jpg
Given a problem, a solution can be found that explicitly composes simple routines in simple ways, or a solution that may do the same thing implicitly, but only exposes an in and an out. The first solution can be said to be following the Unix philosophy; the second follows the monolithic design [0] paradigm. The first solution is superior, in that it is "simple, short, clear, modular, and extensible code that can be easily maintained and repurposed by developers other than its creators" (quote from Wikipedia).
Generally, creating something new in the Unix style goes like this: I have `X`, I want to arrive at `Y`. The process can be summed up as a sequence of these transformations: `f1`, `f2`, `f3`. So the solution can be said to look like this: `Y = X | f1 | f2 | f3`. `f1` and `f3` are already available, possibly created by someone else while solving an entirely different problem, but the same subproblem). That means implementing `f2`. To implement this missing subroutine, the process is the same.
So the benefit of this philosophy is that it leads to pooling of problem solving resources (i.e. people), by allowing multiple problem solvers to arrive at their unique solutions by solving common subproblems (writting common subroutines).
[0] https://en.wikipedia.org/wiki/Monolithic_application
I work at a public institution. Our department is paid by many other institutions around the country (networks). Our code is scary and terrifying. It's 95% perl5. We run bare metal on nearly everything. We have only the barest notions of a testbed.
Put it this way: if we were required to release our code, nobody would use our services. Frankly, I'm surprised it runs at all.
(BTW, other horror stories.. $DB_password in line 200 or so. No primary keys in our DB - or foreign for that matter. We have no idea of a schema... And on and on and on.)
For example, we double check a lot of persons' "status" in our programs. We do these checks to ensure that when someone requests help (for example, financial help) for these persons, they are entitled to receive that help. Now that "someone" is a bunch of other institutions as well as a bunch of private sector operators. They would benefit to see what sort of checks we do. That would help them to understand why we sometimes refuse to help some persons, they could learn from our stuff. That would also help other organisations (e.g. activits/NGO) to criticize the quality of our code (because what we do is quite connected to interpreting the law, thus a second opinion would definitely be useful).
Although I'm all for those two benefits, one has to realize that they are politically incorrect. The first one because, by keeping knowledge you just make your institution more important. The second one because a public institution must give the impression that it is perfect (because you run on public's money...).
edit: (oh and since "it" has happened in the past : this my own opinion, not reflecting the one of my employer)
Their mindset usually is:
"better keep it hidden, even if we leak citizens private informations all over the internet."
"and at least our funding won't be removed because some shitty paper stumbles on those leaks and makes a big deal about it".
But that's a good thing to expose too. Might even provide a deterrent to those that are in the habit of producing bad and over-engineered work as it'll be public and potentially affect their reputation. And even then, we can probably still use that code and learn from it. Just because bad code is written with public money doesn't mean it should be kept private.
I think the message should be that the business process also needs to adapt to the needs of a computer system.
They deliver the same code over and over again to different gov agencies, charge them and leach from tax payers money each and every freaking time. The problem is that politicians know this (heck they even approve it), one would need to eliminate corruption to get it solved unfortunately that's easier said than done.
I do see pitfalls though: I would separate ownership (i.e., one cannot patent the code developed with public money -- IMO a good idea) and visibility (showing the code on request -- IMO a very dangerous one).
My big concern is formalizing things that should be left informal. For example, in applied research, we usually build a number of throwaway prototypes before the real system. Those quick prototypes and proofs of concept are often ugly in many ways. If I have to expose those to the world I have to smooth sharp edges (HR -- "we cannot get this outside"), which means instead of building 10 throwaways I can only build 2-3.
So basically you just do normal development? In the commercial world, one of those prototypes usually becomes the production code.
Just uploading a zip of the final source code is enough to count as public code imho.
Code can't be patented in EU anyway.
There's an example of this in my field that upsets me quite a bit. A set of tools were developed using public funds through federal research grants and other similar mechanisms, with the express purpose of making free alternatives to private tools available. After the project was successfully completed, the leaders of the project decided to start charging hundreds of dollars for "maintenance" of the toolset, which would only run on Apple devices no less. (For some tools, the cost is substantially less than private alternatives; for others it's more expensive or about the same.)
My concern about this isn't so much that they started charging money to maintain the project per se, after federal grants ran dry. My concern is that the product development costs were already paid for using taxpayer dollars, and that if it were open-sourced, it would be almost as usable, if not more usable (if it were ported to other non-Apple platforms), and free.
The other area of this that I find upsetting are these "small business grants" that basically use taxpayer funds for VC software development, with no public return. Government pays for software development, and then the developer gets to charge for it later.
Commercial profitability is usually a desired goal for these business grants, because the government wants their money back (via taxes).
https://github.com/department-of-veterans-affairs
This push is happening in other agencies as well. Check out https://code.gov/ for other shared projects from agencies around the federal government.
It's also the best civics lesson you'll ever have. You get to see up close how the federal government runs, good and bad. There's a lot of nuance on how policy, legislation, and technology all interact that I didn't appreciate before I worked for the USDS.
Would you mind sharing some of your insights?
Thanks for the link - I did not know they were on GH.
Why US Public is allowed to use that code? Andorra's budget is not that big so why people of Andorra should pay for people in States? That's a very good way to not produce Public Code at all.
In the more likely scenario, Andorra will use the code funded by US citizens.
Right now, there are thousands of data entry jobs, whose sole purpose is to translate data from one system to another, often within the same organization.
A friend of mine who has recently learned simple Visual Basic has told me about what he calls "huge" savings simply from writing scripts to copy and transform data rather than pay people to read and type it.