Great post, and something I've been thinking about for quite some time. Would love to see an open source framework made available that simplifies many of the nuances necessary to maintain a "secure" user database.
Don't get me wrong, this won't solve everything, but in my opinion it's a pretty big step in the right direction.
The problem with this assertion is that this data still has to hit persistent storage somehow. Should that be in an arbitrary binary format? How should it be indexed for constant-time access? Essentially the problems that a specialized user database would need to solve would mean re-implementing exactly what generalized databases are built to do.
It's not a problem of how it's being stored. It's a problem of what is being stored.
I would suggest that the world doesn't need a specialized database for this, it needs killer libraries that make all necessary operations dead simple with any backend. And they should be dead simple to find and install.
More fundamentally, though, I'm not sure this problem is entirely solvable. Even with all the libraries and informational websites in the world, if developers don't go out and look for that information (and find the right stuff), they may never know they're doing anything wrong.
I'd be happy to have any sort of alternatives for approaching this problem. Currently there's nothing designed to give me a way to implement a lightweight hack-resistant user database.
I don't think the article is suggesting NIHing the stack all the way down. As I read the article I was imagining a layer on top of SQLite that exposes a very restricted API.
It's a completely misguided concept. Database security doesn't mean having a completely different database implementation for each type of data you're using. There are already plenty of things you can do without going crazy like that:
* Put the user database behind a microservice
* Use a different connection with different credentials and put appropriate permissions on the tables
* Store password hashes using bcrypt, scrypt or PBKDF2
Not to mention that if you write a new database, what you're actually writing is brand new exploitable bugs.
Misguided? The microservice you suggest is precisely what the post suggests. The post suggests that the code of the microservice ensures that the correct hashing is used and that the data is stored correctly.
It's puzzling that you say this is "completely misguided" and describe exactly the same thing.
I don't know if you're really thought about it, but a microservice API is a database.
A Postgres database, for example, with correctly-limited user permissions set, and a good set of views and triggers and stored procedures, is exactly equivalent in power and flexibility to putting some blobs of Sinatra API-routing and model code in front of Postgres-used-as-a-dumb-object-store.
RDBMSes were always meant to do, on their own, exactly what microservices are doing now. That's why they're so much more complicated than just being dumb object stores: you can encode within a RDBMS schema the constraints of the model, and the valid transformations you can make to any given piece of data, to create an API exactly as narrow and high-level as a given user needs.
I think it was just some quirk of their use in mainframe contexts which lead to their auth protocols never being hardened to work on the public Internet, which meant people started to believe there was something inherently insecure about letting users talk to an RDBMS.
There's not; a database, a real database, is a persistent micoservice server.
And now we're all the way around to reinventing them on top of themselves, but this time they're three separate process-spaces, random custom languages are required, and they don't understand questions phrased as relational algebra.
The article is pretty unclear on some basic concepts, such as salting and hashing wrong. This is a pretty well solved problem with bcrypt and other framework-driven identity mechanisms.
And It should encrypt data internally really gives no protection other than physical theft of the disk drive, as in if the portion of the application is compromised that can access the data, the key is available at that point, and the attacker has free run of that corner of the universe.
There is such a beast, it's called LDAP. There are any number of directory and authorization services, and they have probably been around for decades.
There are dozens of directory server options, and probably a few dozen secure authorization solutions, from Kerberos and SAML to proprietary solutions like SiteMinder. Together, these solutions give you EIAM capability that does what you described.
My uninformed perception of OpenLDAP is that it is a large, generalised enterprise class directory service that does alot of stuff. The post suggests a minimal, lightweight user database that does nothing at all else - is that the same thing as OpenLDAP?
I wouldn't characterize the magical detection of uncommon access patterns as a minimalist feature (it can be done, but not out of the box in a meaningful way), but other than that, all of the stuff you mentioned is there.
Typically, you use LDAP as a directory store, and a dedicated authentication protocol like Kerberos or SAML for authentication. You can also use LDAP for auth if desired.
In any case, don't reinvent the wheel. Just because bozos implement user auth in a MySQL table doesn't mean solutions don't exist. It just means that they are bozos. It's one of the reasons I disapprove use of SaaS apps that don't support federated identity at work.
Does OpenLDAP meet the stated need for a minimal solution that does nothing else apart from the small set of required functions?
I get the impression that OpenLDAP is a big beast that does much, much more.
I'm wanting a fast motorbike and you're pointing at a 16 wheel heavy hauler saying "see, same thing, the hauler has at least two wheels and an engine".
Note I didn't endorse OpenLDAP at all. Not sure if that's the best solution -- I'd probably steer towards 389 server or something similar with Netscape heritage.
You also said:
"Another problem is that developers roll their own user and password management systems and get things like salting and hashing wrong, making the data vulnerable."
You're asking for a distributed, secure system for processing user data and credentials. The most common systems in production globally for doing this (Microsoft Active Directory) are based on technologies developed in the 80's and only being supplanted in recent years. There's a reason for that, and if it were a trivial problem, we wouldn't have pervasive solutions using 1980s RPC in 2015 still running most businesses!
I wouldn't characterize the magical detection of uncommon access patterns as a minimalist feature (it can be done, but not out of the box in a meaningful way)
I'm not even sure about a meaningful way. For sake of portability the DB will need to have some way to dump everything. And if a server admin can do that, then someone who can get the same access as a server admin can do it, and we're back to square one.
OpenLDAP comes with a lot of optional modules to implement enterprise directory features. You don't need to use them all. You can build a minimal OpenLDAP system that's dedicated to user authentication that runs in under 4MB of RAM, with full TLS/SASL/GSSAPI security. You can't even do that with SQLite, the SQLite binary alone is 2MB and that doesn't give you any network interface or security features.
True, but having been involved in the development of an LDAP client, LDAP is a horribly complex system to implement against (and its built on top of ASN.1, which is possibly worse) and it was slow to get security features (LDAPS, etc.)—and in many cases, no two LDAP servers work quite the same (the most common LDAP server, ActiveDirectory, is the worst for this).
There is a need for something secure by default that is easy to use, easy to set up, easy to deploy, etc. Where this will be difficult is https://xkcd.com/927/.
While technically you're right (as in, LDAP is great for handling many user entries and auth[nz] correctly), in practice it's not going to happen. Nobody knows LDAP (as in, take a random sample of non-corporate developers and most likely none of them will know LDAP). It's a pain to integrate and operate in a small environment. That's why there's a call for a simple solution.
It's why everyone uses JSON and not ASN.1. Why people will save a single number in a file rather than standing up Oracle database. Or why they still use random invented rot-level "encryption" for messages rather than pgp.
LDAP doesn't expose operations like "create new user", "verify password", "deactivate user", "set encryption method to ...". It's a database where you "bind" your user description against your hierarchy of objects/attributes - explain that to a guy who just wants to get his new app idea working ASAP... It's got its own query language too. You can even do LDAP query injections - just like in SQL!
Like you said - LDAP is a beast. It's great and I'd use it as soon as number of users goes over 10k, or employees on different levels over 10. But otherwise, no way. A new component with its own security issues is just not worth it.
(Active Directory trigger warning) Even when the number of users goes over 10k I'm not sure it's worth it. I've lost days chasing down random errors to do with DNS or a misbehaving DC... when the error reference looks like http://www.selfadsi.org/errorcodes.htm (although I wish I'd known about MS's err.exe at the time)
And that's just when you're using the recommended connectors (ODBC/ADO.NET). FSM help you if you're using JDBC and the vendor of the $20k+ product hasn't tested against Server 2012.
I wonder if there's any money in providing an LDAP/AD -> REST server.
Nonsense. That page is utter tripe. The 2nd example " ( cn = jonys ) ( | (password = * ) ) " isn't even a valid filter, and a proper LDAP server will discard it. An OpenLDAP libldap-based app won't even be able to generate the request because the library will reject it.
More importantly, an LDAP search filter only goes with an LDAP search request, which is read-only. You cannot use silly parser games like this to destroy a database, like you can in SQL with Little Bobby Tables. https://xkcd.com/327/
Playing games with LDAP search filters can't return you more information than the app was originally authorized to see. It most often won't even return you more than a single record, since most apps that make LDAP queries and expect only a single entry to be returned won't even look at any subsequent entries if they happen to be in the reply.
Canonical LDAP application code to search for a user record, for authentication/authorization purposes, sets the LDAP search sizelimit to 1. Getting more than 1 matching result is an error.
Uncommon - yes. Can do less than SQL injections - yes. Read-only - yes.
But it does exist. Saying that LDAP auth[nz] code should set sizelimit to 1 is like saying SQL auth[nz] code should set LIMIT 1. Correct and defensive code does that. But there's still code out there which doesn't and is vulnerable.
Then again, I don't think you understood the second example. It's not for auth, but rather user search - which is the probably the second most common corporate ldap usage.
As for LDAP injection being nonsense, how about searching some CVEs:
- Graylog2 before 0.92 allows remote attackers to bypass LDAP authentication via crafted wildcards. - CVE-2014-9217
- PowerDNS before 2.9.18, when running with an LDAP backend, does not properly escape LDAP queries, which allows remote attackers to cause a denial of service (failure to answer ldap questions) and possibly conduct an LDAP injection attack. - CVE-2005-2301
For user search you still would set a size limit, and a server admin would set a server-side size limit to avoid other resource hogging issues.
I concede the point that there's poorly written code out there that is vulnerable. Still, to lump this into the label of "injection attack" along the lines of SQL injection attacks is grossly misleading. An SQL injection attack can cause destruction of data, invocation of arbitrary commands on the backend, and innumerable kinds of other mischief. A misparsed LDAP filter can do none of these things.
OpenLDAP is limited by LMDB, which is pretty much limited only by what your filesystem and address space can handle. For scaling out it does support multi-master and slave configurations. So yes, it should.
You can probably list the organizations (outside of China, perhaps) that have over a hundred million of users in a twitter message - (Facebook;Twitter;Google;Walmart;WhatsApp;Instagram;Hotmail;Snapchat;Yahoo;IRS[0];Blizzard;AOL;AT&T;Verizon) is probably more than 30% of the list already. And AFAIK not a single one of them uses LDAP - but assume one of them does.
So, practically speaking - does it matter?
[0] And equivalent for each country with >70M people
You're quite wrong here. Google, AT&T and Verizon on your list use OpenLDAP. OpenLDAP scales to billions of users in a single instance, and of course it distributes across multiple nodes for even greater scaling as needed. Most of the telcos around the world use it now; it scales far beyond every other data storage technology out there.
80% of the US population that owns cellphones are OpenLDAP users, whether they realize it or not. 90% of the US population that subscribes to cable TV (Comcast, Time-Warner, etc.) as well. Nothing else actually works for hundreds of millions of users. These companies know, they've tried a lot of "solutions" already.
Then I'm glad to stand corrected - I suppose I'm horribly out of date from the days they tried other "solutions". I'm less surprised about AT&T and Verizon, more surprised about Google.
My own personal experience with LDAP in general (and some with OpenLDAP in particular) dates back to 2000 or so -- and at some point I just gave up on the complex and unreliable beast that it was (for whatever reason, BDB would occasionally lock up or get corrupted) - but I know a lot has changed since, LMDB is amazing -- I guess it is time for another look.
OpenLDAP didn't start to become seriously usable until about 2003 (that's when HP dropped SunDS and adopted OpenLDAP for all of their global infrastructure). A lot of work went into it in the 2001-2002 timeframe to get it there. Yeah, a lot has changed, you've missed a couple generations of advancements. BDB's complexity was definitely a pain, but that's all in the past.
I'm pretty sure that with the appropriate infrastructure, The answer is yes. From a scaling limits POV, AD can support billions of objects. Pretty sure the Netscape derivatives (Oracle, the former Sun Directory Server, Red Hat) have implementations that large. I've worked with one with 40M user accounts and we weren't near any scaling limits.
"What developers need is a minimal, single purpose database specifically designed for protecting user information and designed to move user data access away from the rest of the application data to minimise the impact of access by hackers." - Yea this sort of homogenization of security worked great for OpenSSL
I'd argue that it worked exactly as expected. The bug was found, fixed, and patches released immediately. For most people that was just a package-manager update away.
If anybody imagined that openssl had no bugs, they've obviously been proven wrong several times already, and I'm sure they will be proven wrong again.
The point of a single library (or few libraries) is (at least) twofold. It's much less likely to have defects, and when those defects that remain are found, answering the "does this affect me" question is easier and prompts quick action.
This entry really boils down to separating user data from other data. But some of the items I don't think work in real usage (or at least I would like some clarification if possible).
> It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed.
Constrained in what ways? Especially in management / support roles you may need to do bulk actions against users (full listing, bulk changes, etc) so I'm not sure what constraints would be compared to a normal database.
> Its API should have password salting and hashing built in.
I disagree; I think password hashing should happen before the data hits the database.
> Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data.
Does someone have an implementation idea for this? Depending on the type of user data we're talking the data could be very small. So the throttling may have to be quite significant but even then what's to prevent the attacker from bypassing it by using multiple connections? Multiple threads that each request a subset? I don't think you can get around those and still have a performant system.
> It should encrypt data internally.
How? Does it encrypt using a passcode specified by the system administrator (which means it exists somewhere on the system or nearby system making it decrypt-able anyway)? Or does this mean encryption using the user's password thus separating the data? If so how do you handle the cases when administrators need to modify the user's information for support or other reasons?
I’m working on an application where we have separated the Authentication concerns from the rest of the user data. When a user is registered, we generate a UID and communicate that with the other services. We’re going to be using common role terminology across the services, but Authorization is up to each service in relation to those roles.
Our needs are a little different as well, because each service may need to reach out to a third-party service and we are holding a bearer token for the third-party service on behalf of the user. We provide that to the services behind an auth middleware through a header.
(We had considered changing this from an Auth Service to a general User Service, but I am strongly leaning toward having a separate User Service and leaving Auth tightly constrained.)
Right, there's a few classes of data that can see that would do well to separate from the database that a website may have unfettered access to.
1) Authentication data
(login id?, password)
2) Personal data that is not used in an online manner (depends on the site of course)
(full name, email address, etc.)
3) Data that shows up on the site
If this data is segregated in to separate silos with no 'read everything' apis exposed to the web servers, you might stand a chance of not leaking all of your users data when the web servers are compromised. This of course will not help you if the authentication servers are also compromised.
> Especially in management / support roles you may need to do bulk actions against users (full listing, bulk changes, etc) so I'm not sure what constraints would be compared to a normal database.
I was imagining a social-network use case while reading the post, so when a similar objection came to mind, the answer seemed obvious: make "users" and "accounts" separate things. The business-logic keeps around its own "users" table. The business logic can, and must, index that.
Each user then HAS A account; there's a 1:1 correspondence. The accounts are in a separate service (a AAA microservice, effectively) that exposes this minimal API where you can only talk about individual accounts to it.
How does a backoffice user-management dashboard do its job? Like the name suggests, by reading off the users table—and then doing O(N) queries to the accounts service to effect changes. These sort of one-off administrative operations are allowed to be costly like that.
Welcome to the world of IAM (identity and access management). There are many solutions to the above stated problem. If you don't absolutely need to store it don't. That includes passwords, ssn, dob, or anything of the sort.
There are a ton of services you can federate with, it's easier for the user less passwords to remember.
If you really want users to authenticate natively. Take a look at one of the new-er players out there Storm Path. It's basically your IAM backend to-go. Don't write your own security if you don't have to :)
> If you really want users to authenticate natively. Take a look at one of the new-er players out there Storm Path. It's basically your IAM backend to-go. Don't write your own security if you don't have to :)
But depend on an NSA aware third party to store your client's credentials?
Easier said than done. User data is needed in some way, shape, or form in almost any transaction the system performs. Yes you can separate the authentication credentials, and the sensitive attributes from the more routine stuff, and that's a good thing, but things often don't break that cleanly.
I agree with OP that the user authentication data should be segregated, simple, and very well protected. It should act like a hardware security module: it will authenticate users, but never give up their keys.
I tried to find decent hashing in OpenLDAP and Apache DS but didn't. Perhaps I didn't look hard enough. Do they throttle by default?
Who's gonna make LDAP webscale and hipster friendly?
I disagree. I think millions of accounts are compromised because users are required give up too much information and can't verify that their user data is stored in a secure way. The solution to this is on the client side, in the browser, not in the server.
The core proposition of this article is that if we make it really easy to store user data correctly, people will do it. This proposition is simply not borne out by the data. Many parts of user security are easy. Take password storage, for example: it's so easy a caveman could do it. Every major languge has a library that takes a password and a salt and a number of iterations and runs it through pbkdf2 or scrypt. Many have these functions builtin. But do people do it? No. It seems like once a month some company exposes X-thousand passwords because they stored it in plaintext or MD5. And I bet a significant portion of HN readers think SHA1 is a valid way of storing passwords, eager to sacrifice security as efficiently as possible. pbkdf2 and scrypt aren't hard to use, people are just idiots.
Making a specialized database just gives people another easy thing to not use.
Ultimately there's no reason to be giving your password to a service in the first place. You only should be giving them cryptographic proof that you have your password, not giving them your password. This is possible with zero-knowledge password proofs[1], but it would require browsers and other client-side software to implement a ZKPP algorithm. This would force servers to either implement a ZKPP algorithm correctly or expose their incompetence to their users. Similar algorithms can be hypothesized for many other user data needs, but passwords would be an excellent start.
I wonder why is the call for a dedicated database, rather than for a simplified API instead? Many frameworks provide it already, but base it on the existing database which reduces the integration issues.
For example Symfony has FOSUserBundle [1] which provides nice support for most listed things. Django comes with its auth module [2]. There are more examples like that. They allow easy auth / user management without touching any of the underlying details. And once you actually want to go the direction of LDAP/kerberos/..., it's usually just a matter of switching the backend implementation for something that already exists.
1. s/specialized/separate/
2. Use SASL, or another authentication protocol to talk to your auth DB.
3. Profit!
If you're trying to solve a fundamental problem like authentication, odds are someone else has already spent a lot of time thinking up a good solution.
Re: LDAP
This may not be as good as a specialized solution, but it's probably a step up from throwing all your data into the same MySQL DB. ;-)
It's not particularly hard, either, just different from what you may be used to.
60 comments
[ 3.4 ms ] story [ 120 ms ] threadDon't get me wrong, this won't solve everything, but in my opinion it's a pretty big step in the right direction.
It's not a problem of how it's being stored. It's a problem of what is being stored.
I would suggest that the world doesn't need a specialized database for this, it needs killer libraries that make all necessary operations dead simple with any backend. And they should be dead simple to find and install.
More fundamentally, though, I'm not sure this problem is entirely solvable. Even with all the libraries and informational websites in the world, if developers don't go out and look for that information (and find the right stuff), they may never know they're doing anything wrong.
P.S. Hello again, alter ego.
https://github.com/wesabe/grendel
From HN front page to solution within hours!
If it said "built with Golang" or "built with Rust" it would certainly hit the front page.
* Put the user database behind a microservice
* Use a different connection with different credentials and put appropriate permissions on the tables
* Store password hashes using bcrypt, scrypt or PBKDF2
Not to mention that if you write a new database, what you're actually writing is brand new exploitable bugs.
It's puzzling that you say this is "completely misguided" and describe exactly the same thing.
A Postgres database, for example, with correctly-limited user permissions set, and a good set of views and triggers and stored procedures, is exactly equivalent in power and flexibility to putting some blobs of Sinatra API-routing and model code in front of Postgres-used-as-a-dumb-object-store.
RDBMSes were always meant to do, on their own, exactly what microservices are doing now. That's why they're so much more complicated than just being dumb object stores: you can encode within a RDBMS schema the constraints of the model, and the valid transformations you can make to any given piece of data, to create an API exactly as narrow and high-level as a given user needs.
I think it was just some quirk of their use in mainframe contexts which lead to their auth protocols never being hardened to work on the public Internet, which meant people started to believe there was something inherently insecure about letting users talk to an RDBMS.
There's not; a database, a real database, is a persistent micoservice server.
And now we're all the way around to reinventing them on top of themselves, but this time they're three separate process-spaces, random custom languages are required, and they don't understand questions phrased as relational algebra.
thanks
It should not provide generalised database query functionality. YES - it only proivdes the subset of LDAP functionality necessary for authentication.
Its API should have password salting and hashing built in. NO - hashed but needs salting&scrypt.
Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data. NO - todo.
It should encrypt data internally. YES (hashed not encrypted).
It should communicate only over encrypted connections. YES.
It should be distributed. YES.
It should not be run on any web server, should run "behind the scenes" and be accessible only via its API. YES.
It should include triggers and alerts based on uncommon access patterns or recognised nefarious access patterns. NO - todo.
It should have no other purpose. YES.
And It should encrypt data internally really gives no protection other than physical theft of the disk drive, as in if the portion of the application is compromised that can access the data, the key is available at that point, and the attacker has free run of that corner of the universe.
There are dozens of directory server options, and probably a few dozen secure authorization solutions, from Kerberos and SAML to proprietary solutions like SiteMinder. Together, these solutions give you EIAM capability that does what you described.
I wouldn't characterize the magical detection of uncommon access patterns as a minimalist feature (it can be done, but not out of the box in a meaningful way), but other than that, all of the stuff you mentioned is there.
Typically, you use LDAP as a directory store, and a dedicated authentication protocol like Kerberos or SAML for authentication. You can also use LDAP for auth if desired.
If you want to go deep on identity management, the NIST documents are pretty much the canonical source of information. http://csrc.nist.gov/projects/iden_ac.html
In any case, don't reinvent the wheel. Just because bozos implement user auth in a MySQL table doesn't mean solutions don't exist. It just means that they are bozos. It's one of the reasons I disapprove use of SaaS apps that don't support federated identity at work.
I get the impression that OpenLDAP is a big beast that does much, much more.
I'm wanting a fast motorbike and you're pointing at a 16 wheel heavy hauler saying "see, same thing, the hauler has at least two wheels and an engine".
You also said:
"Another problem is that developers roll their own user and password management systems and get things like salting and hashing wrong, making the data vulnerable."
You're asking for a distributed, secure system for processing user data and credentials. The most common systems in production globally for doing this (Microsoft Active Directory) are based on technologies developed in the 80's and only being supplanted in recent years. There's a reason for that, and if it were a trivial problem, we wouldn't have pervasive solutions using 1980s RPC in 2015 still running most businesses!
If you want a secure system you shouldn't be prioritising a "minimal solution". As the previous poster said, don't reinvent the wheel.
I'm not even sure about a meaningful way. For sake of portability the DB will need to have some way to dump everything. And if a server admin can do that, then someone who can get the same access as a server admin can do it, and we're back to square one.
It supports a lot of backends (SQL and NoSQL alike) (openLDAP has nice proxy features)
It also supports role based authentication.
But at the opposite of actual NoSQL data bases, even though it has no structures, it is strongly typed and secure.
You map your NoSQL structures to structured backend through a mapping.
It is awesome but most devs hates it, because they are (at my opinions) ignorants. And developers hates it because they are lazy.
There is a need for something secure by default that is easy to use, easy to set up, easy to deploy, etc. Where this will be difficult is https://xkcd.com/927/.
It's why everyone uses JSON and not ASN.1. Why people will save a single number in a file rather than standing up Oracle database. Or why they still use random invented rot-level "encryption" for messages rather than pgp.
LDAP doesn't expose operations like "create new user", "verify password", "deactivate user", "set encryption method to ...". It's a database where you "bind" your user description against your hierarchy of objects/attributes - explain that to a guy who just wants to get his new app idea working ASAP... It's got its own query language too. You can even do LDAP query injections - just like in SQL!
Like you said - LDAP is a beast. It's great and I'd use it as soon as number of users goes over 10k, or employees on different levels over 10. But otherwise, no way. A new component with its own security issues is just not worth it.
And that's just when you're using the recommended connectors (ODBC/ADO.NET). FSM help you if you're using JDBC and the vendor of the $20k+ product hasn't tested against Server 2012.
I wonder if there's any money in providing an LDAP/AD -> REST server.
http://stackoverflow.com/questions/2453769/active-directory-... might be an idea, too.
Unless you escape your "(cn=$username)" properly, the users can do a lot of fun things.
More importantly, an LDAP search filter only goes with an LDAP search request, which is read-only. You cannot use silly parser games like this to destroy a database, like you can in SQL with Little Bobby Tables. https://xkcd.com/327/
Playing games with LDAP search filters can't return you more information than the app was originally authorized to see. It most often won't even return you more than a single record, since most apps that make LDAP queries and expect only a single entry to be returned won't even look at any subsequent entries if they happen to be in the reply.
Canonical LDAP application code to search for a user record, for authentication/authorization purposes, sets the LDAP search sizelimit to 1. Getting more than 1 matching result is an error.
But it does exist. Saying that LDAP auth[nz] code should set sizelimit to 1 is like saying SQL auth[nz] code should set LIMIT 1. Correct and defensive code does that. But there's still code out there which doesn't and is vulnerable.
Then again, I don't think you understood the second example. It's not for auth, but rather user search - which is the probably the second most common corporate ldap usage.
As for LDAP injection being nonsense, how about searching some CVEs:
- LDAP injection and resulting STORED Cross Site Scripting in Events Planner - SmarterMail 7.x (7.2.3925) - https://www.exploit-db.com/exploits/15189/
- Graylog2 before 0.92 allows remote attackers to bypass LDAP authentication via crafted wildcards. - CVE-2014-9217
- PowerDNS before 2.9.18, when running with an LDAP backend, does not properly escape LDAP queries, which allows remote attackers to cause a denial of service (failure to answer ldap questions) and possibly conduct an LDAP injection attack. - CVE-2005-2301
And many many more...
I concede the point that there's poorly written code out there that is vulnerable. Still, to lump this into the label of "injection attack" along the lines of SQL injection attacks is grossly misleading. An SQL injection attack can cause destruction of data, invocation of arbitrary commands on the backend, and innumerable kinds of other mischief. A misparsed LDAP filter can do none of these things.
I can find no clear answer after a few minutes of searching.
So, practically speaking - does it matter?
[0] And equivalent for each country with >70M people
http://symas.com/docs/2014-LOADays-jkozyra.pdf
80% of the US population that owns cellphones are OpenLDAP users, whether they realize it or not. 90% of the US population that subscribes to cable TV (Comcast, Time-Warner, etc.) as well. Nothing else actually works for hundreds of millions of users. These companies know, they've tried a lot of "solutions" already.
My own personal experience with LDAP in general (and some with OpenLDAP in particular) dates back to 2000 or so -- and at some point I just gave up on the complex and unreliable beast that it was (for whatever reason, BDB would occasionally lock up or get corrupted) - but I know a lot has changed since, LMDB is amazing -- I guess it is time for another look.
If anybody imagined that openssl had no bugs, they've obviously been proven wrong several times already, and I'm sure they will be proven wrong again.
The point of a single library (or few libraries) is (at least) twofold. It's much less likely to have defects, and when those defects that remain are found, answering the "does this affect me" question is easier and prompts quick action.
> It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed.
Constrained in what ways? Especially in management / support roles you may need to do bulk actions against users (full listing, bulk changes, etc) so I'm not sure what constraints would be compared to a normal database.
> Its API should have password salting and hashing built in.
I disagree; I think password hashing should happen before the data hits the database.
> Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data.
Does someone have an implementation idea for this? Depending on the type of user data we're talking the data could be very small. So the throttling may have to be quite significant but even then what's to prevent the attacker from bypassing it by using multiple connections? Multiple threads that each request a subset? I don't think you can get around those and still have a performant system.
> It should encrypt data internally.
How? Does it encrypt using a passcode specified by the system administrator (which means it exists somewhere on the system or nearby system making it decrypt-able anyway)? Or does this mean encryption using the user's password thus separating the data? If so how do you handle the cases when administrators need to modify the user's information for support or other reasons?
Our needs are a little different as well, because each service may need to reach out to a third-party service and we are holding a bearer token for the third-party service on behalf of the user. We provide that to the services behind an auth middleware through a header.
(We had considered changing this from an Auth Service to a general User Service, but I am strongly leaning toward having a separate User Service and leaving Auth tightly constrained.)
1) Authentication data (login id?, password) 2) Personal data that is not used in an online manner (depends on the site of course) (full name, email address, etc.) 3) Data that shows up on the site
If this data is segregated in to separate silos with no 'read everything' apis exposed to the web servers, you might stand a chance of not leaking all of your users data when the web servers are compromised. This of course will not help you if the authentication servers are also compromised.
I was imagining a social-network use case while reading the post, so when a similar objection came to mind, the answer seemed obvious: make "users" and "accounts" separate things. The business-logic keeps around its own "users" table. The business logic can, and must, index that.
Each user then HAS A account; there's a 1:1 correspondence. The accounts are in a separate service (a AAA microservice, effectively) that exposes this minimal API where you can only talk about individual accounts to it.
How does a backoffice user-management dashboard do its job? Like the name suggests, by reading off the users table—and then doing O(N) queries to the accounts service to effect changes. These sort of one-off administrative operations are allowed to be costly like that.
There are a ton of services you can federate with, it's easier for the user less passwords to remember.
If you really want users to authenticate natively. Take a look at one of the new-er players out there Storm Path. It's basically your IAM backend to-go. Don't write your own security if you don't have to :)
But depend on an NSA aware third party to store your client's credentials?
I tried to find decent hashing in OpenLDAP and Apache DS but didn't. Perhaps I didn't look hard enough. Do they throttle by default?
Who's gonna make LDAP webscale and hipster friendly?
The core proposition of this article is that if we make it really easy to store user data correctly, people will do it. This proposition is simply not borne out by the data. Many parts of user security are easy. Take password storage, for example: it's so easy a caveman could do it. Every major languge has a library that takes a password and a salt and a number of iterations and runs it through pbkdf2 or scrypt. Many have these functions builtin. But do people do it? No. It seems like once a month some company exposes X-thousand passwords because they stored it in plaintext or MD5. And I bet a significant portion of HN readers think SHA1 is a valid way of storing passwords, eager to sacrifice security as efficiently as possible. pbkdf2 and scrypt aren't hard to use, people are just idiots.
Making a specialized database just gives people another easy thing to not use.
Ultimately there's no reason to be giving your password to a service in the first place. You only should be giving them cryptographic proof that you have your password, not giving them your password. This is possible with zero-knowledge password proofs[1], but it would require browsers and other client-side software to implement a ZKPP algorithm. This would force servers to either implement a ZKPP algorithm correctly or expose their incompetence to their users. Similar algorithms can be hypothesized for many other user data needs, but passwords would be an excellent start.
[1] https://en.wikipedia.org/wiki/Zero-knowledge_password_proof
LDAP can be painful to work with, even when using such "industry standards" like Active Directory
I think a specialised server (or even Postgres extension) could be useful but couldn't you set up the following?
* Create a stored proc or similar that does this: SELECT COUNT(PasswordHash) WHERE Username = @Username AND PasswordHash = @PasswordHash
* Create a user account on the database that only has EXEC permissions, not SELECT or anything else
* Restrict the user account to only be able to log in from the app servers (or whatever)
* Restrict the DB admin to only login via a jump box / VPN.
For example Symfony has FOSUserBundle [1] which provides nice support for most listed things. Django comes with its auth module [2]. There are more examples like that. They allow easy auth / user management without touching any of the underlying details. And once you actually want to go the direction of LDAP/kerberos/..., it's usually just a matter of switching the backend implementation for something that already exists.
[1] https://github.com/FriendsOfSymfony/FOSUserBundle [2] https://docs.djangoproject.com/en/1.8/topics/auth/default/
Re: LDAP
This may not be as good as a specialized solution, but it's probably a step up from throwing all your data into the same MySQL DB. ;-)
It's not particularly hard, either, just different from what you may be used to.