I'd venture to say most certifications in general are junk, CYA processes. If you are basing your hiring decision on the letters someone puts after their name, you're doing it wrong.
The same could probably be said for a degree, particularly with the current snowflake "everyone must pass" mentality at so many schools.
Certifications and degrees are nice, but neither guarantees a person is competent in any field.
I feel this way about certifications in general. I work with tons of Microsoft and Cisco certified people and the basic computer science errors they make has be doubting the value of those certifications.
The problem is, you can't tell the difference between the person that did the time, studied, did labs, etc... And the one that downloaded a test answer brain dump and memorized the answers.
And even in the case of the former, you can't cram 20 years of knowledge and experience into a certification.
I have just an MCITP that I had to get as a job requirement. My employer paid for a 2-week bootcamp. I did very little studying and already knew 90% of what was covered.
I have about 10 people around me that have various combinations of certifications and degrees that all use me as a lifeline when they get stumped.
I mean simple mistakes, like not understanding that information on an air gapped machine is unknowable to another machine on the network.
"If you took the tax id and social from 'airGappedMachine'"
"It has no connection to any other machine"
"Just query the database"
"Store it on a thumbdrive and walk it over? It changes quite often I don't think that's a good workflow"
"No, just query it"
"How?"
"SQL!"
Or the classic:
"That password only has N characters, I can crack that in Y seconds"
"You are the Administrator of that box, you can simply reset the password and do whatever you like."
"Yeah but the hackers.... Y seconds"
"It's not Y seconds for them, if they already are an Administrator or have read database access it's game over anyway they have to use the API and that locks you out after 3 attempts. Also your are assuming a much easier hashing method than is actually in use."
"Yeah but my calculator says Y seconds so it's Y seconds"
I think soon that this sentiment will start to apply to Universities. It seems inevitable at some point in the near future there will be an online 'university' (for lack of a better word) who's graduates will be considered equal or even better than a standard university education, particularly for tech related degrees.
Universities have been a centralized source of accreditation for a long time. All it takes is for someone to figure out how to restrict graduation and filter good candidates using testing or some other means to gain accreditation and acceptance of its graduates by industry.
That's exactly why a lot of bootcamps have come into existence, along with a guaranteed job in the industry at the end of it. Though many employers hire university grads as a sort of "signal" for people who can work hard, think critically, finish what they started etc.
And I'm not saying one is better then the other, just noticing this trend of bootcamps popping up everywhere to replace university CS/CE education.
anecdotally most people I know who have gone through bootcamp have had major issues getting hired and the ones that did were hired into support/saleseng rather than software engineering.
I've seen the same as well, more so that most of them were taught one structured way to look at problems and only how to use specific tools, rather than why. There's definitely tradecraft learning necessary beyond a bootcamp.
Western Governors University is a choice in the US, too. I am currently attending, and it's different from any other college I have gone too. All of the classes are competency based and self-paced, meaning theoretically you can get a Bachelor's degree in 6 months.
It already has for me, I won't hold it against someone but it no longer indicates any basic level of knowledge.
This is based on my last batch of interns that had masters degrees but couldn't handle hello world. Their spoken english skills made it clear that they were completely incapable of understanding the lecturers.
Universities are a business, they are paid a lot to provide a piece of paper, so they provide it.
There's 'compliance security' and then there's 'street-smart security'. They are very different things.
Most organizations aim for compliance (it's cheap and easy). They base security on contracts, certs and insurance policies.
Street-smart security practitioners are appalled by this. And, management doesn't understand why the 'security people' aren't on-board with 'compliance'.
It's a lot like the old west with Cowboys and Indians. Two totally different world views.
IMHO, I dont think it will pan out this way. Companies are being breached, and neither their marketshare nor profits are being significantly affected.
Aside from https://www.dailydot.com/layer8/code-spaces-hacked/ -- how many other companies do you know which really paid a significant price for being hacked?
Yes, Anthem/BCBS, Target, HD, Sony, etc, etc have all had losses.. but they really havent been long-term impacted it seems.
I dont know what the answer is, this sucks hard as both a consumer and an infosec person. I tend to view security as a "hidden performance" factor. As long as the security flaws don't inconvenience the paying customers too much, they simply don't care if they exist or not.
Isn't compliance the reason we have "at least one upper and lower case letter, at least one number, at least one special character and a drop of blood from your first born" style passwords that ruin security?
It doesn't seem like the certs themselves are causing more harm than good, it seems like the lack of evaluation employers do for potential employees is the real issue. Giving too much credence to essentially any degree is problematic, I don't think security certs are exceptional, in this case.
I was involved once in a criminal forensics case. The defense's "expert" witness was a one man computer shop. He had created his own "certifications" and listed them on his resumé as indications to the court of his suitability as a witness.
It was literally "person's-company-name Certified Forensic Examiner".
He had created about 6 certifications, all of which he held.
It's kinda funny, but also kinda scary that the court accepted this as proof of his qualifications. The prosecution never raised an objection to it either.
So he got the court to accept a self-signed cert as a trusted root.
I don't see how that's much different than asking someone to solemnly swear they are telling the truth, when most humans are as capable as lying about whether or not they are truthful as they are of lying about anything else.
If the court has no one capable of gauging the expertise of a witness, it has to trust in someone to do that for them, and if neither party in the case objects to the witness certifying himself as an expert, it has no reason to gainsay that assertion. It's really the prosecutor's failure alone, for letting that detail slip past.
The court has plenty of people capable of gauging the expertise of the witness, just not at that level. Impeaching credibility is not the job of the court - it is the job of the opposing party.
If the other side of the case had an expert that didn't bring up the lack of credentials, or the litigators on the file didn't bother to do the most cursory of credibility impeachments, that's on them.
There is something very different about telling the truth and having knowledge.
Everyone can tell the truth, but not everyone knows technical details about something. He is using his certifications to show he knows technical details; he might even believe he knows those things, but he very well could be wrong.
Hell, he may even be right. He might be a real expert. He doesn't make his claim to know things any more credible by way of vouching for himself even if he's an actual expert.
Certifications sometimes set a terrible baseline, but at least it's an independent baseline.
I think requiring someone to swear to tell the truth is more a way to nail someone for perjury later than it is to make them tell the truth. It's like those immigration forms where they ask you if you have ever performed an act of terror (or somesuch). Nobody in their right mind is going to check the YES box, but if they find out later that you lied, it's much easier to charge you and jail or deport you.
There is a huge problem in IT. It's not certifications. It's the totally illogical bias against certifications. There's no reason someone can't have both skills and certifications, but everyone treats them as mutually exclusive. Certs help with administrative things like HR requirements, contractual obligations, audits, etc... No, those things do not make one secure, but running a business is not only about being secure. The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.
I had to get the CEH when I was hired on by a security consulting firm back in Northern Virginia.
I now do not include this cert on my CV for (perhaps irrational) fear that someone in some HR department may think "Oh, no! This guys is a hacker!" Sadly, the word has a negative connotation, because the word "cracker" or "bad actor" never bubbled up past the IT security world.
I also do not include my military service dates, as they reliably peg my age. Most men join at 17 or 18, so they would immediately know my age.
I'm debating whether to include any certifications at all, just include my degree.
Yeah, when I started years back I had the Security+ as it was a requirement by my university at the time. During an interview I had the technical manager ask me about it and openly wonder why I would waste my time with such a useless certification. I didn't get that job, and I still wonder if it's because he thought less of me because of a certification I was required to obtain in order to graduate.
I leave my certifications off my resume now, unless the job posting asks for certifications which is frustrating trying to decide if adding this cert to my resume will help or hurt me in the interview.
The problem is people outside the field of whichever cert pile into them as a means to break in, so the median cert holder winds up being inexperienced and overall unqualified. The cert itself then becomes associated with that. You're not wrong that its silly to be biased against, in the larger context of someones qualifications, but only certs that are sufficiently exclusive will get any respect and that probably isn't going to change.
What you are saying points out problems with contracts, audits, HR. Audits are supposed to meaningful. They aren't supposed to be a waste of time that exists to check off a box. HR is supposed to add value, not subtract. Counterparties should ask for things in contracts that actually benefit them.
From a certain point of view you are right -- an IT manager should cooperate with other facets of the business in order to help the organization succeed, not obstruct because he thinks their requirements are dumb. But from a larger point of view, either organizational or super-organizational, dumb requirements shouldn't exist. To the extent that they do CEOs, industry groups, and/or governments should be modifying them to keep them relevant.
Technology changes so fast that most of this tests are obsolete by the time you take them. The best technologists rely on their knowledge, so they spend their time and efforts staying current. On the other hand, you have weak applicants who know a certification is their only way to a job so they put their efforts into it.
> Certs help with administrative things like HR requirements, contractual obligations, audits, etc.
You're pointing out things that are generally considered to be failures in our sphere anyway. I don't think that's unrelated.
HR brings us terrible candidates? Certs (currently) don't help that, and when HR over emphasizes them, we can blame the certs. (In theory, certs COULD help, but the people that get the cert instead of the experience are the problem ones, and those are the people HR brings us)
Contracts are an attempt to preset agreements between two parties, and in the tech world usually one of those parties are at a disadvantage in the tech space (otherwise we'd not be making the contract). Certs are used as a means of asserting competence in the employees, but the other party has every incentive to focus on the cert instead of the competence. So when it goes bad (as it often does), we can blame the certs as not achieving the goal.
Audits are likewise, trying to filter for competence (in action or in people, depending). But again, if the incentive is for the cert over the competence, it is the competence that suffers. We blame the cert (or more correctly, the emphasis on the cert).
IT definitely has a bias against certs. And it's not really against the certs themselves...it's that emphasis of the certs is harmful, not helpful. So we argue against that emphasis, which sounds a lot like hating the certs.
> The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.
But those business issues are "how do we ensure the technical issues are addressed?" And it turns out Certs in practice do a terrible job at that.
So how would you approach scaling the IT Security industry without some form of industry certification process?
I'm definitely not trying to defend the CISSP/CEH style certs here but I don't see how you reliably expand the industry at scale without some form of certification process.
A company hiring it's first security person or a company trying to hire a lot of security people, need some form of base benchmark to work from, just like with most other professions (e.g. Law, accountancy, architecture etc)
What does "certification" have to do with scaling the industry? Training and nurturing talent is a hard problem, but expensive tests don't do anything to mitigate that problem.
Certifications are used by many industries to provide a demonstration of a common baseline level of knowledge and experience, so that each individual person doesn't need to be assessed by each hiring organisation.
For example Certified accountants, Lawyers etc.
Without some common baseline, how do people looking to hire security types who don't have the experience to assess their skills and knowledge avoid getting bad people?
Also the article's argument that "it's experience that counts" really doesn't help get new people into the industry, where do they get the experience in the first place, it's a catch-22
so, out of curiousity, that implies to me that you don't rate any IT security certifications?
So would I be right in thinking you don't think that any of the Offsec certs (OSCP/OCSE), CREST certs (CCT etc) or SANS certs are usful?
Also, and I'd be genuinely interested to hear your thoughts here, why do you think that IT/Info Sec will take a different path than other professions (medicine, law, accountancy, engineering etc) which fairly universally have evolved into a certified professional model?
The "certifications" in medicine and law accompany postgraduate degrees and are far, far more recognized than the random certificates you listed, some of which are profit-making enterprises from for-profit companies.
Ah ok, so would it be fair to say that you're not opposed to the concept of certifiation, per se, but that you're not a fan of existing options in the field?
Of course one problem is "how does a certification become recognized", I mean in IT security it's going to have to start somewhere...
In the UK the IISP are perhaps closest to the "traditional profession" certifications, but they're struggling a bit to get traction.
You can't wish professionalism into being. You have to build a profession. We're not there yet with any aspect of information security. The hard work of defining the field and its requirements has not yet been done. No organization currently extant on this planet has any business pretending that they know the answers to these questions, let alone charging money to take tests about them.
Obviously it takes time to build a profession, but you've got to start somewhere, and part of that path is certification.
Unfortunately the industry is growing far faster than perhaps happened for previous emergent professions, so the time needed to slowly grow professional bodies isn't available.
If it's not commercial organisations that start providing those services, the only other options I can see are some form of union, or some government mandated body. Those are options, but both have their challenges.
No, you're describing a cart that is pulling its horse. The "certification", in whatever form it takes, must follow the professionalization of the field.
Regardless, none of the certificates you've mentioned --- OSCP, CREST, or SANS --- will define information security. None of them have any meaningful credibility to experts.
The burden of proof should be on whoever is suggesting that security has anything at all to do with those professions, but I'll throw out a couple of observations anyway.
Those professions have rules, and are backed by either legislation or science. All participants are bound by said rules. For a lawyer, certain things are legal, certain things are not.
Security is a game where the whole objective is to either break the rules (and often the law) or to defend against someone who is.
How are you going to tell me that person A is qualified for the job based on his exam results, and person B is not, when person B got a root shell on your server and stole your data?
It's like the 1989 draft when the Giants tried to make Dion Sanders write an exam to see if he was qualified to play in the NFL.
"“They sat me down and gave me a thick book,” Sanders recalled. “I mean, this thing was thicker than a phone book. I said, ‘What’s this?’ They said, ‘This is our test that we give all the players.’ I said, ‘Excuse me, what pick do you have in the draft?’ They said, I think, 10th [actually 18th]. I said, ‘I’ll be gone before then. I’ll see y’all later. I ain’t got time for this.’ That’s a true story."
It's difficult to believe that anyone who can claim to really know computers thinks they aren't based on a series of interacting rules. That's basically all they are.
Understanding how those rules interact, how to trigger certain interactions others didn't intend, and the best practices to not get bit by those interactions is what security is all about. It's much like law or medicine in that you are looking at unexpected consequences of multiple complex systems interfering with one another and looking for compromises and best practices to keep the most disastrous interactions the least likely to happen.
No. Computers are rules. They are multiple, complex, partially abstract and partially concrete rules systems with messy and often poorly defined interactions among those systems.
The individual NAND, AND, NOR, OR, and/or XOR logic? Rules - tabular even. The base IA? Rules. The microcode? Rules. The VMM? Rules. The TLB? Rules. The assembly? Rules. The OS kernel? Rules. The C library? Rules. The ABI calling convention? Rules. The application language? Rules - syntax and semantics. The libraries under the application - rules. The application itself is a list of rules for how data is processed. If it's Turing complete it's basically equivalent to the lamda calculus.
Every security issue is some misapplication of these rules due to someone not understanding the implications of the interactions of the rules. Every single one. Smashing the stack? It's applying a set a rules in a way the code author didn't anticipate. Overflowing a buffer? The code author didn't anticipate more data being stuffed in than the buffer was made to hold. Rowhammer? There are rules of semiconductor electronics interacting with the programming language, the IA, and the logic layout. SQL injection? Someone's applying the wrong rules to sanitize the input and someone else is giving input that takes advantage of the underlying rules of the programming language and the RDBMS that they were allowed to invoke because proper sanitization wasn't in place.
There is not a single person on this forum who needs your lecture about how, in one sense of the word, computers are based on rules. The obviousness of this fact should indicate that the disconnect is elsewhere.
The rules under discussion are policy, and the question is how to define policy for evaluating people's skill at running roughshod over policy. The fact that there are underlying "hard" rules is literally universal and therefore uninteresting in this context.
Lawyers and doctors deal in interacting complex systems of rules. So do information security people.
If you can make a certification that works for one expert in dealing with interacting complex systems of rules, you absolutely can make a certification for another expert in dealing with interacting complex systems of rules.
The details of what you test are different, but the fact that it's been done for law, medicine, medical specialties, dentistry, mechanical engineering, electrical engineering, civil engineering, and many other fact and rule based fields means it can most likely be done in general for people looking at how different systems of rules intersect.
No, you can't. That isn't how security works. Your offensive adversary follows no rules. They exist to break any rules you can think of. You can make all the rules you want and you can test people on their knowledge of them. Hackers do not care.
On the defense side, they simply do not work. Everybody gets hacked. The best companies with the biggest security budgets employing people at the cutting edge of security research still get hacked. Security experts get hacked. If the best in the industry still haven't solved this problem, you can't even begin to make the framework that you're proposing.
The discipline cannot be described as experts dealing with interacting complex systems of rules.
People still die. People still lose lawsuits. I-85 still partially collapsed. Yet people can be certified as knowing and following best practices in those fields.
Illness doesn't follow rules the doctor dictates. Rainfall and earthquakes aren't set by the engineering boards. How people adjust, prepare, and respond are where the rules humans can set work. Everything else is unchangeable rules.
In computers, as in some parts of law, we have ample opportunity to address underlying rules as well as the rules around how we adjust, prepare, and react.
The challenge that I've been trying to express is that individual excellence is hard to replicate at scale. Sure in sports, that works ok, the numbers are small and the rewards are great, the incentives are there for people to comb through thousands of people to find that one candidate who's truly excellent.
But that's not the reality for most companies, they're not trying to hire the absolute brightest and best, realistcally not everyone can. They're looking for some measurable indications that a candidate has a baseline level of knowledge in a given field.
Now I feel that a good certifiation can be part of that. So a valuable activity for the industry is to try and create better certifications to help companies who aren't in a position to judge for themselves whether someone is great at something or not, that there's a level of knowledge and understanding there.
If current certifications are poor, then it should be possible to articulate why they are poor, and describe what would make them better.
My references to other professions were designed to reference the fact that those professions have had to face similar issues as they've grown and in science, engineering, law, medicine, accountancy, etc etc they've pretty much all decided that some forms of professional certifications are the right way to go.
Not to say that they're perfect, but that they could be better than the alternatives.
When a lawyer is admitted to the bar, that is generally taken as proof that they have some idea of what they are doing. Bar exams are hard.
And the bar also provides a forum for dealing with shady lawyers. If a lawyer treats you badly, you can file a complaint with their bar association and they might get disbarred. This is an area where cyber training and certs is not as good yet, I think: as a forum for resolving disputes.
The problem is one of trust, methinks. Most certs require only that you "know" the material well enough to pass a test that uses your memory. If the tests were empirical, say like the Red Hat tests or the Cisco CCIE, then the trust that someone actually has skills might be more believable.
Having worked in IT security for many years, I can attest to the fact that IT security is more of a subjective set of processes rather than a specifc product or set of products.
There are skills involved as far as tools and knowledge of how to use tools, but these change depending on the use case.
Yep the problem is that some of the well known certs are bad, not that the concept of IT security certifications are bad.
I've been in IT/Infosec for 17 years now. I have certs that I literally maintain as a HR/sales checkbox (e.g. CISSP) and certs that make me think every time I need to refresh them (e.g. CREST CCT). It's possible to have good IT sec. certs.
There's both some truth and some falsehood to the 'certifications don't prove anything' argument:
Answer a multiple choice test for an MCSE or whatever? Doesn't prove much.
Receive a server that's been wrecked and won't boot, turn it into a load balancing HTTPS server, SMTP server, a bunch of required cron jobs and a boat load more requirements for RHCE? Proves you can do those things.
Disclaimer: used to work at Red Hat. Still love their stuff. Cisco has task-based tests too.
Yeah it's just a shame that multiple choice tests are the standard rather than practical examinations. All the multiple choice tests look for is if you have memorized their documentation and can figure out the test format rather than if you can actually apply it, which are different skill sets.
I'd agree that the problem isn't so much that certification has to be bad, but that bad certifications (e.g. those that examine though purely multi-choice) aren't appropriate/good for the industry...
I just recently had someone ask me to take an assessment test for a senior developer position. There's always some silly hoop to jump through, so I thought "why not".
Well I got booted out of the test because I hit Ctrl-C to copy something for the first warning, and hit Ctrl-L (muscle memory) for the final revocation of the test.
I just thought to myself ... did I just fail an assessment test because I hit Ctrl-L?
The test gets reset and allows me to start back up, only it took 20 minutes off the allotted time for some reason. I didn't even get to the last question on the test.
The results showed me scoring in the 96 percentile. So I basically threw 3 questions away on this test and still scored that high. And I skipped one question because it was asking about building/creating msi files on an C#/ASP.Net/MVC assessment. I have no idea why it was there, but I don't regularly build msi executables (although I regularly automate them).
And the worst part is that a lot of the questions were inane things like "given this inheritance hierachy, sally adds the new keyword in front of one of the child methods, and then this other code uses this inheritance hierarchy. what is the output?".
At no point do I feel like anything on that assessment came even close to assessing my ability as a senior technical person. These were things a college student could have answered just as accurately.
I know it's not quite the same thing as a certification, but I seriously dislike assessment tests. Unfortunately it seems like every company has their games you have to play in order to actually get TO the technical folks.
I think I've done this test or a similar one. The worst was going for an MVC role and it was giving me a bunch of questions on webforms lifecycles and "what does the X.Y" method do.
> There's no reason someone can't have both skills and certifications
Of course you're right that it's not impossible. But here's why it happens anyway and why the heuristic of them being roughly mutually exclusive is not insane:
1. There's a certification that's nearly meaningless because it's so easy to obtain without also having the relevant expertise that the certificate is supposed to represent.
2. People who are actually good at the thing will notice the certificate doesn't measure the skill correctly, and will also note that there are people in the world with this certificate who don't know the skill.
3. Those experts will not use the certificate when they hire people, and will not get the certificate since it doesn't work and no one who is an expert is using it to hire anyway.
4. Meanwhile, there are basically only two groups that care about the certification:
> a. People who are clueless: a clueless hiring manager who doesn't understand the domain they are hiring for so they are looking for cheap proxies for skill and experience. Clueless wannabe professionals who don't understand the domain or the industry enough to have been in the expert group above but who are still looking for jobs in the field. Clueless clients who are impressed by the certification because they don't know any better.
> b. The people who are taking advantage of the clueless clients: Professionals and hiring managers who know very well that the certification is worthless, but who use it for sales and marketing anyway because it mollifies clueless clients.
If that trend holds, then you have a signal (the certification) the repels experts, while attracting the clueless and those who would exploit them.
So when you find someone in the world with that certification, you should expect on average for them to be clueless or preying on the clueless. That's why treating it as mutually exclusive isn't insane heuristically, even though it's not impossible that someone who is good also has a certification.
Additionally technical certificates supposedly measure competency in a particular technology. That technology is obsolete roughly at the time you're sitting to write the exam. So you've spent time learning material (assuming you didn't already know it) to cover a technology that will be in the process of being replaced as you sit down to write the exam. You may as well have spent the time using the skills productively rather than using it to obtain a piece of paper that says your skills are obsolete...
Plus there's the financial outlay of it; costs to obtain the course material and sit the exam.
These points (including yours) basically amount to my bias against obtaining certificates.
I'm not against the material itself, if it's useful. But I'm not in the habit of paying for or spending time on education for anything other than the value of my interest in the material. A piece of obsolete paper does nothing for me at this point in my career.
This is a great comment to illustrate grandparent's point that irrational hate of certifications permeates the industry. Many certifications under discussion here are meant to assist at the entry-level. That's what the certificate is "supposed to represent". That's what skill the certificates "measure correctly." That's the 3rd group of people who care: people operating at or near the entry level.
It is no surprise that experts hiring experts in any field couldn't care less about certifications (aside from ones required by regulation). At the expert level, you hire real, verifiable experience.
You've ignored the point of the post you're replying to. You're looking at the credential as a employee signalling tool, not a tool for other parties to satisfy a business need.
Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.
If an applicant applies without the certificate to a post which requires it, he will recognize his cover letter/interview should make a point of demonstrating competence in the area of the certificate. This is an instance of employer signalling.
If an applicant fakes having acquired the certificate and there is an easy way to determine whether or not he is faking, then HR now has an easy sieve for removing employees willing to misrepresent their skills to obtain the job. This is a step in the HR QA process.
Some employers work outside the 'work for hire' states. If they contain a certificate requirement in a job posting with enumerated skills, they can point to ineptitude in the certificate skillset as reasons for dismissal depending on the jurisdiction they're in. This is a tool in legal's toolbox.
Perhaps the certificate system is the result of the collaboration of a number of employers attempting to pool credential and training requirements for an industry, then create requirements for credentialing into legislation to control the industry pipeline of workers. This is a method of signalling to legislators or restricting competitors by indirect means.
And so on.
Additionally, your fact pattern has a small problem: expert acting as hiring managers who have knowledge of the problems with the certificate are free to adjust their screening procedure to obtain more fully vetted candidates. The only time avoiding the certificate entirely is when the signal it provides is negative.
This doesn't mean that certs are useful, merely that they might be useful to stakeholders you aren't considering.
I did ignore that mostly, you're right. My basic claim is that on average this is actually true:
> The only time avoiding the certificate entirely is when the signal it provides is negative.
And further, I think all your examples are perfectly valid, real-world examples that I don't dispute exist and that lots of people find important. I also think perfectly good and reasonable people operate in the reality of their industries and play ball with these things when necessary. AND I think all the use cases you mentioned are bad for the system overall. As in, they are real, and in a practical sense we can't just ignore them, but ideally we wouldn't have them.
HR using negative signals for filtering is bad. Job requirements tailored to the actual job are good. Broad and mostly arbitrary requirements from a third party are bad. Applicants to a well specified job also know to address weaknesses in their cover letter. Specific and well-specified requirements are also useful in legal disputes. Using arbitrary requirements to provide legal for firing people is bad. Employers colluding to control the training pipeline using arbitrary requirements encoded in law is bad.
I agree with you that they useful in the real world, but I'm arguing that that usefulness is evidence that the system is worse than it could be.
> Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.
The real root of the issue here is that HR can't do their job. An unintended side effect of all this signalling is that future employees will be glad to get filtered out because they don't want to work somewhere that hired on certification instead of competence.
> The only time avoiding the certificate entirely is when the signal it provides is negative.
It is possible for the same action send a positive signal when done by some people and a negative signal when done by others. Specifically there can be 'contersignaling' [0], which is basically signaling that you don't need to signal.
So the article alluded to reading books and hacking on your own. But for those who need some sort of curriculum, progress bar, or structure, what would HN recommend to get to some sort of level of competency in the infosec field (like intermediate level/beginner-advanced).
Bear in mind that IT security goes far beyond something with a processor in it.
There are physical access controls, personnel assessments, probability and impact assessments, budgeting, people-monitoring, process analysis and modelling...
Computers are a tiny part of it. This being HN I have understanding for the bias though.
So the article alluded to reading books and hacking on your own. But for those who need some sort of curriculum, progress bar, or structure, what would HN recommend to get to some sort of level of competency in the infosec field (like intermediate level/beginner-advanced).
I joined the workforce at the same time that people were realizing that certs were a joke. That both hurt and helped me, as people hired me based on talent, and not a piece of paper.
But I wasn't in management, and I've since learned how very little technical skill you actually need to be an effective manager, and now I realize that certs are a great way for a manager to understand a complete baseline of the concepts needed for a particular field.
A manager does not need to be a hacker, but they need to understand a baseline of security concepts. That's what certs are really useful for.
I wanted to get my consulting company into PCI auditing and you need certifications to do that. One problem with the certifications is they aren't actually skill based.
I wouldn't be able to get one despite having experience because:
For several of them it requires years of work experience with a specific job titles (your job needs to be security, it can't just be part of your job) and the continuing education credits are expensive and largely not helpful.
I would love if there was a recognized certification that was actually interested in proving your skills not for making money.
I know other disciplines are the same way.
They don't really serve to keep unskilled people out, they serve as kind of an elite paywall where you need time and money to break through. And breaking through is largely an exercise in brute force not in skill.
"I wanted to get my consulting company into PCI auditing and you need certifications to do that. One problem with the certifications is they aren't actually skill based."
From most people I talk to, the exception is the OSCP since it requires you to actually pop real, live boxes. Anyone holding that cert has actually exploited a buffer overflow, escalated privileges, etc. CEH, CISSP, etc are just too theoretical with no hands-on requirements.
Don't know if you have taken it in the last year or so since they updated it, but it's pretty tough. You may be able to use a public exploit to elevate your shell once on a box, but getting code execution was the difficult part. One of the challenges involved fuzzing, writing custom buffer overflow exploits, and dealing with weird stack pivots. That only got me about 20% of the way to passing the test. All in 24hrs. My girlfriend was taking the GPEN at the same time. While I was banging my head against a debugger she was making flash cards. I think that highlighted the difference between the certs.
Unfortunately I can't get into too much detail because I had to sign an NDA (to prevent cheating). But the process was similar to when I have found them in the wild: identify the app, install it locally, fuzz various parameters (it was a real application, albeit an old one), find the crash, figure out stack space, figure out bad characters, find the right JMP ESP or equivalent instructions in a loaded library, write shell-code, encode shell-code, slap it all together, hope your hex math doesn't suck, run the exploit. No DEP, ASLR bypass, SEH manipulation, use after free, or heap related work - I learned that on my own.
Their web app challenges were fun too. LFI to code execution, SQL injection, things like that. They have a bunch of network related recon, standard red-teaming stuff.
The OSCE involves ASLR bypass, AV bypass, and using egg hunters.
The big thing about the OSCP, OSCE, OSEE certs is that you actually have to _do_ all of the stuff they teach you. Not a multiple choice or written question in sight. For the test they drop you in a network with vulnerable machines and you have 24, 48, and 72 hours (depending on the cert) to get code execution on each through various techniques. It was challenging, interesting, and satisfying.
Edit - it's worth mentioning that I still find vanilla buffer overflows on projects. These days most thick-client applications that I see are old as hell and are still vulnerable to exploitation techniques from decades ago. So while the skills that the cert makes you prove are cursory and introductory, they are still useful. In any case it's a good starting place for those that want to learn stuff on their own but do better when they are given the push to prove it.
It doesn't.
The OSCE targets are Vista and 2K3 Server.
That being said, the nice thing about OSCP imo is that it gives you some structure and a well set up environment to play in. I think OSCP is a great entry-level certificate and serves as a good filter to interview junior candidates.
Does this help at a more elite level, nope, but that's also not the purpose of it.
Having a 1-2 certifications on a specific domain means that we speak the "same language" regarding our work.
Red flag: someone that has an email signature with 50 letters next to his/her name,there is NO WAY someone is spent enough time on each: coding, security, audit, accountancy, at the age of 30 AND be proficient in all these domains.
>Having a 1-2 certifications on a specific domain means that we speak the "same language" regarding our work.
Or it means they can drop the same jargon and have maybe a passable understanding about the ideas that jargon is meant to convey.
Often people with extremely narrowly focused bases on knowledge (as indicated by having BAs, MAs, and a giant rap-sheet full of certs all in the same topic) have wound up being thoroughly incapable of actually applying any of their knowledge to the benefit of the team they're on because they just don't know how to get what's in their heads into the heads of people who weren't steeped in the same language as they were.
I run into this issue on my resume. Do i throw random skills i spent 4 months learning for some project and never used again? I feel like overloading these things devalues the skills i actually AM exceptionally competent at, as opposed to just capable.
My advice? Remember that your resume goes through at least 2 filters, HR and IT.
What happened was:
IT boss: "we need to hire a coder"
HR: "What skills?"
IT Boss: "Oh, Foo language. But if they're a good coder they can pick it up, so just a good coder"
HR: "...you're kidding, right? There are millions of resumes out there, most from people with no skills just trying to land a great job. Give me enough to filter"
IT Boss: (provides list of three things)
HR: "This still isn't enough. Practically EVERYONE will have these. Give me years of experience, skillsets, processes, etc.
IT Boss: "Fine, here" (gives long list of things that MAY be useful)
HR: (starts filtering resumes based on these words, removing lots of good people and including lots of bad people)
IT Boss: (looks at resumes) "These people are clearly lying and all over the place, I'm going to focus on one or two things to decide who to interview"
So in writing your resume, you want to make sure you have the buzzwords for the job to get past HR. These buzzwords are pretty much guaranteed to be on the job listing, even if they end up not being very essential to the job. Did they mention Scrum? Better have it on your resume, because you may be filtered out if it isn't, even if it's something you'd not consider worth listing. Also, use the same words. I once was asked if I had "shell experience", even though BASH was on my resume. I assume "Agile" and the various implementations are the same. If they mention XP, you better mention XP.
BUT when your resume then makes it to IT, who (1) know what these words mean and (2) aren't looking for the same things at all, you need to have what they want. I tend to use a sidebar on my resume to capture the HR buzzwords, and emphasize my work experience in the main body, so an IT person skimming it will see what they want to see.
One technique I've taken to handle HR buzzwords on things that I don't think are actually a big deal: If the job listing says "Must know React, Angular, Backbone, or other JS frameworks" and I wasn't really strong in any of them, I'd do enough research and testing coding to do a Hello World in them, then add "Exposure to Foo, Bar" on my resume. It tends to get through HR (word is present!), and I'm not lying to the IT people - they understand that I'm not claiming expertise, but I'm also saying I'm willing to give it a go.
As a corollary to all of this, you need to tweak your resume for every job posting, to match their buzzwords and remove ones they didn't list that aren't really core to your skills.
A lot of the bigger certs have all kinds of high-dollar prerequisites and accounting that only larger firms are going to bankroll. Perhaps the certs aren't so much an expression of how "good" you are as much as they're a token of how much faith "the man" has that you will stick around long enough to recover the investment.
I'm going to partially disagree with the article. the problem with the approach of "just learn to be a good security person" is that it doesn't scale. Sure back when I, and a lot of other people who are a bit older, learned security that was the only option, there weren't structured courses and certifications.
However when we're working at scale, certifications can be useful as providing a demonstration that the holder has some level of exposure/knowledge of the arena in question.
what complicatates this quite a bit is that some of the more popular certifications are, rightly, not considered that good as the process to get them lends itself to rote learning. So things like the CEH and CISSP where the exam is multi-choice, not so great.
On the other hand things like the CREST CCT definitely require a decent level of knowledge to pass and you need to be able to apply that knowledge in a practical situation and in time limited conditions.
I find the anti-certification bias in IT and security a bit odd really. If you look at other professions (e.g. law, accountancy, architecture etc etc) it's recognised that these things are required to get a minimum level in place in situations where you have a large body of people, I don't really see why IT Security should be any different.
To me, the problem of "these certifications are bad" should have an answer of "lets make better certifications" not "lets not use certification"
The only certifications I've picked up so far along my industry journey (I have no college degree and don't plan to get one, so these are necessary) are the Redhat RCSA and RHCE. Both of these certs can't be solved with rote memorization, and required me to log into a virtual machine and solve problems in a live environment, running through a plethora of common systems administration tasks that were then graded by how well I accomplished the requirements. Often the requirements were vague enough that I had to do some digging.
There was no real memorization needed, and I had the full man pages of the operating system at my fingertips, but the time limit ensured that I needed to have at least a certain degree of proficiency with each tool to finish all of my tasks before the end of the exam.
I assumed this was the norm with certifications, but the comments here strongly suggest that it is not. Are Security Certifications really multiple choice questions without any practical applications? That seems like it could stand to be improved greatly.
the original article, I felt, made that point. It provided several examples of people learning on their own and provided those as an argument for why certifications were unecessary.
While I agree that many of the certifications are worthless they do help newcomers to get into the security field.
I was lucky enough to start early when the only thing you had to do was to show your skills and willingness to learn. Those days are long gone but you can still prove yourself by delivering awesome security research or commentary - and people will hire you based on that.
That being said, any IT field which requires some sort "technical" certification to get hired is probably not the field you would like to get into. Why? Too much competition and race for the bottom line.
Luckily this is not exactly the case with Information Security - yet. In our line of work there are people who do a lot of the uplifting where certifications do matter. I would not say these are very technical jobs and they are totally dispensable. Other professionals acquire specific skill sets which are rare or difficult to obtain and as a result they tend to have the upper hand. There is of course a lot in between.
I do not mean that everyone who has certifications next to their name sucks. Not at all. But unfortunately, unless you are applying for a commoditized IT security field or security manager type of role, it will raise some questions.
Keep in mind that HR is also partially to blame. Many people do not know how internal HR teams typically work which is essential to understand why certain things happen the way they do when hired or when progressing through the ranks of a company.
HR are typically not technical and they are not experts in security either so they do not know what exactly they should be looking for. Of course they are not completely clueless but a seasoned hacker can smell bullshit a far while a well informed HR cannot. HR's filter is your CV and the job spec and these two are simply not enough to evaluate a successful candidate. Some job specs are written by HR themselves which is crazy because they are not in the position to formulate the role so they have to stick to what is known - i.e. certifications.
That being said there is a shortage of IT security professionals. As a result of that almost anyone can get hired if they show the right attributes. It does not take a long time.
The only thing that bothers me with the InfoSec industry these days are not the certs but that companies tend to have really bad hires (maybe due to bad certifications) who due to lack of deep understandings of the subject work on things that practically do not matter. As a result of that these companies have the illusion that they are doing something while the fact is that they do not in a practical sense. This is why in many places no one knows what the security department does - I am not kidding.
Every comment seems to be about whether the certs demonstrate anything but this article says, "a job description requiring a CISSP was a warning flag to industry elite not to apply."
In other words: if the company asks for certs it's the equivalent of wanting "6 years of react.js experience". I completely agree.
The thing with infosec is that no matter if you're a consultant pen tester or an in-house member of a blue team, a high proficiency in technical writing is required. And few certs demonstrate that the person is a good technical writer. It's not enough to know the answers to multiple choice questions. It's not even enough to know how to exploit things. If you don't understand something well and can discuss it in technical detail to a number of different audiences, I don't believe you'll get very far in the industry.
There are a couple of exceptions, of course. OSCP is a good certificate to have. To pass the exam, you are required to not only demonstrate proficiency in several areas (i.e SQL injection, buffer overflows), but you must also write and submit a technical report to a review team. The technical report must address vulnerability overview, impact, risk rating, reproduction steps, and more. Of course the exam isn't perfect, but it's probably the biggest test of real technical understanding and ability I've ever seen.
nice to see OSCP still regarded well. I got it about a decade about when our CTO had an initiative that everyone doing customer interaction (R&D team + professional services support for me at the time) should have a cert.
I vehemently disagreed, pointing out that those of us with meaningful degrees from well regarded programs shouldn't have to get one, but in the end I complied to keep the peace.
Have to say I enjoyed it at the time and even learned a thing or two. I wasn't exactly new to offensive work, but was nice to get things sharpened up and familiar with a few different approaches. I remember being a bit worked up about the final challenge at the end of it that I took the day off work, but iirc I was able to finish in just a couple hours. It wasn't the most cutting edge content at the time, but it was well organized and ensured that the person being certified could demonstrate some level of practical application and that is far better than most certifications of any sort.
This. So much this. Writing and general social skills are the number 1 thing lacking with people I interact with in the industry. The I know more than you attitude is great amongst peers, but with clients, you don't have to prove you're smarter, instead your job is to make them smarter.
Break it down to a 4th grade level, if you can't, you likely don't understand it yourself.
I would agree that in general certs have too much weight, but the reality of it is, as it stands, they're the closest bridge / standardization that the market has built for non-Security ilk.
Also, Security is not IT. That's another thing that needs to change.
As for OSCP, I think it's great and out of all the certs i've taken for various reasons, it's the only one I felt challenged with, in a good way.
Knowing basic English is a prerequisite to any tech job. There's nothing else you need to know to explain a bug. And there's TOEFL/IELTS if you're looking for English language certificate.
In theory, maybe, but there are many people who are fluent in basic English but who can't usefully communicate technical matters like bugs (except, perhaps, to someone who already looked at that specific problem with them).
Edit: And, in contrast, I've met people who I had a hard time communicating with in spoken English but, had no problems with once we did our communications in writing or with a written point of reference.
Communication in tech, and especially infosec is a dramatically underrated skill. Infosec consultant for 10 years. I write. A lot. Being able to jump from explaining how we reversed something to devs to an executive who just wants a certain view of how that impacts a release is hard and has taken me a long time. You essentially have to understand the various consumers of your writing at a pretty deep level for it to get read and have impact.
A well written report that speaks at the right level to its audiences will generate more proactive security activity than a terse, passive voice bomb of dense technical information.
It's unlikely that typical OSCP-holder could write a modern buffer overflow exploit, or even judge exploitability of a memory corruption flaw given the source code and a traceback.
Equally importantly: memory corruption exploit development and SQL injection are different skills, and most people who do SQL injection don't need proficiency in "buffer overflows". Why is superficial coverage of "buffer overflows" part of the rubric for that certificate? I don't know, and I don't know that anyone else does either.
Is there a single coherent security certificate anywhere in the industry? I'm interested in examples.
Totally agree with this. I think security / infosec is just seen as a small niche, so people lump the cryptographers, auditors, SOC analyst, malware analyst, appsec, incident response type people all in one group. It's hard to find someone who can cover all that ground proficiently. Of course it's also been my findings that people who end up in infosec tends to be generalists, but I wonder if that is shifting now that you see more cyber security school initiatives.
Total tangent but, I am absolutely grossed out by "cyber" winning out in the name game. Who let the DoD drive that? Damnit!
This is why when I was in an infosec bootcamp I begged you to talk to my class and give a dose of reality.
The leet kids, a minority few and the rest naïve, I would bribe while they whittled away at online CTFs and MicroCorruption and irritate them with mediocre questions until they tuned me out. I did not care for tools; approach and mindset are order of magnitudes harder to explain.
I thought you could be a wakeup call had you told them all the certs pro se is a waste in a program that pushed that nonsense. I talk trash of my certs and skills the whole time and they did not get why.
I know you're busy, but I've read your blog and you're preaching to the choir. Starfighter folded, but I would pay for you to test me as a customer and find a mentor to answer my stupid questions that I would pay handsomely for the privilege. I feel I'm not the only one, if you get tired of NCC, that would be amazing!
I totally agree that most web pentesters don't generally need to know how buffer overflow and binary exploitation techniques work but I think an understanding of how low level systems function and how they can be exploited is useful across all security sub-fields.
I don't think Offensive Security is trying to pump out exploitation experts from their entry level cert program. Maybe the higher levels OSCE and OSEE. The intro cert emphasizes breadth over depth. It felt a lot like a cert built around the Exploitation Hackers Handbook.
I think you're thinking of the certs in the wrong light. They are meant to validate baseline knowledge and proficiency, not mastery. If you want to validate mastery you need to look at the persons personal record and work product.
What you think about GIAC Platinum, GSE, or whatever it was called? And with a mix of software, networking, and incident handling before the interviews and pentests?
It's the only one I ever thought might be valuable since it combines specific knowledge with some pentest against real systems. Curious what you think as you're more qualified to assess such a thing given all your security evaluations and hiring those people.
> The thing with infosec is that no matter if you're a consultant pen tester or an in-house member of a blue team, a high proficiency in technical writing is required.
As much as I admire people who show the courtesy to the trash bin that will eat their report, before any human reading it, of not feeding it "bad" reports: The OPs point of "sense of false security" unfortunately already sets in once someone is hired to "take care of security". This anecdote from about 2000 illustrates that:
A friend was hired to do black box penetration testing on the DMZ and internal network of one of the largest travel agencies in Europe. He, of course, found a lot of problems. He wrote a nice report. Like really nice. Point for point "this is your problem, this is what you have to do right now and this is a user-friendly policy you could implement to prevent such issues in the future." It was very pleasant to read, he knows how to handle language. And somewhere on page hundred-something "the first one to claim it, will get a bottle of champagne!".
The bottle was claimed a few months later by someone 3 levels above his position. By some high-level manager who did not know anything about IT. But he was the only one to read it (and he only claimed the bottle to figure out whether anyone below him had actually read the report). Two years later company politics allowed those 2 illiterates between my friend and the one manager who could read to be removed from the company. My friend was hired as their chief of security. His first task: work through his own report to fix the 90% that had not been fixed since he wrote it 2.5 years before (the report listed passwords to core routers in plain text as examples for "very stupid passwords" - they all still worked).
I used to pentest for a living. Still do some red team exercises every now and then, but far less now that I'm mainly blueteam focused.
I personally organized my report into three sections, which seemed to work well. Clients seemed to enjoy the formatting:
1. Executive - Summarize everything in one page at a high level. You could skim it fast if you chose to. Highlight potential negative business impact of each finding.
2. Management - A little more detailed. 2-3 pages max. Most severe findings at the top and recommended action for remediation.
3. Narrative - This is the bulk 80-90% of the report detailing your step by step process including screenshots so that if someone wanted to duplicate your findings they could.
The last (and only) pen test report I saw had some "escalation of such and such via this vague vector, please do x and y to close off this avenue" - strongly implying that they felt the actual trick being used was their IP to be protected?
This was for using citrix boxes to provide saas to a client
About a decade ago (give or take a year) our small web company had a PCI test against our network and a 500-page PDF ended up in my email. The "report" was obviously the vomit of a program detailing every issue for every site [1]. You would think that after the 50th time our DNS resolvers were "open to the public" (as it were) that the stupid program vomiting the report would realize it's reported the same DNS resolvers each time.
[1] It didn't help my perception of the report when it screamed that "ICMP echo was enabled and nefarious scalawags might be able to do unspeakable acts against our computers, best cut the network cable" type of advice [2] (yes, I know port 443 is open! WE'RE A WEB HOSTING COMPANY SERVING UP COMMERCIAL WEBSITES! ARE YOU PCI AUDITORS STUPID?)
[2] Okay, the ICMP echo thing was reported, but did not need to be disabled to pass the audit. If so, why even bring it up?
At one company I'd do a similar thing - about two-thirds of the way down a list of dot-points, I'd mention something about free chocolate or a beer. No-one ever mentioned it, not even to reference it in the slightest or tell me I was wasting my time.
223 comments
[ 2.8 ms ] story [ 234 ms ] threadThe same could probably be said for a degree, particularly with the current snowflake "everyone must pass" mentality at so many schools.
Certifications and degrees are nice, but neither guarantees a person is competent in any field.
The problem is, you can't tell the difference between the person that did the time, studied, did labs, etc... And the one that downloaded a test answer brain dump and memorized the answers.
And even in the case of the former, you can't cram 20 years of knowledge and experience into a certification.
I have just an MCITP that I had to get as a job requirement. My employer paid for a 2-week bootcamp. I did very little studying and already knew 90% of what was covered.
I have about 10 people around me that have various combinations of certifications and degrees that all use me as a lifeline when they get stumped.
"If you took the tax id and social from 'airGappedMachine'"
"It has no connection to any other machine"
"Just query the database"
"Store it on a thumbdrive and walk it over? It changes quite often I don't think that's a good workflow"
"No, just query it"
"How?"
"SQL!"
Or the classic:
"That password only has N characters, I can crack that in Y seconds"
"You are the Administrator of that box, you can simply reset the password and do whatever you like."
"Yeah but the hackers.... Y seconds"
"It's not Y seconds for them, if they already are an Administrator or have read database access it's game over anyway they have to use the API and that locks you out after 3 attempts. Also your are assuming a much easier hashing method than is actually in use."
"Yeah but my calculator says Y seconds so it's Y seconds"
Like the "Mongodb is web scale" one.
Universities have been a centralized source of accreditation for a long time. All it takes is for someone to figure out how to restrict graduation and filter good candidates using testing or some other means to gain accreditation and acceptance of its graduates by industry.
http://www.uoc.edu since 1994
This is based on my last batch of interns that had masters degrees but couldn't handle hello world. Their spoken english skills made it clear that they were completely incapable of understanding the lecturers.
Universities are a business, they are paid a lot to provide a piece of paper, so they provide it.
Most organizations aim for compliance (it's cheap and easy). They base security on contracts, certs and insurance policies.
Street-smart security practitioners are appalled by this. And, management doesn't understand why the 'security people' aren't on-board with 'compliance'.
It's a lot like the old west with Cowboys and Indians. Two totally different world views.
Yes, Anthem/BCBS, Target, HD, Sony, etc, etc have all had losses.. but they really havent been long-term impacted it seems.
I dont know what the answer is, this sucks hard as both a consumer and an infosec person. I tend to view security as a "hidden performance" factor. As long as the security flaws don't inconvenience the paying customers too much, they simply don't care if they exist or not.
It was literally "person's-company-name Certified Forensic Examiner".
He had created about 6 certifications, all of which he held.
It's kinda funny, but also kinda scary that the court accepted this as proof of his qualifications. The prosecution never raised an objection to it either.
I don't see how that's much different than asking someone to solemnly swear they are telling the truth, when most humans are as capable as lying about whether or not they are truthful as they are of lying about anything else.
If the court has no one capable of gauging the expertise of a witness, it has to trust in someone to do that for them, and if neither party in the case objects to the witness certifying himself as an expert, it has no reason to gainsay that assertion. It's really the prosecutor's failure alone, for letting that detail slip past.
The differente against lying is that you know when you lie. You dont know when you are overly confident but incompetent.
If the other side of the case had an expert that didn't bring up the lack of credentials, or the litigators on the file didn't bother to do the most cursory of credibility impeachments, that's on them.
Everyone can tell the truth, but not everyone knows technical details about something. He is using his certifications to show he knows technical details; he might even believe he knows those things, but he very well could be wrong.
Certifications sometimes set a terrible baseline, but at least it's an independent baseline.
I now do not include this cert on my CV for (perhaps irrational) fear that someone in some HR department may think "Oh, no! This guys is a hacker!" Sadly, the word has a negative connotation, because the word "cracker" or "bad actor" never bubbled up past the IT security world.
I also do not include my military service dates, as they reliably peg my age. Most men join at 17 or 18, so they would immediately know my age.
I'm debating whether to include any certifications at all, just include my degree.
I leave my certifications off my resume now, unless the job posting asks for certifications which is frustrating trying to decide if adding this cert to my resume will help or hurt me in the interview.
From a certain point of view you are right -- an IT manager should cooperate with other facets of the business in order to help the organization succeed, not obstruct because he thinks their requirements are dumb. But from a larger point of view, either organizational or super-organizational, dumb requirements shouldn't exist. To the extent that they do CEOs, industry groups, and/or governments should be modifying them to keep them relevant.
You're pointing out things that are generally considered to be failures in our sphere anyway. I don't think that's unrelated.
HR brings us terrible candidates? Certs (currently) don't help that, and when HR over emphasizes them, we can blame the certs. (In theory, certs COULD help, but the people that get the cert instead of the experience are the problem ones, and those are the people HR brings us)
Contracts are an attempt to preset agreements between two parties, and in the tech world usually one of those parties are at a disadvantage in the tech space (otherwise we'd not be making the contract). Certs are used as a means of asserting competence in the employees, but the other party has every incentive to focus on the cert instead of the competence. So when it goes bad (as it often does), we can blame the certs as not achieving the goal.
Audits are likewise, trying to filter for competence (in action or in people, depending). But again, if the incentive is for the cert over the competence, it is the competence that suffers. We blame the cert (or more correctly, the emphasis on the cert).
IT definitely has a bias against certs. And it's not really against the certs themselves...it's that emphasis of the certs is harmful, not helpful. So we argue against that emphasis, which sounds a lot like hating the certs.
> The problem most IT people have is thinking that certs address technical issues, when in fact they address business issues.
But those business issues are "how do we ensure the technical issues are addressed?" And it turns out Certs in practice do a terrible job at that.
I'm definitely not trying to defend the CISSP/CEH style certs here but I don't see how you reliably expand the industry at scale without some form of certification process.
A company hiring it's first security person or a company trying to hire a lot of security people, need some form of base benchmark to work from, just like with most other professions (e.g. Law, accountancy, architecture etc)
For example Certified accountants, Lawyers etc.
Without some common baseline, how do people looking to hire security types who don't have the experience to assess their skills and knowledge avoid getting bad people?
Also the article's argument that "it's experience that counts" really doesn't help get new people into the industry, where do they get the experience in the first place, it's a catch-22
So would I be right in thinking you don't think that any of the Offsec certs (OSCP/OCSE), CREST certs (CCT etc) or SANS certs are usful?
Also, and I'd be genuinely interested to hear your thoughts here, why do you think that IT/Info Sec will take a different path than other professions (medicine, law, accountancy, engineering etc) which fairly universally have evolved into a certified professional model?
Of course one problem is "how does a certification become recognized", I mean in IT security it's going to have to start somewhere...
In the UK the IISP are perhaps closest to the "traditional profession" certifications, but they're struggling a bit to get traction.
Unfortunately the industry is growing far faster than perhaps happened for previous emergent professions, so the time needed to slowly grow professional bodies isn't available.
If it's not commercial organisations that start providing those services, the only other options I can see are some form of union, or some government mandated body. Those are options, but both have their challenges.
Both those options have their downsides.
Regardless, none of the certificates you've mentioned --- OSCP, CREST, or SANS --- will define information security. None of them have any meaningful credibility to experts.
Those professions have rules, and are backed by either legislation or science. All participants are bound by said rules. For a lawyer, certain things are legal, certain things are not.
Security is a game where the whole objective is to either break the rules (and often the law) or to defend against someone who is.
How are you going to tell me that person A is qualified for the job based on his exam results, and person B is not, when person B got a root shell on your server and stole your data?
It's like the 1989 draft when the Giants tried to make Dion Sanders write an exam to see if he was qualified to play in the NFL.
"“They sat me down and gave me a thick book,” Sanders recalled. “I mean, this thing was thicker than a phone book. I said, ‘What’s this?’ They said, ‘This is our test that we give all the players.’ I said, ‘Excuse me, what pick do you have in the draft?’ They said, I think, 10th [actually 18th]. I said, ‘I’ll be gone before then. I’ll see y’all later. I ain’t got time for this.’ That’s a true story."
Understanding how those rules interact, how to trigger certain interactions others didn't intend, and the best practices to not get bit by those interactions is what security is all about. It's much like law or medicine in that you are looking at unexpected consequences of multiple complex systems interfering with one another and looking for compromises and best practices to keep the most disastrous interactions the least likely to happen.
The individual NAND, AND, NOR, OR, and/or XOR logic? Rules - tabular even. The base IA? Rules. The microcode? Rules. The VMM? Rules. The TLB? Rules. The assembly? Rules. The OS kernel? Rules. The C library? Rules. The ABI calling convention? Rules. The application language? Rules - syntax and semantics. The libraries under the application - rules. The application itself is a list of rules for how data is processed. If it's Turing complete it's basically equivalent to the lamda calculus.
Every security issue is some misapplication of these rules due to someone not understanding the implications of the interactions of the rules. Every single one. Smashing the stack? It's applying a set a rules in a way the code author didn't anticipate. Overflowing a buffer? The code author didn't anticipate more data being stuffed in than the buffer was made to hold. Rowhammer? There are rules of semiconductor electronics interacting with the programming language, the IA, and the logic layout. SQL injection? Someone's applying the wrong rules to sanitize the input and someone else is giving input that takes advantage of the underlying rules of the programming language and the RDBMS that they were allowed to invoke because proper sanitization wasn't in place.
The rules under discussion are policy, and the question is how to define policy for evaluating people's skill at running roughshod over policy. The fact that there are underlying "hard" rules is literally universal and therefore uninteresting in this context.
Lawyers and doctors deal in interacting complex systems of rules. So do information security people.
If you can make a certification that works for one expert in dealing with interacting complex systems of rules, you absolutely can make a certification for another expert in dealing with interacting complex systems of rules.
The details of what you test are different, but the fact that it's been done for law, medicine, medical specialties, dentistry, mechanical engineering, electrical engineering, civil engineering, and many other fact and rule based fields means it can most likely be done in general for people looking at how different systems of rules intersect.
On the defense side, they simply do not work. Everybody gets hacked. The best companies with the biggest security budgets employing people at the cutting edge of security research still get hacked. Security experts get hacked. If the best in the industry still haven't solved this problem, you can't even begin to make the framework that you're proposing.
The discipline cannot be described as experts dealing with interacting complex systems of rules.
In computers, as in some parts of law, we have ample opportunity to address underlying rules as well as the rules around how we adjust, prepare, and react.
But that's not the reality for most companies, they're not trying to hire the absolute brightest and best, realistcally not everyone can. They're looking for some measurable indications that a candidate has a baseline level of knowledge in a given field.
Now I feel that a good certifiation can be part of that. So a valuable activity for the industry is to try and create better certifications to help companies who aren't in a position to judge for themselves whether someone is great at something or not, that there's a level of knowledge and understanding there.
If current certifications are poor, then it should be possible to articulate why they are poor, and describe what would make them better.
My references to other professions were designed to reference the fact that those professions have had to face similar issues as they've grown and in science, engineering, law, medicine, accountancy, etc etc they've pretty much all decided that some forms of professional certifications are the right way to go.
Not to say that they're perfect, but that they could be better than the alternatives.
When a lawyer is admitted to the bar, that is generally taken as proof that they have some idea of what they are doing. Bar exams are hard.
And the bar also provides a forum for dealing with shady lawyers. If a lawyer treats you badly, you can file a complaint with their bar association and they might get disbarred. This is an area where cyber training and certs is not as good yet, I think: as a forum for resolving disputes.
Having worked in IT security for many years, I can attest to the fact that IT security is more of a subjective set of processes rather than a specifc product or set of products.
There are skills involved as far as tools and knowledge of how to use tools, but these change depending on the use case.
I've been in IT/Infosec for 17 years now. I have certs that I literally maintain as a HR/sales checkbox (e.g. CISSP) and certs that make me think every time I need to refresh them (e.g. CREST CCT). It's possible to have good IT sec. certs.
Answer a multiple choice test for an MCSE or whatever? Doesn't prove much.
Receive a server that's been wrecked and won't boot, turn it into a load balancing HTTPS server, SMTP server, a bunch of required cron jobs and a boat load more requirements for RHCE? Proves you can do those things.
Disclaimer: used to work at Red Hat. Still love their stuff. Cisco has task-based tests too.
For me the answer is better certifications.
Well I got booted out of the test because I hit Ctrl-C to copy something for the first warning, and hit Ctrl-L (muscle memory) for the final revocation of the test.
I just thought to myself ... did I just fail an assessment test because I hit Ctrl-L?
The test gets reset and allows me to start back up, only it took 20 minutes off the allotted time for some reason. I didn't even get to the last question on the test.
The results showed me scoring in the 96 percentile. So I basically threw 3 questions away on this test and still scored that high. And I skipped one question because it was asking about building/creating msi files on an C#/ASP.Net/MVC assessment. I have no idea why it was there, but I don't regularly build msi executables (although I regularly automate them).
And the worst part is that a lot of the questions were inane things like "given this inheritance hierachy, sally adds the new keyword in front of one of the child methods, and then this other code uses this inheritance hierarchy. what is the output?".
At no point do I feel like anything on that assessment came even close to assessing my ability as a senior technical person. These were things a college student could have answered just as accurately.
I know it's not quite the same thing as a certification, but I seriously dislike assessment tests. Unfortunately it seems like every company has their games you have to play in order to actually get TO the technical folks.
Of course you're right that it's not impossible. But here's why it happens anyway and why the heuristic of them being roughly mutually exclusive is not insane:
1. There's a certification that's nearly meaningless because it's so easy to obtain without also having the relevant expertise that the certificate is supposed to represent.
2. People who are actually good at the thing will notice the certificate doesn't measure the skill correctly, and will also note that there are people in the world with this certificate who don't know the skill.
3. Those experts will not use the certificate when they hire people, and will not get the certificate since it doesn't work and no one who is an expert is using it to hire anyway.
4. Meanwhile, there are basically only two groups that care about the certification:
> a. People who are clueless: a clueless hiring manager who doesn't understand the domain they are hiring for so they are looking for cheap proxies for skill and experience. Clueless wannabe professionals who don't understand the domain or the industry enough to have been in the expert group above but who are still looking for jobs in the field. Clueless clients who are impressed by the certification because they don't know any better.
> b. The people who are taking advantage of the clueless clients: Professionals and hiring managers who know very well that the certification is worthless, but who use it for sales and marketing anyway because it mollifies clueless clients.
If that trend holds, then you have a signal (the certification) the repels experts, while attracting the clueless and those who would exploit them.
So when you find someone in the world with that certification, you should expect on average for them to be clueless or preying on the clueless. That's why treating it as mutually exclusive isn't insane heuristically, even though it's not impossible that someone who is good also has a certification.
Plus there's the financial outlay of it; costs to obtain the course material and sit the exam.
These points (including yours) basically amount to my bias against obtaining certificates.
I'm not against the material itself, if it's useful. But I'm not in the habit of paying for or spending time on education for anything other than the value of my interest in the material. A piece of obsolete paper does nothing for me at this point in my career.
YMMV
It is no surprise that experts hiring experts in any field couldn't care less about certifications (aside from ones required by regulation). At the expert level, you hire real, verifiable experience.
Your HR department needs avenues to sift through referrals and comparison points. If an individual has the certificate and compares equally with a non-certificate candidate, the first individual has signaled, through the certificate, that he is interested in the field, as well as willing to invest time and resources into advancing in that field. This is the flipside of the employee signalling.
If an applicant applies without the certificate to a post which requires it, he will recognize his cover letter/interview should make a point of demonstrating competence in the area of the certificate. This is an instance of employer signalling.
If an applicant fakes having acquired the certificate and there is an easy way to determine whether or not he is faking, then HR now has an easy sieve for removing employees willing to misrepresent their skills to obtain the job. This is a step in the HR QA process.
Some employers work outside the 'work for hire' states. If they contain a certificate requirement in a job posting with enumerated skills, they can point to ineptitude in the certificate skillset as reasons for dismissal depending on the jurisdiction they're in. This is a tool in legal's toolbox.
Perhaps the certificate system is the result of the collaboration of a number of employers attempting to pool credential and training requirements for an industry, then create requirements for credentialing into legislation to control the industry pipeline of workers. This is a method of signalling to legislators or restricting competitors by indirect means.
And so on.
Additionally, your fact pattern has a small problem: expert acting as hiring managers who have knowledge of the problems with the certificate are free to adjust their screening procedure to obtain more fully vetted candidates. The only time avoiding the certificate entirely is when the signal it provides is negative.
This doesn't mean that certs are useful, merely that they might be useful to stakeholders you aren't considering.
> The only time avoiding the certificate entirely is when the signal it provides is negative.
And further, I think all your examples are perfectly valid, real-world examples that I don't dispute exist and that lots of people find important. I also think perfectly good and reasonable people operate in the reality of their industries and play ball with these things when necessary. AND I think all the use cases you mentioned are bad for the system overall. As in, they are real, and in a practical sense we can't just ignore them, but ideally we wouldn't have them.
HR using negative signals for filtering is bad. Job requirements tailored to the actual job are good. Broad and mostly arbitrary requirements from a third party are bad. Applicants to a well specified job also know to address weaknesses in their cover letter. Specific and well-specified requirements are also useful in legal disputes. Using arbitrary requirements to provide legal for firing people is bad. Employers colluding to control the training pipeline using arbitrary requirements encoded in law is bad.
I agree with you that they useful in the real world, but I'm arguing that that usefulness is evidence that the system is worse than it could be.
The real root of the issue here is that HR can't do their job. An unintended side effect of all this signalling is that future employees will be glad to get filtered out because they don't want to work somewhere that hired on certification instead of competence.
It is possible for the same action send a positive signal when done by some people and a negative signal when done by others. Specifically there can be 'contersignaling' [0], which is basically signaling that you don't need to signal.
[0]: https://kelley.iu.edu/riharbau/cs-randfinal.pdf
http://www.kalzumeus.com/2016/12/30/kalzumeus-software-year-...
Book: Web Application Hacker Handbook http://www.wiley.com/WileyCDA/WileyTitle/productCd-111802647...
I've seen it highly recommended and if you're not familiar with the field it's a good overview of exploit types for web apps.
Online training for free or cheap: Cybrary - mostly okay, but free.
PluralSight - https://www.pluralsight.com/browse/it-ops/security
Coursera has a Cybersecurity Fundamentals specializationd that's pretty good - https://www.coursera.org/specializations/cyber-security
Other books, if you wanted to go down the reverse engineering route:
Assembly Language Step-by-Step: Programming with Linux
The IDA Pro Book (for the strangely hard to buy IDA Pro, but the free version is pretty good)
Practical Malware Analysis
There are physical access controls, personnel assessments, probability and impact assessments, budgeting, people-monitoring, process analysis and modelling...
Computers are a tiny part of it. This being HN I have understanding for the bias though.
But I wasn't in management, and I've since learned how very little technical skill you actually need to be an effective manager, and now I realize that certs are a great way for a manager to understand a complete baseline of the concepts needed for a particular field.
A manager does not need to be a hacker, but they need to understand a baseline of security concepts. That's what certs are really useful for.
I wouldn't be able to get one despite having experience because:
For several of them it requires years of work experience with a specific job titles (your job needs to be security, it can't just be part of your job) and the continuing education credits are expensive and largely not helpful.
I would love if there was a recognized certification that was actually interested in proving your skills not for making money.
I know other disciplines are the same way.
They don't really serve to keep unskilled people out, they serve as kind of an elite paywall where you need time and money to break through. And breaking through is largely an exercise in brute force not in skill.
http://www.rsync.net/resources/regulatory/pci.html
Their web app challenges were fun too. LFI to code execution, SQL injection, things like that. They have a bunch of network related recon, standard red-teaming stuff.
The OSCE involves ASLR bypass, AV bypass, and using egg hunters.
The big thing about the OSCP, OSCE, OSEE certs is that you actually have to _do_ all of the stuff they teach you. Not a multiple choice or written question in sight. For the test they drop you in a network with vulnerable machines and you have 24, 48, and 72 hours (depending on the cert) to get code execution on each through various techniques. It was challenging, interesting, and satisfying.
Edit - it's worth mentioning that I still find vanilla buffer overflows on projects. These days most thick-client applications that I see are old as hell and are still vulnerable to exploitation techniques from decades ago. So while the skills that the cert makes you prove are cursory and introductory, they are still useful. In any case it's a good starting place for those that want to learn stuff on their own but do better when they are given the push to prove it.
That being said, the nice thing about OSCP imo is that it gives you some structure and a well set up environment to play in. I think OSCP is a great entry-level certificate and serves as a good filter to interview junior candidates.
Does this help at a more elite level, nope, but that's also not the purpose of it.
Red flag: someone that has an email signature with 50 letters next to his/her name,there is NO WAY someone is spent enough time on each: coding, security, audit, accountancy, at the age of 30 AND be proficient in all these domains.
Or it means they can drop the same jargon and have maybe a passable understanding about the ideas that jargon is meant to convey.
Often people with extremely narrowly focused bases on knowledge (as indicated by having BAs, MAs, and a giant rap-sheet full of certs all in the same topic) have wound up being thoroughly incapable of actually applying any of their knowledge to the benefit of the team they're on because they just don't know how to get what's in their heads into the heads of people who weren't steeped in the same language as they were.
I'd keep them if they are and lose them otherwise.
What happened was:
IT boss: "we need to hire a coder"
HR: "What skills?"
IT Boss: "Oh, Foo language. But if they're a good coder they can pick it up, so just a good coder"
HR: "...you're kidding, right? There are millions of resumes out there, most from people with no skills just trying to land a great job. Give me enough to filter"
IT Boss: (provides list of three things)
HR: "This still isn't enough. Practically EVERYONE will have these. Give me years of experience, skillsets, processes, etc.
IT Boss: "Fine, here" (gives long list of things that MAY be useful)
HR: (starts filtering resumes based on these words, removing lots of good people and including lots of bad people)
IT Boss: (looks at resumes) "These people are clearly lying and all over the place, I'm going to focus on one or two things to decide who to interview"
So in writing your resume, you want to make sure you have the buzzwords for the job to get past HR. These buzzwords are pretty much guaranteed to be on the job listing, even if they end up not being very essential to the job. Did they mention Scrum? Better have it on your resume, because you may be filtered out if it isn't, even if it's something you'd not consider worth listing. Also, use the same words. I once was asked if I had "shell experience", even though BASH was on my resume. I assume "Agile" and the various implementations are the same. If they mention XP, you better mention XP.
BUT when your resume then makes it to IT, who (1) know what these words mean and (2) aren't looking for the same things at all, you need to have what they want. I tend to use a sidebar on my resume to capture the HR buzzwords, and emphasize my work experience in the main body, so an IT person skimming it will see what they want to see.
One technique I've taken to handle HR buzzwords on things that I don't think are actually a big deal: If the job listing says "Must know React, Angular, Backbone, or other JS frameworks" and I wasn't really strong in any of them, I'd do enough research and testing coding to do a Hello World in them, then add "Exposure to Foo, Bar" on my resume. It tends to get through HR (word is present!), and I'm not lying to the IT people - they understand that I'm not claiming expertise, but I'm also saying I'm willing to give it a go.
As a corollary to all of this, you need to tweak your resume for every job posting, to match their buzzwords and remove ones they didn't list that aren't really core to your skills.
However when we're working at scale, certifications can be useful as providing a demonstration that the holder has some level of exposure/knowledge of the arena in question.
what complicatates this quite a bit is that some of the more popular certifications are, rightly, not considered that good as the process to get them lends itself to rote learning. So things like the CEH and CISSP where the exam is multi-choice, not so great.
On the other hand things like the CREST CCT definitely require a decent level of knowledge to pass and you need to be able to apply that knowledge in a practical situation and in time limited conditions.
I find the anti-certification bias in IT and security a bit odd really. If you look at other professions (e.g. law, accountancy, architecture etc etc) it's recognised that these things are required to get a minimum level in place in situations where you have a large body of people, I don't really see why IT Security should be any different.
To me, the problem of "these certifications are bad" should have an answer of "lets make better certifications" not "lets not use certification"
There was no real memorization needed, and I had the full man pages of the operating system at my fingertips, but the time limit ensured that I needed to have at least a certain degree of proficiency with each tool to finish all of my tasks before the end of the exam.
I assumed this was the norm with certifications, but the comments here strongly suggest that it is not. Are Security Certifications really multiple choice questions without any practical applications? That seems like it could stand to be improved greatly.
One problem is that some of the "bad certs" e.g. CEH, CISSP , are well established.
My feeling is that the answer isn't "don't have certs" but "have better certs"
So not my dichotomy.
I was lucky enough to start early when the only thing you had to do was to show your skills and willingness to learn. Those days are long gone but you can still prove yourself by delivering awesome security research or commentary - and people will hire you based on that.
That being said, any IT field which requires some sort "technical" certification to get hired is probably not the field you would like to get into. Why? Too much competition and race for the bottom line.
Luckily this is not exactly the case with Information Security - yet. In our line of work there are people who do a lot of the uplifting where certifications do matter. I would not say these are very technical jobs and they are totally dispensable. Other professionals acquire specific skill sets which are rare or difficult to obtain and as a result they tend to have the upper hand. There is of course a lot in between.
I do not mean that everyone who has certifications next to their name sucks. Not at all. But unfortunately, unless you are applying for a commoditized IT security field or security manager type of role, it will raise some questions.
Keep in mind that HR is also partially to blame. Many people do not know how internal HR teams typically work which is essential to understand why certain things happen the way they do when hired or when progressing through the ranks of a company.
HR are typically not technical and they are not experts in security either so they do not know what exactly they should be looking for. Of course they are not completely clueless but a seasoned hacker can smell bullshit a far while a well informed HR cannot. HR's filter is your CV and the job spec and these two are simply not enough to evaluate a successful candidate. Some job specs are written by HR themselves which is crazy because they are not in the position to formulate the role so they have to stick to what is known - i.e. certifications.
That being said there is a shortage of IT security professionals. As a result of that almost anyone can get hired if they show the right attributes. It does not take a long time.
The only thing that bothers me with the InfoSec industry these days are not the certs but that companies tend to have really bad hires (maybe due to bad certifications) who due to lack of deep understandings of the subject work on things that practically do not matter. As a result of that these companies have the illusion that they are doing something while the fact is that they do not in a practical sense. This is why in many places no one knows what the security department does - I am not kidding.
In other words: if the company asks for certs it's the equivalent of wanting "6 years of react.js experience". I completely agree.
It's interesting that his argument is against certs for companies but then he looks at regulation as unchangeable.
There are a couple of exceptions, of course. OSCP is a good certificate to have. To pass the exam, you are required to not only demonstrate proficiency in several areas (i.e SQL injection, buffer overflows), but you must also write and submit a technical report to a review team. The technical report must address vulnerability overview, impact, risk rating, reproduction steps, and more. Of course the exam isn't perfect, but it's probably the biggest test of real technical understanding and ability I've ever seen.
I vehemently disagreed, pointing out that those of us with meaningful degrees from well regarded programs shouldn't have to get one, but in the end I complied to keep the peace.
Have to say I enjoyed it at the time and even learned a thing or two. I wasn't exactly new to offensive work, but was nice to get things sharpened up and familiar with a few different approaches. I remember being a bit worked up about the final challenge at the end of it that I took the day off work, but iirc I was able to finish in just a couple hours. It wasn't the most cutting edge content at the time, but it was well organized and ensured that the person being certified could demonstrate some level of practical application and that is far better than most certifications of any sort.
Break it down to a 4th grade level, if you can't, you likely don't understand it yourself.
I would agree that in general certs have too much weight, but the reality of it is, as it stands, they're the closest bridge / standardization that the market has built for non-Security ilk.
Also, Security is not IT. That's another thing that needs to change.
As for OSCP, I think it's great and out of all the certs i've taken for various reasons, it's the only one I felt challenged with, in a good way.
Do you mean that
or do you mean ? The amount of stress that you mean to put on the word "not" doesn't come through very well in this medium.Security should have an advisory role to the corporate governance folks who maintain policy... usually the lawyers.
Edit: And, in contrast, I've met people who I had a hard time communicating with in spoken English but, had no problems with once we did our communications in writing or with a written point of reference.
A well written report that speaks at the right level to its audiences will generate more proactive security activity than a terse, passive voice bomb of dense technical information.
Equally importantly: memory corruption exploit development and SQL injection are different skills, and most people who do SQL injection don't need proficiency in "buffer overflows". Why is superficial coverage of "buffer overflows" part of the rubric for that certificate? I don't know, and I don't know that anyone else does either.
Is there a single coherent security certificate anywhere in the industry? I'm interested in examples.
Total tangent but, I am absolutely grossed out by "cyber" winning out in the name game. Who let the DoD drive that? Damnit!
The leet kids, a minority few and the rest naïve, I would bribe while they whittled away at online CTFs and MicroCorruption and irritate them with mediocre questions until they tuned me out. I did not care for tools; approach and mindset are order of magnitudes harder to explain.
I thought you could be a wakeup call had you told them all the certs pro se is a waste in a program that pushed that nonsense. I talk trash of my certs and skills the whole time and they did not get why.
I know you're busy, but I've read your blog and you're preaching to the choir. Starfighter folded, but I would pay for you to test me as a customer and find a mentor to answer my stupid questions that I would pay handsomely for the privilege. I feel I'm not the only one, if you get tired of NCC, that would be amazing!
I don't think Offensive Security is trying to pump out exploitation experts from their entry level cert program. Maybe the higher levels OSCE and OSEE. The intro cert emphasizes breadth over depth. It felt a lot like a cert built around the Exploitation Hackers Handbook.
I think you're thinking of the certs in the wrong light. They are meant to validate baseline knowledge and proficiency, not mastery. If you want to validate mastery you need to look at the persons personal record and work product.
It's the only one I ever thought might be valuable since it combines specific knowledge with some pentest against real systems. Curious what you think as you're more qualified to assess such a thing given all your security evaluations and hiring those people.
As much as I admire people who show the courtesy to the trash bin that will eat their report, before any human reading it, of not feeding it "bad" reports: The OPs point of "sense of false security" unfortunately already sets in once someone is hired to "take care of security". This anecdote from about 2000 illustrates that:
A friend was hired to do black box penetration testing on the DMZ and internal network of one of the largest travel agencies in Europe. He, of course, found a lot of problems. He wrote a nice report. Like really nice. Point for point "this is your problem, this is what you have to do right now and this is a user-friendly policy you could implement to prevent such issues in the future." It was very pleasant to read, he knows how to handle language. And somewhere on page hundred-something "the first one to claim it, will get a bottle of champagne!".
The bottle was claimed a few months later by someone 3 levels above his position. By some high-level manager who did not know anything about IT. But he was the only one to read it (and he only claimed the bottle to figure out whether anyone below him had actually read the report). Two years later company politics allowed those 2 illiterates between my friend and the one manager who could read to be removed from the company. My friend was hired as their chief of security. His first task: work through his own report to fix the 90% that had not been fixed since he wrote it 2.5 years before (the report listed passwords to core routers in plain text as examples for "very stupid passwords" - they all still worked).
I personally organized my report into three sections, which seemed to work well. Clients seemed to enjoy the formatting:
1. Executive - Summarize everything in one page at a high level. You could skim it fast if you chose to. Highlight potential negative business impact of each finding.
2. Management - A little more detailed. 2-3 pages max. Most severe findings at the top and recommended action for remediation.
3. Narrative - This is the bulk 80-90% of the report detailing your step by step process including screenshots so that if someone wanted to duplicate your findings they could.
This was for using citrix boxes to provide saas to a client
[1] It didn't help my perception of the report when it screamed that "ICMP echo was enabled and nefarious scalawags might be able to do unspeakable acts against our computers, best cut the network cable" type of advice [2] (yes, I know port 443 is open! WE'RE A WEB HOSTING COMPANY SERVING UP COMMERCIAL WEBSITES! ARE YOU PCI AUDITORS STUPID?)
[2] Okay, the ICMP echo thing was reported, but did not need to be disabled to pass the audit. If so, why even bring it up?
It appears to say that if you ever hung out on IRC and tried to keep your handle private, you're ineligible.
"Omit user identities or screen names with which you were publicly identified."