And it is still garbage. Can it differentiate between the declaration, definition, and calls of a function Foo, versus just substrings "Foo" found anywhere in the file, such as comments? Nope.
Kythe was slightly after my time on Grok. It was a side-effort by the Grok team, and it's frankly a miracle they got it out at all. Google only really supports a few big open-source projects - Chrome, YT, k8s. Kythe fell waaaay below the line.
I'll be taking another look at Kythe, and reaching out to the current Grok team, as we expand scip. But ultimately it didn't matter what protocol/format we standardize on. We just need to standardize. So it's scip!
I loved it when I was younger, but I had trouble recommending it more recently because its tone is adjacent to personally attacking people who like Java.
Agree, but it helps to remember that it’s a product of its time. Feels like we’re overdue for a similar piece poking fun at Node/JavaScript and all of its foibles.
As I get older, I do tend to be more conciliatory, but I do think that even the people who like Java are perpetuating pain and struggle for others that will have to learn some lessons the hard way.
I remember in 2008 he predicted that text editors/IDEs will have to compete against the browser. When Jupyter notebook hit the scene I thought how prescient he was. There's a move toward bringing back the editor/IDE-like environment but I suppose with VS Code being based on electron it's just another form of the browser.
Our code is public, regardless of whether it is open-source or enterprise licensed. The open source code includes most of the product features; the enterprise code includes things like enterprise AuthN, AuthZ, and admin capabilities that are needed for large companies to use Sourcegraph.
Sourcegraph.com/search indexes over 6 million open-source libraries, nearly every github.com repository with at least 5 stars and the most popular projects across NPM, PyPI, Maven Central, and many more independent projects and code hosts.
Isn't the main caveat of the OSS version that you can't use any extensions? Last time I checked, most of the stuff that made Sourcegraph actually interesting for me were extensions, so the OSS version was fairly limited in what it could do.
I can't help but read Steve Yegge's announcement in the context of Novig's Law ("compiler research leads to a doubling of compute power every 18 years")[0].
If his presence and enthusiasm can get the compiler community aiming its collective brain power at real developer productivity problems (the why behind SourceGraph's exponential growth) rather than focused on compiler optimization problems, this is going to be a really great time to be writing software!
Small nit: if you actually read your citation, it's not Norvig's law, but Proebsting's law[1]. It's a pessimistic spoof of Moore's law, but IMO just reflects how much closer to optimal we started from than the first transistors were. Or maybe how much of a head start we've had doing this on paper for centuries.
Like, I would frankly be surprised if we ever got more than one doubling out of Proebsting's law.
You're right - Norvig is a much easier name for me to remember than Proebstring, which probably has something to do with why I misfiled the law mentally.
Reading your cite, Norvig's law is different. He credits Todd Proebsting with that observation. (Peter Norvig's law is that something that is at 50% adoption will never double again.)
* Precise code navigation (vs. more fuzzy-level nav), powered by SCIP (spiritual successor to Grok, the system Steve built at Google)
* More powerful search language beyond regex (supports comby.dev) + user-friendly (smart search)
* Works across multiple GitHub instances + other code hosts (GitLab, Bitbucket, Perforce, enterprise Git repositories)
* Self-hosted deployment and enterprise scale
cs.github.com is a significant improvement over github.com/search—kudos to the team there!—but is about feature parity with something like OpenGrok, Hound, or Google Code Search before Steve built and integrated Grok (primarily cross-codebase regex search).
> cs.github.com is a significant improvement over github.com/search
It's a significant improvement, but still frustrating to use (especially after being spoiled by Google's codesearch). Sourcegraph has been far more pleasant to use in my experience (e.g., faster and more relevant navigation).
I'm trying to figure out if Sourcegraph requires git and it's not clear to me. I saw a page about non-git code hosts but it looks like it still builds a git repository to mirror the actual repository. Is that correct?
LSP has addressed the editor code nav problem, but not the "just works in my web browser across all my code and multiple revisions" problem or the "let me build on top of a structured representation of code to enhance all my dev tools" problem. LSP's sister protocol LSIF aims to address the indexed code nav use case, but Steve calls out some of its shortcomings in the post, and these motivated our development of a new protocol, SCIP: https://about.sourcegraph.com/blog/announcing-scip
But what if Google+ doesn't exist, possibly some posts aren't written. SNS encourages people to write something. I remember some Linux/Unix developers or Googler wrote good posts on Google+. Perhaps they still write at somewhere but I don't know.
It is interesting reading that second paragraph many years later. Most of the things that Steve Yegge brags about that Google "does right" (e.g. how they do recruiting, their engineering "standards", SREs running products rather than the engineers, their cushy offices and benefits packages, etc) now read to me the opposite of the intended way. As in, they read more like a list of reasons why Google slowly declined from being the shining city on the hill to the dysfunctional embarrassment it is today.
Having recently left Google after more than 10 years, I don't think those are the reasons for any decline at Google. I think it's very hard to scale a company to that size without many layers of middle management, and I think it's very hard to be an effective middle manager at Google (and maybe anywhere).
Until I see a more functional company with 100,000+ employees, I attribute all the issues to scale.
Also the guy who wrote the follow up to Joel Spolsky's brilliant "Smart and Gets Things Done" article with one entitled "Done, Gets Things Smart" which is nearly a tome and also brilliant.
He's one of the first developer bloggers. He wrote a load of famous articles, the echos of which you'll have been exposed to even if you've never read them.
Bazel support in what sense? We have basic code nav across every major language, select precise indexers known to work with Bazel (with some degree of work), and we have improved code nav for Bazel files themselves (https://github.com/sourcegraph/sourcegraph/pull/37858), too. Search, of course, works across all languages. Happy to chat more about supporting your specific build environment: https://discord.gg/mBVP4JVBcj
For C++, we do support Bazel via compile_commands.json; we have customers who have used it successfully. Depending on the editor you're using, you probably need to get Bazel to generate a compile_commands.json anyways.
So core code navigation functionality ought to work. There are some issues with different language features (macros, newer C++17 and C++20 features), as well as robustness (crashes on indexing certain code) but we're looking into bringing C++ support up to par with other languages.
I've watched that Grok video a dozen times and drooled at the possibilities if it were available to the world at large. And now it is!! It'll also be nice that people will remember that Grok came long before LSP :P
Someone help me understand what I'm missing. It sounds like in this article Steve Yegge is describing a tool like Atlassian Fisheye or grepcode and talking about it like it doesn't exist. These tools are out there. I don't see what's missing.
I've never tried grepcode before it's gone (not a Java guy), but what I miss most is precise reverse search, i.e. "find all reference", instead of "go to definition". For example, try Chromium code search:
* Works cross-repositories, over our indexed corpus of 6M OSS repositories, or your private code
* Multi-language (some projects at compiler accuracy, some at CTAGS-level accuracy)
"This was the first leadership interview loop in the past 12 months (20+ companies) in which anyone had asked me to write code."
Interesting data point for the question: at what point in your career will you stop being asked to write code on a whiteboard to prove you aren't lying on your resume.
I’ve never been through an interview cycle where candidates didn’t lie about this, so I assume pretty high. It always boggled my mind that people would claim programming skills but not be able to write hello world.
If its not the pressure cooker, this is trick question and i am going to blast you for missing the return and void in parenthesis types question then we can partially blame the candidate.
Many of us have faltered like this once or twice unless its an outright lie kind of situation.
I’m not a fan of trick questions. It’s usually something as simple as “you have 10 minutes write out a program in any language you like that takes in a string and prints out the reverse.”
The idea is just a litmus test and I don’t think it’s useful for filtering anything other than liars. I never had someone completely freeze up.
I'm really curious to see how this plays out. My ex-roomate has a very bad experience working at sourcegraph and the way he was pushed out of the company left a bad taste in my mouth. I'm curious to see if Steve's brand is strong enough that he'll have the power to be able to improve the engineering culture, my fear is that this is a hire primarily for external optics reasons.
Anyway, just sharing an anecdote and hope this works out well for all involved.
I really like the idea of sourcegraph but the price seems bonkers. $100/month/user is more than I spend on ides, more than GitHub, the same as GitLab ultimate (that I don’t use because it’s so expensive), more than o365, more than windows, etc etc etc.
I want to have source intelligence but I can’t see the biggest chunk of my dev stack to be sourcegraph.
2
write a lengthy, self-important diatribe/novella about why he joined
3
Write several lengthy, self-important diatribes, often namedropping previous places he worked and/or how he accidentally influenced some C level officer just by dint of his unique persona =) That rascally Steve!
4
Repeat step 3 anywhere between 10-50 times
5
Quit job, write lengthy self-important diatribe about why he left (optionally leaping straight into step 1 again, sometimes with a break inbetween)
Should be an interesting 6 months for Sourcegraph, at least! Looking forward to seeing how this progresses.
78 comments
[ 2.3 ms ] story [ 144 ms ] thread1) How can you write this whole article without saying "Kythe"?
2) How exactly can github search be as bad as it is? With all of Microsoft behind it, you'd think it would be a lot better than it is.
Finding a technology preview for a radically improved cross-language code search capability within GitHub was an unexpectedly nice surprise.
Well, it does discuss Grok (the closed source predecessor) extensively.
I'll be taking another look at Kythe, and reaching out to the current Grok team, as we expand scip. But ultimately it didn't matter what protocol/format we standardize on. We just need to standardize. So it's scip!
Youngbloods: if you have not spent time with Steve Yegge's old writings, please go check them out. Much good received wisdom there.
Oh cool - AFAICT, that "our code" link is a link to a demo instance of sourcegraph on sourcegraph's code. This looks like an interesting product.
And it's open source? What's present in the commercial offering that's missing from the open source one? Just support, or features too?
Our code is public, regardless of whether it is open-source or enterprise licensed. The open source code includes most of the product features; the enterprise code includes things like enterprise AuthN, AuthZ, and admin capabilities that are needed for large companies to use Sourcegraph.
Sourcegraph.com/search indexes over 6 million open-source libraries, nearly every github.com repository with at least 5 stars and the most popular projects across NPM, PyPI, Maven Central, and many more independent projects and code hosts.
If his presence and enthusiasm can get the compiler community aiming its collective brain power at real developer productivity problems (the why behind SourceGraph's exponential growth) rather than focused on compiler optimization problems, this is going to be a really great time to be writing software!
[0] https://norvig.com/norvigs-law.html
Like, I would frankly be surprised if we ever got more than one doubling out of Proebsting's law.
[1]: https://www.gwern.net/docs/cs/algorithm/2001-scott.pdf
* Precise code navigation (vs. more fuzzy-level nav), powered by SCIP (spiritual successor to Grok, the system Steve built at Google)
* More powerful search language beyond regex (supports comby.dev) + user-friendly (smart search)
* Works across multiple GitHub instances + other code hosts (GitLab, Bitbucket, Perforce, enterprise Git repositories)
* Self-hosted deployment and enterprise scale
cs.github.com is a significant improvement over github.com/search—kudos to the team there!—but is about feature parity with something like OpenGrok, Hound, or Google Code Search before Steve built and integrated Grok (primarily cross-codebase regex search).
Rust Dependencies (crates.io) Python Dependencies (PyPI) Go Dependencies NPM Dependencies src.fedoraproject.org JVM Dependencies GitLab.com GitHub Public
and as of yesterday Savannah/GNU - https://github.com/sourcegraph/community/issues/38
More to come.
We're currently exploring what it means to support non-git VCSs natively in Sourcegraph, but we're not there yet.
https://sourcegraph.com/search?q=context:global+repo:%5Epyth...
Adding to my list: https://github.com/sourcegraph/community/issues/36
Until I see a more functional company with 100,000+ employees, I attribute all the issues to scale.
Check it here: http://steve-yegge.blogspot.com/2008/06/done-and-gets-things...
His first set were from his time at Amazon: https://sites.google.com/site/steveyegge2/blog-rants
He then wrote a load of later ones when at Google: https://steve-yegge.blogspot.com/index.html
And, yes, the (in)famous platforms rant.
While the blogs range widely, a major overarching theme was static vs dynamic typing.
Basically everything he wrote is worth reading; and definitely a thousand times more that whatever's on the HN front page these days.
https://github.com/sourcegraph/sourcegraph/issues/2982
Here's some docs on Bazel support for Java/Scala precise code nav: https://sourcegraph.github.io/scip-java/docs/getting-started.... Any languages you're looking for in particular?
So core code navigation functionality ought to work. There are some issues with different language features (macros, newer C++17 and C++20 features), as well as robustness (crashes on indexing certain code) but we're looking into bringing C++ support up to par with other languages.
https://source.chromium.org/chromium/chromium/src/+/main:v8/...
Click on the "ToBoolean" function, it brings up all call sites across all Chromium projects.
* Works cross-repositories, over our indexed corpus of 6M OSS repositories, or your private code * Multi-language (some projects at compiler accuracy, some at CTAGS-level accuracy)
Interesting data point for the question: at what point in your career will you stop being asked to write code on a whiteboard to prove you aren't lying on your resume.
Many of us have faltered like this once or twice unless its an outright lie kind of situation.
The idea is just a litmus test and I don’t think it’s useful for filtering anything other than liars. I never had someone completely freeze up.
Anyway, just sharing an anecdote and hope this works out well for all involved.
I want to have source intelligence but I can’t see the biggest chunk of my dev stack to be sourcegraph.
1 Join a company
2 write a lengthy, self-important diatribe/novella about why he joined
3 Write several lengthy, self-important diatribes, often namedropping previous places he worked and/or how he accidentally influenced some C level officer just by dint of his unique persona =) That rascally Steve!
4 Repeat step 3 anywhere between 10-50 times
5 Quit job, write lengthy self-important diatribe about why he left (optionally leaping straight into step 1 again, sometimes with a break inbetween)
Should be an interesting 6 months for Sourcegraph, at least! Looking forward to seeing how this progresses.