The third option is that they are adding features to a laid product. I already pay a lot for enterprise GitHub and the search is really bad. This will help me not leave GitHub and motivate more people to signup and use it.
I've been working on building plugins on top of JupyterLab and their documentation leaves much to be desired. I usually have 15 tabs open on their public GitHub repositories learning how to use their own API. GitHub does a decent job of parsing symbols (class names, etc.) so you can double-click on those to see dependencies and references of some symbols quite helpful. Of course this feature breaks down when code is split across multiple repositories.
Also one thing that has been missing from all these code search engines is tracking how a variable might traverse through the code. In cybersec this is called taint analysis. But for my purposes it greatly helps with just program understanding.
My .02 is whatever GitHub does here should be directly integrated into the main GitHub UI.
For the other comments who wonder. in their announcement [1] they said that this is for beta trial and then once they are ready it will be integrated into main github search
I've been using it for a couple months and it's kind of a weird experience. If it works you can feed it regex and stuff which is nice. But sometimes it will just tell you that it doesn't know anything about the repository, or it will miss results entirely that the normal GitHub search (bad as it is) finds easily. And the website is this weird div soup that doesn't really let you copy or anything, which makes it kind of frustrating to work with.
I know they’ve said it’ll be integrated into the main GitHub site, but the way it’s pitched on the landing page makes me wonder if they’ll roll it into an individual subscription like the Copilot product.
Sourcegraph wisely chased the enterprise market, so use it to index your private repositories. Plus it can other things such as finding all definitions and modifying them across multiple repositories. They don’t really exist in the same space, at least yet anyway.
Also there is no money to be made in public code search. Take my word for it. However it is something that all code hosing repository should probably have as a useful too, even if it’s not a direct revenue stream.
I mean… it feels obvious that this will index private repositories for users authenticated to access those repositories?
I don’t think this will be a separate paid product, just a standard feature as part of paying for GitHub. Sourcegraph will be in the same position as every other SaaS vendor that tries to compete with Microsoft is: a bad one with a worse goto market strategy than MSFT.
Sourcegraph CEO here. I posted my thoughts on the positioning. Code search for Microsoft/GitHub is not a winner-take-all proposition unlike many of their other products (eg Microsoft Teams vs. Slack), so the analogy is a bit different. A lot of code lives outside of GitHub, thanks to developers historically having more freedom and choice. See https://news.ycombinator.com/item?id=31968184. Would love your thoughts on this.
I've been in the alpha for about six months now and I love this - I use it several times a day.
It's become my default option for API documentation: given basically any API I want to use, this gets me real-world examples of it in seconds - often far more useful than the official documentation.
I wish they would have some concept of "reputable" projects, which would be shown more prominently. For example, if I'm trying to search all of GitHub for a certain language API or library reference, I would rather get code examples from large actively maintained repositories first instead of 10 forks of the same long forgotten repo.
I've tried it for several months and compared the results to https://grep.app (I normally use it many times a day) and while GitHub cs has way more indexed repositories, the results of grep.app are still way better and more helpful to me.
I remember back in the there was code.google.com (or something like that) which allowed searching over repositories. I was assigned to creating firefox addon including native part with C++ and it was kind of daunting task, there were little documentation. This tool was indispensable for me, it allowed to quickly look at any API usage. I wouldn't be able to complete that project without it.
Since Google closed this project, I never used anything similar, but I think that I should be. I tried few times to use ordinary Github search, but it wasn't good enough to be useful. Hopefully this project will change it. Done properly it would be a very powerful tool.
What's not clear to me is whether that tool searches only on Github or on all repositories in the Internet? Hopefully it's the latter.
Awesome! I sometimes search across very niche projects on Github, where their codesearch doesn't show the results I want. I KNOW that the result exists, and can see it in the repository, but their codesearch doesn't show the results I want.
So I can get a better sense of how to use this tool, if I wanted to see examples of say a 'Users' collection API from multiple Django projects, how would I do so? Looks like I can do keyword matches and language specific filtering but would love tips beyond that!
Ha ha this is going to end up with paid advertisements formatted as code comments in underfunded, but popular OS projects. Oh god I really hope that’s not going to happen.
I don’t understand why I would ever need to do this?
If I need to replace or find something across all repos in my organisation, or god forbid, Github, then something went extremely wrong architecturally.
I use the current search to answer stuff like “what projects are using GitHub actions” by searching for the action yamls and look forward to this new tool working better.
1. When your code is too big for your IDE's built in global search.
2. When you want to share searches with others.
#2 is great for design docs. You can say "<link> shows the bug is common in API usages" or "<link> shows x% of our files do this"
Edit 3. When something does go wrong.
> If I need to replace or find something across all repos in my organisation, or god forbid, Github, then something went extremely wrong architecturally.
If your plan B for messing something up is to not have a plan B you're going to have a bad time when plan A fails :)
This is very useful to see examples of how people have used APIs that are either poorly documented or not at all. Or even that are well documented, really. Going from docs to code is not always straightforward.
To give you just one example, recently I've been using it to find how people have written code to interface with e-ink displays. I usually have the datasheet which lists all the commands the protocol support, but building it all into a valid startup sequence of ~20 commands to activate the display is left as an exercise for the reader.
Hard disagree. A very common pattern we face is we bugfix an internal npm package and then have to roll out that change to a large subset 200 microservices that consume the package. Our goto right now is sourcegraph search to figure out what the subset is.
GitHub code search has been terrible since forever. They keep braying about preview stuff, AI search, etc. but when will this make it into their mainstream (let alone their on-prem) Enterprise product? (This has been in preview since December last year, but https://github.blog/2018-09-18-towards-natural-language-sema... was posted back in 2018 yet is still to be released.)
I agree that semantic code search would be a nice tool in exploratory and onboarding scenarios. The GitHub blog you posted and the CodeSearchNet project was a catalyst that got me interested in semantic code search. That's why I've been hacking on https://codesearch.ai in my spare time at Sourcegraph, which implements a version of semantic code search w/ transformers.
I've been using the preview for a few months, it works pretty well. That said, one thing I miss from the regular code search feature is the ability to use a form to filter search results, or to refine results using suggested filters after an initial search.
The regular search results page usually has a few pre-set choices that you can quickly click on to refine the results, like a list of languages found. Code Search supports many such filters[1], and even supporting suggestions or completion in the search box would be a good improvement. With the new UI once you're on the search results page the examples of the home page are no longer visible, so I never know if I should use lang:py or language:python, or if it's path:*.txt or filetype:txt.
Interesting because on https://lsif.dev/ I see that LSIF support for C++, which basically is just a wrapper around clangd AFAIU, is deprecated. Is there something else that replaced it?
Hey, I’m the developer of a popular tool for searching all of public GitHub for sensitive information (GitHub.com/tillson/git-hound) … would love to get access to play with it and see how it can improve secret detection. github.com/tillson
If you search something that is also used in a popular opensource project, github codesearch will then show the same exact hit in a zillion forks, completely drowning out the interesting hits.
78 comments
[ 4.4 ms ] story [ 138 ms ] threadOf course they must see a way to make money with it in the long term.
Like they did with GitHub itself: as a data source for CoPilot ;)
Also one thing that has been missing from all these code search engines is tracking how a variable might traverse through the code. In cybersec this is called taint analysis. But for my purposes it greatly helps with just program understanding.
My .02 is whatever GitHub does here should be directly integrated into the main GitHub UI.
[1] https://github.blog/2021-12-08-improving-github-code-search/
https://news.ycombinator.com/item?id=29487237 (217 comments)
Also there is no money to be made in public code search. Take my word for it. However it is something that all code hosing repository should probably have as a useful too, even if it’s not a direct revenue stream.
I don’t think this will be a separate paid product, just a standard feature as part of paying for GitHub. Sourcegraph will be in the same position as every other SaaS vendor that tries to compete with Microsoft is: a bad one with a worse goto market strategy than MSFT.
It's become my default option for API documentation: given basically any API I want to use, this gets me real-world examples of it in seconds - often far more useful than the official documentation.
Since Google closed this project, I never used anything similar, but I think that I should be. I tried few times to use ordinary Github search, but it wasn't good enough to be useful. Hopefully this project will change it. Done properly it would be a very powerful tool.
What's not clear to me is whether that tool searches only on Github or on all repositories in the Internet? Hopefully it's the latter.
https://source.chromium.org/chromiumos/chromiumos/codesearch
https://source.chromium.org/chromium/chromium/src
for more Google related projects
If I need to replace or find something across all repos in my organisation, or god forbid, Github, then something went extremely wrong architecturally.
As for searching all public repo code I am not sure I see the need unless maybe you have a super-specific error code to search for.
That would be terrible indeed. But it might happen. Good to have a powerful tool at hand in that case.
1. When your code is too big for your IDE's built in global search.
2. When you want to share searches with others.
#2 is great for design docs. You can say "<link> shows the bug is common in API usages" or "<link> shows x% of our files do this"
Edit 3. When something does go wrong.
> If I need to replace or find something across all repos in my organisation, or god forbid, Github, then something went extremely wrong architecturally.
If your plan B for messing something up is to not have a plan B you're going to have a bad time when plan A fails :)
Unfortunately, hope is not a strategy.
To give you just one example, recently I've been using it to find how people have written code to interface with e-ink displays. I usually have the datasheet which lists all the commands the protocol support, but building it all into a valid startup sequence of ~20 commands to activate the display is left as an exercise for the reader.
So the docs will look like this: https://www.waveshare.com/w/upload/6/6a/4.2inch-e-paper-spec...
And what I need is a sequence like this: https://github.com/joeycastillo/The-Open-Book/blob/5c5054c58...
• searchcode: https://searchcode.com/
• Debian Code Search: http://codesearch.debian.net/
• Python Code Examples: https://www.programcreek.com/python/
• Linux kernel code search: https://livegrep.com/search/linux
C and C++ code search: https://codesearch.isocpp.org/ (uses Debian as source)
The regular search results page usually has a few pre-set choices that you can quickly click on to refine the results, like a list of languages found. Code Search supports many such filters[1], and even supporting suggestions or completion in the search box would be a good improvement. With the new UI once you're on the search results page the examples of the home page are no longer visible, so I never know if I should use lang:py or language:python, or if it's path:*.txt or filetype:txt.
[1] https://cs.github.com/about/syntax
How do I get Safari to show as in screenshot without the Red / Yellow / Green buttons. The Z height of the title bar also seems to be thinner.
Or is that a different browser?
I would be delighted to getfast-tracked and get access to better search. github.com/fiatjaf
The domain is a bit annoyingly long but other than that..?
https://github.com/WillyKess
https://github.com/search
https://github.com/search/advanced
Is this somehow fixed in the new codesearch?