After looking, I'm astonished that this doesn't exist in Emacs. There is a completion plugin using libclang; maybe the highlighting could just be added into it.
On vim's side, YouCompleteMe explicitly denied my approach to integrate highlighting.
I hacked together something to add at least #ifdef highlighting* to YouCompleteMe. It's not quite ready to share due to some bugs, but I'm planning to release it as a patch / friendly fork when I find some time.
I believe there's no reason to have two plugins each running an instance of clang.
* i.e. showing parts of code that are ifdef'ed out in grey
Yeah, that's something I considered from the start. The issue is that, in order for that to work reliably, color_coded would basically need an embedded Python interpreter.
YCM's config file can contain arbitrary python functions and doesn't really need to follow any standards. This makes it hard on color_coded and ultimately didn't seem worth the trouble.
Have you got a plain text version of the highlighted source examples somewhere? I want to compare it with emacs' default highlighting (I think default emacs highlights almost as much as the "After color_coded" screen shot).
So it looks like color_coded can highlight at least a few things that default emacs doesn't: function calls, variables (when they are used, both can highlight declarations) and non-standard type template parameters.
The way I see it, clang is an implementation detail; this might be implemented with something else, if only anything better existed. If anything, it should reference vim. Alas, I'm a sucker for wordplay.
It can be pointed out that YouCompleteMe, for example, also doesn't have vim in the name. I guess it's not so bad!
I've always thought YouCompleteMe was a stupid name too. I guess I'm just no fun :/
I don't think clang is an implementation detail. Surely the custom syntax highlighting engine is a vim feature. It's the "using clang" that makes this interesting.
The point is that this project is impossible without embedding an compiler. Vim's syntax capabilities alone are based on matching; they don't actually compile the code.
The benefit provided by color_coded is that the highlighting is based on how your code looks to the compiler, not to a (very clever) regex machine. Given that, the fact that I chose libclang doesn't change the functionality of the project.
Syntax highlighting is a complex problem, because it needs to be fast enough, and the more token types you have, the longer it takes.
Each time you type a character, you have to parse the code (clang is used here), select the according color, and draw the text (often the OS will deal with this, and I don't know if you can tell an OS to just update one line of drawn code).
This is a vim plugin, and it's way easier to colorize text in a console environment than in a cross platform GUI window (not to mention UI text editors use a dedicated data structure, the rope: http://en.wikipedia.org/wiki/Rope_%28data_structure%29 to make text editing responsive when using the mouse).
My guess is that sublime text completely shortcuts this problem by using webkit to draw its own text surface.
Anyway, it's another example of how competing windowing systems are inflexible and make things harder on the programmers, which often overcome the problem by avoiding the OS. Cross platform software programming is more than a philosophy or an art, it's a war against OS manufacturers. It's won by finding the easiest solution to make it work on all platforms.
In this case, you can clearly understand how making an editor with syntax highlighting requires many different things, so editing the colors and adding a language will greatly improve complexity.
My take is that one day someone will have the decency to just release an OS that just offer developers to use the hardware directly (or something like cairo) to draw even a simple window, without anything like Xorg, android, wayland, gnome, KDE, etc. The cost of coding something to display the results of your software is way too high when you want to do it the "mainstream way". I'm a little angry at Torvalds, not having tried to introduce something to build simpler GUI stuff.
So, in short, don't expect it to work on sublime text if the dev don't get involved. I guess it's possible since sublime text support scripting, but I wonder if it's possible to do it with decent performance. Even Sublime text start to hang a little when you have a large html file.
I was once very frustrated with text editors (always some missing feature, and I never accepted to switch to vim or emacs), so I tried to make my own, and I failed miserably and settled for what is already there. I still wonder if I can be more productive with vim, but if it becomes necessary to use it because a 11" laptop can't run an IDE, maybe I'll do the switch.
Some points you mention seem to have some sense, but:
* rope - do you actually know of any editors using it? I'd love to hear about them, especially if open-source, so that I could try exploring the sources, and maybe contribute/fork. I know MS Word does (did?), and ABIWord does (albeit with perf issues). Especially among the "programmer editors", AFAIK, most settle on "line gap" model, or other similar simpler ones;
* easier to colorize text in a console env than a GUI??? wtf? I suppose it's rather opposite, if different at all; in GUI, if you can render text, you can usually render it in any color you like;
* sublimetext using webkit??? wtf again? is your guess in any way educated, as for me it sounds like a totally random shoot in the dark; from quick googling, https://www.sublimetext.com/forum/viewtopic.php?f=2&t=7416 doesn't seem so, at least for ST2.
> Syntax highlighting is a complex problem, because it needs to be fast enough, and the more token types you have, the longer it takes.
Yes, it scales with the number of tokens. Why wouldn't it? This isn't a problem. It's 2015, we have ~4GHz umptheenth core computers, we can do this in real-time.
> Each time you type a character, you have to parse the code (clang is used here), select the according color, and draw the text
You seem to have no idea what is actually happening here. You don't need to rerun the whole parsing step at each character if your tool understands the language, only when you hit a token that changes the parser's state (so when you enter a paren, you're no longer typing an identifier, so now we need to reparse.) There are a huge number of ways to optimize this step from there, but I don't think anyone is actually employing any of them yet, either because they're too complicated or computers are just so fast we don't care (like maintaining a document AST and simply updating the nodes as they change around so you can get away with vastly simpler "reparses").
The actual layout and rendering of text is then done - each OS has their own text layout engine, and there's several open source ones (e.g. Mozilla and Blink/Webkit have their own text layout kits, Pango does it on GNOME, etc). Color highlighting is a part of text rendering, which happens next, which also every platform has it's own solution for (like, e.g. Cairo, an image compositing library, which uses FreeType, the Windows or Mac font rendering systems for rendering glyphs).
What should be colorized/bolded/treated as different is usually tracked by a set of span data structures (e.g. line number X, character Y to character Z should have this set of attributes applied).
> My guess is that sublime text completely shortcuts this problem by using webkit to draw its own text surface.
Sublime Text implements its own editor widget. Here is the author noting some bits about it: https://news.ycombinator.com/item?id=2822114 . Shipping Webkit is not an easy task, nor is it a small one in disk footprint. I'm not sure how you randomly came to the conclusion Webkit had to be involved.
The rest of your post seems as if it was generated by some kind of buzzword Markov Chain, so I'm ignoring it.
SublimeClang [0] does syntax highlighting and even code completion in Sublime Text. I used it for Objective-C sometimes when I'd get annoyed with Xcode. The author has since started another project that will hopefully avoid some of its problems, but last time I checked, it still worked OK.
With the annoying number of systems that don't have boost installed, such projects are forced to accommodate (by shipping with their own). After the accommodations are made, there's little reason to allow for the usage of the system's boost.
Similarly, color_coded downloads its own clang. Why?
1) People may not have clang installed (like boost)
2) (more importantly) color_coded is tested and proven to work with a specific version; it's more portable this way
If we suddenly allow users to use system-level boost and clang and foobar implementations instead of that which we ship, we're also opening a can of worms for incompatibilities. The amount of coverage that needs to be tested grows surprisingly quickly.
37 comments
[ 2.5 ms ] story [ 92.1 ms ] threadOn vim's side, YouCompleteMe explicitly denied my approach to integrate highlighting.
I believe there's no reason to have two plugins each running an instance of clang.
* i.e. showing parts of code that are ifdef'ed out in grey
YCM's config file can contain arbitrary python functions and doesn't really need to follow any standards. This makes it hard on color_coded and ultimately didn't seem worth the trouble.
http://truongtx.me/2013/03/10/emacs-setting-up-perfect-envir...
https://github.com/brianjcj/auto-complete-clang
That being said, if the core infrastructure is there, the hard part (Clang backend) is there, how much more can semantic highlighting be?
(I duck because I write amateurish code and do not do C/C++ yet, but would hopefully learn to do something awesome like that one day.)
https://github.com/jeaye/color_coded/blob/master/include/asy...
The second picture is from my C++14 JSON library:
https://github.com/jeaye/jeayeson/blob/master/include/jeayes...
http://imgur.com/Rddi9QH
compare with color_coded: https://camo.githubusercontent.com/5cf719c6357f27b86c45bf9b5...
So it looks like color_coded can highlight at least a few things that default emacs doesn't: function calls, variables (when they are used, both can highlight declarations) and non-standard type template parameters.
Isn't the name a bit vague though? "color_coded" could refer to anything, surely something that references clang directly would be more appropriate?
It can be pointed out that YouCompleteMe, for example, also doesn't have vim in the name. I guess it's not so bad!
I don't think clang is an implementation detail. Surely the custom syntax highlighting engine is a vim feature. It's the "using clang" that makes this interesting.
The benefit provided by color_coded is that the highlighting is based on how your code looks to the compiler, not to a (very clever) regex machine. Given that, the fact that I chose libclang doesn't change the functionality of the project.
Each time you type a character, you have to parse the code (clang is used here), select the according color, and draw the text (often the OS will deal with this, and I don't know if you can tell an OS to just update one line of drawn code).
This is a vim plugin, and it's way easier to colorize text in a console environment than in a cross platform GUI window (not to mention UI text editors use a dedicated data structure, the rope: http://en.wikipedia.org/wiki/Rope_%28data_structure%29 to make text editing responsive when using the mouse).
My guess is that sublime text completely shortcuts this problem by using webkit to draw its own text surface.
Anyway, it's another example of how competing windowing systems are inflexible and make things harder on the programmers, which often overcome the problem by avoiding the OS. Cross platform software programming is more than a philosophy or an art, it's a war against OS manufacturers. It's won by finding the easiest solution to make it work on all platforms.
In this case, you can clearly understand how making an editor with syntax highlighting requires many different things, so editing the colors and adding a language will greatly improve complexity.
My take is that one day someone will have the decency to just release an OS that just offer developers to use the hardware directly (or something like cairo) to draw even a simple window, without anything like Xorg, android, wayland, gnome, KDE, etc. The cost of coding something to display the results of your software is way too high when you want to do it the "mainstream way". I'm a little angry at Torvalds, not having tried to introduce something to build simpler GUI stuff.
So, in short, don't expect it to work on sublime text if the dev don't get involved. I guess it's possible since sublime text support scripting, but I wonder if it's possible to do it with decent performance. Even Sublime text start to hang a little when you have a large html file.
I was once very frustrated with text editors (always some missing feature, and I never accepted to switch to vim or emacs), so I tried to make my own, and I failed miserably and settled for what is already there. I still wonder if I can be more productive with vim, but if it becomes necessary to use it because a 11" laptop can't run an IDE, maybe I'll do the switch.
Some points you mention seem to have some sense, but:
* rope - do you actually know of any editors using it? I'd love to hear about them, especially if open-source, so that I could try exploring the sources, and maybe contribute/fork. I know MS Word does (did?), and ABIWord does (albeit with perf issues). Especially among the "programmer editors", AFAIK, most settle on "line gap" model, or other similar simpler ones;
* easier to colorize text in a console env than a GUI??? wtf? I suppose it's rather opposite, if different at all; in GUI, if you can render text, you can usually render it in any color you like;
* sublimetext using webkit??? wtf again? is your guess in any way educated, as for me it sounds like a totally random shoot in the dark; from quick googling, https://www.sublimetext.com/forum/viewtopic.php?f=2&t=7416 doesn't seem so, at least for ST2.
Yes, it scales with the number of tokens. Why wouldn't it? This isn't a problem. It's 2015, we have ~4GHz umptheenth core computers, we can do this in real-time.
> Each time you type a character, you have to parse the code (clang is used here), select the according color, and draw the text
You seem to have no idea what is actually happening here. You don't need to rerun the whole parsing step at each character if your tool understands the language, only when you hit a token that changes the parser's state (so when you enter a paren, you're no longer typing an identifier, so now we need to reparse.) There are a huge number of ways to optimize this step from there, but I don't think anyone is actually employing any of them yet, either because they're too complicated or computers are just so fast we don't care (like maintaining a document AST and simply updating the nodes as they change around so you can get away with vastly simpler "reparses").
The actual layout and rendering of text is then done - each OS has their own text layout engine, and there's several open source ones (e.g. Mozilla and Blink/Webkit have their own text layout kits, Pango does it on GNOME, etc). Color highlighting is a part of text rendering, which happens next, which also every platform has it's own solution for (like, e.g. Cairo, an image compositing library, which uses FreeType, the Windows or Mac font rendering systems for rendering glyphs).
What should be colorized/bolded/treated as different is usually tracked by a set of span data structures (e.g. line number X, character Y to character Z should have this set of attributes applied).
> My guess is that sublime text completely shortcuts this problem by using webkit to draw its own text surface.
Sublime Text implements its own editor widget. Here is the author noting some bits about it: https://news.ycombinator.com/item?id=2822114 . Shipping Webkit is not an easy task, nor is it a small one in disk footprint. I'm not sure how you randomly came to the conclusion Webkit had to be involved.
The rest of your post seems as if it was generated by some kind of buzzword Markov Chain, so I'm ignoring it.
[0] https://github.com/quarnster/SublimeClang
Similarly, color_coded downloads its own clang. Why?
1) People may not have clang installed (like boost) 2) (more importantly) color_coded is tested and proven to work with a specific version; it's more portable this way
If we suddenly allow users to use system-level boost and clang and foobar implementations instead of that which we ship, we're also opening a can of worms for incompatibilities. The amount of coverage that needs to be tested grows surprisingly quickly.
Neat.
-x objective-c
Nothing for CCerror. `make run` with the .cpp sample works.