I think the most astonishing thing to me about the Rust language project is not the language it self (sure it's innovative but there's also still a lot of work to be done to make it get out of your way). It's the speed and tenacity with which the project has managed to push development of the supporting tooling. The development tools surrounding a language can often be more important for productivity than the language itself, and it feels like the Rust team and community really get that.
I think that's a side effect of making a welcoming community a priority. The Rust Language Server grew out of feedback from the Rust Language Survey, in which many people responded that they wanted good IDE support.
That isn't a glamorous task to work on, especially if you're already comfortable with the level of editor support you have, but it helps bring more people into the community, which has a multiplicative effect.
I think it's a side effect of having a punishing language. Coding Rust is always described as fighting with the compiler, I don't think Rust would have ever taken off if there weren't great tools to fight the compiler with. I think that's a part of it, anyway.
I don't think "fighting the compiler" is an accurate description of the Rust coding experience. There is a learning curve during which you feel like you fight the borrow checker a lot, but once you get it you will find that the compiler is actually helpful.
Hmm fair enough, harder than I thought (I don't know Rust, the error just sounded sensible). It reads like the function you're passing into filter is supposed to be "lifetime-generic" taking a parameter whose lifetime comes from the for, but the function returned by apply has a fixed lifetime. But I can't see where it's getting the lifetime for that from, or what to do about it. I guess at that point I'd try adding an explicit lifetime to the return type of apply (or maybe an IDE could show it without having to write it)?
The problem is with the function `second` that I get from the other crate. If I replace the `second` function with my own implementation it works. But for the `second` implementation from tool.rs to work, Rust needs better higher ranked lifetime support.
It would also work with a closure, but I wrote my code this way to avoid closures in the first place (I was going for point-free style).
I had to change the `apply` function to dereference as well, that way it doesn't run into the lifetime issue. It has something to do with higher ranked lifetimes (the inability to express the higher ranked lifetime)
It sounds like you're returning closures, which is a very recent feature—it only became possible last month with the implementation of "impl Trait" IIRC. It takes time for features to mature.
> I think it's a side effect of having a punishing language.
Rust is a punishing language in the same way a notifying spell checker is a punishing tool. For someone used to writing without stopping and reviewing at the end, spell checking as you go might at first be annoying. After a while, you integrate it into your style, and choose the best time to deal with the errors when they come up yourself, whether that be immediately, after the paragraph/idea is expressed, or at the very end. In any case, having those handy notifications of what you need to fix is useful for a good 90% or more of the problems you might encounter.
Its not punishing, it's helping. Fixing bugs in development sure beats fixing bugs in production. And most of the things it complains about are in fact bugs.
I agree. At least for me it was a common pattern that by fixing a bug I've introduced another (sometimes suprising one), and by fixing that I've introduced another and so on. I would have given it up without strong editor support giving hints on the fly.
(I don't remember similar experiences learning other languages even with strong static checks (e.g. haskell). Either my errors were 1 level deep there, or the propagation of errors (like changing a type somewhere) was easier to follow/predict to me)
Not a Rust developer, but this project sounds sexy to me! :)
I use Emacs as much as possible, but for Java dev use IntelliJ because the Java plugins I've tried were always relatively clunky. Having great language support regardless of editor sounds wonderful!
Yeah, that seems nearly a constant for them. For example, in ReSharper (and presumably Project Rider), despite C# having the excellent, very fast (and, critically, fully open source!) compiler-as-a-service, Roslyn, they continue to use their own internal compiler service for code autocompletion, refactoring, and correctness analysis.
Resharper has been in development far longer than Roslyn and seems to be much more suited to working with partially incorrect code (the kind you have while typing away) and incremental changes.
I guess JetBrains is heavily invested in ReSharper (and the underlying AST code) which would explain why they won't want to switch right away, if at all.
I think he's saying he'd rather use Emacs for Java dev, but IntelliJ is a necessary evil. For Rust, there's https://github.com/racer-rust/emacs-racer which seems pretty solid.
P.S. Regarding Java dev on Emacs, ENSIME (http://ensime.github.io) now works with java-mode rather than only Scala. I use it for Android development, and it's not perfect but works well enough.
I can't upvote this enough. I found the Rust community incredibly welcoming, and loved the focus on tooling - making our day-to-day lives so much better. So much so, that after a few weeks of working with Rust I contributed a few small fixes to the IntelliJ Rust plugin.
So, yeah - the community inspired me, and I hope the RLS inspires other people to contribute to Rust and the tooling around it as well!
I might be wrong but one possible explanation is that the language is so hard to learn that people would need tools to help in the beginning.
Within Mozilla, they can use servo as a selling point or turn to creators for help, but from the outside world, we've got so many languages available, while some might not be good when the software grow big, they are more beginner-friendly than Rust. Hence Rust needs all those tooling from the very start.
I actually agree, it's not that hard, yes one can definitely grasp that with some efforts. But my point really is: the time from start learning Rust till when you can deliver your project is definitely longer than many other languages out there. So Rust needs more tooling than many other languages out there, which is not a bad thing considering all the merits in Rust. That's why we are seeing tooling growing very rapidly in this space.
I think it's more because we like making development easier and having tools to do that is great because it means I can spend more time with the compiler telling me I coded something wrong.
I agree that it depends on where you approach it from (my very first question for tutoring any new Rust user is "what languages have you used previously?"), but I'm afraid that saying "it's really not that hard" glosses over the experience of a large number of potential users. I don't consider Rust to be a particularly complex language--I put it on par with Python--but unlike Python the learning curve for Rust is remarkably steep, which is enough for me to classify Rust as a hard language to learn.
A foreknowledge of pointers helps, but for anyone for whom Rust is their first systems language, it's going to be Rust's task to teach them e.g. the stack/heap distinction, or what "aliasing" means, or why there are two string types (let's hope they're already versed in Unicode...), or how to design a system in the absence of pure OO.
> I don't consider Rust to be a particularly complex language
> I put it on par with Python
Python doesn't have complicated lifetime errors, confusing trait not being satisfied errors. Python function declarations don't need three lines of code.
I only have very little Rust experience, and would like to hear from a Rust veteran - as much as one can be - if that is considered good Rust, or if it's just an example of how bad Rust can be if you don't know what you're doing, or are trying to showcase bad Rust.
I would think it's the latter, but I don't know, to be honest.
You could write <'a, T: Monoid + From<&'a str>> and omit the where clause, but that seems even less readable to me. The rest basically has to be there for the function to do what it does.
It's basically an example of what happens when you try to use Haskell concepts in Rust; the type parameters explode quickly. Most rust functions are not like that. Some are. It's not exactly bad rust to have functions like that, but such functions should probably be uncommon in your codebase.
Note that Monoid here is ONLY for the reason that Cow<str> doesn't satisfy Add. In fact, String doesn't satisfy it either. So I need to have a trait that allows me to call a function that concatenates my types. I already wrote how to concatenate Strings and Cow<str>s in my Monoid crate so I'm just re-using that.
It could be T: Concatenable for all I care, but the rest of the function needs to look this way, more or less.
I only have one type parameter, that's very common. I'm abstracting over String and Cow<str> because I want to bench the performance of both. (Cow<str> is always faster)
I'm not just talking about Monoid, I'm talking about the overall signature -- with the Fn and the abstractions it's very Haskelly. Such functions are not common in Rust. You find them (all the iterator adaptors, for one), and they can be "good Rust", but they're not common. Which is okay.
(I don't disagree that Rust is more complicated than Python, just addressing GP's comment whether or not this is "good" rust.)
It's actually very common to pass around hashmaps of functions in dynamic languages. If anything, since you don't even have to know how they're implemented it's easier. I realize that my function incurs a run time cost because I'm making an array of trait objects. When I'm writing JavaScript I don't know and I don't care.
Its non-trivial and it could be written simpler (the tuples part could get its own type alias, making it more readable and shorter), but its not particularly bad or good.
You will find examples of this mostly in libraries, but rarely its something a beginner would write. You can actually do a lot without ever using lifetimes. Eventually you'll need to learn to read this though, and its just a matter of getting used to it. I'd say none of those is any harder to understand then let say nested list comprehension in python.
I don't agree with kibwen about Rust being just as simple as Python, but I also think it's a bit of a pointless comparison, given that Python is dynamically typed and Rust is statically typed. Any type system feature is something that Python trivially doesn't have.
I consider "complexity" to refer to both the quantity of language features and the number of significant or surprising interactions that exist between every language feature. Rust and Python are both medium-complexity languages, but the difference is that in Python you can totally ignore e.g. generators or decorators or (oh god) metaclasses and still get work done. In Rust you really can't get work done until you understand references and move semantics. The total height of each learning curve is the same, but Rust's is more like a cliff than a gentle slope, which is why it's crucial that we put so much focus on documentation.
I ran into higher rank lifetimes while trying to use a library in my program. You can totally pass closures around in a dynamic language like JavaScript without any issues (you just pay for it in performance). In Rust `-> impl Trait` is a new feature so it's still not at all polished.
I would argue that Rust has a higher learning curve overall than any dynamic language because of its type system.
We've actually heard from production users over the last year or two that the compiler itself helps you learn the language; it's been compared to a pair at times. And that was long before new errors...
I definitely agree! While I was reading about Rust couple of weeks ago I was thinking maybe exposing the compiler's detail would help refactoring and diagnosing bugs here. It's so awesome that you guys have already think of this! Really awesome work!
Again, just want to mention that my point is simply that Rust needs more tooling to overcome its learning hurdles, and to me this is not a bad thing. Because once we have tools solving the learning curve problem, we can enjoy all the benefits Rust provides :)
I don't know. I love haskell, but it fits the bill of a language so hard to learn that people need tools to help in the beginning. Yet haskell tooling isn't that great. I'd guess that it's a question of priorities.
I suspect haskell might be in a difficulty class beyond rust, where really good tooling still wouldn't be enough to make it significantly more accessible. The things rust requires a C++ programmer to learn also seem to be relatively amenable to mechanical assistance.
If haskell is that much harder, it should benefit from better tooling that much more. I think there really is a big difference in the priorities. I've never played around with GHC, so I have no idea how hard tooling is, but I know some good old humans are at least able to contribute (https://izbicki.me/blog/error-messages-in-ghc-vs-g%2B%2B.htm...).
Writing Rust code is more difficult than writing Haskell code. Haskell is more than okay with taking a performance hit for convenience. Rust APIs actually have you pass in references to buffers where you want your data to end up. Rust doesn't use GC and tries to focus on pay-as-you-use abstractions. That's a lot more control, but also more verbosity.
>It's the speed and tenacity with which the project has managed to push development of the supporting tooling.
Compared to what ? Community designed languages like D, Nim, w/e - no dispute here. But corporately backed languages such as C#, F#, Swift, TypeScript, Kotlin - not really. I guess it's fair since Rust is middle ground - sponsored by Mozilla but not getting the level of investment as the others mentioned.
I think that's the point. Lots of community languages leave the tooling to the community, which only works if your language gets adopted (and doesn't help it to get adopted).
Rust is a community language (admittedly with the organisational backing of Mozilla), but is driving the whole ecosystem.
For anyone interested in similar tools, Microsoft recently published a protocol[1] for this type of service, and an implementation[2] of it for VSCode+NodeJS.
Swift also has a library[3] called SourceKit which does the same thing for Swift – it can run as an out-of-process daemon, and people have written integrations for Atom, Emacs, etc. to get autocomplete, syntax highlighting, def links, etc.
Thanks, I totally missed that. That's pretty great – I thought it was a neat idea from MS, but I was worried it would get any traction. Glad to have been wrong!
I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead.
It's great that Rust will decode and encode the JSON quickly (I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project), but the project I work on cares about optimizations on the level of in-memory data layout and memory allocations. Moving syntax highlighting into a pluggable system would be a huge performance regression.
There was a talk at RustConf by Raph Levien about his editor, Xi. It almost entirely uses JSON-RPC to communicate through a distributed architecture, and next-level speed is one of his design goals: https://confreaks.tv/videos/rustconf2016-a-modern-editor-bui...
> I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project
Ah! What things are you looking for in its maturity? In some ways, it's actually the most mature backend; stack probes work with it, for example. Windows isn't my personal area of expertise though.
I've been following https://github.com/rust-lang/rfcs/issues/1061 for a while. It seems that high-level pieces are there, but that certain elements useful in day-to-day software engineering still need some more time.
If you have the time, you should really consider helping to improve the Rust experience on Windows. There's a huge shortage of Windows developers in the Rust community.
Unfortunately I am already overcommitted with open source projects. For now I am content to wait and see how it all plays out. It will be great to have another language where Windows support is really first-rate.
Probably the biggest drawback of the MSVC backend is that its debugging symbols only have line numbers, no variable/type information. The rest is mostly Windows-specific issues like manifests and subsystems.
Actually, the PDB debug info has variable and type information and a lot of it actually works. There's just a few minor issues here and there with stuff like fat pointers.
It might have been due to either https://github.com/rust-lang/rust/pull/31319 or an LLVM upgrade. But stuff like looking at the values of basic integer types, the fields of structs, tuple fields, thin pointers, and even many enums works fine. You just have to make sure you have debug info enabled.
Keep in mind std in the Rust distribution was built without debuginfo, so if you step into functions from that, things don't work too well.
MSVC is commonly used to refer to Visual Studio, more specifically the C++ compiler that's part of it (originally "Microsoft Visual C++"), which would explain the parent confusing the two.
Perhaps I'm not understanding you, but how does the speed of syntax highlighting (or, more generally, the editor) affect your project's performance? I'm trying and failing to imagine some sort of realtime system controlled by someone coding away.
(Also, you wouldn't necessarily need to transmit the whole editor state--just changes).
> I'm imaging more along the lines of serializing editor state with each key press, encoding it, sending it to another process, having that process unserialize it, lex it, encode all of the cascading token changes and information about those tokens back into JSON, sending it back to the editor process which then has to decode the JSON and apply all of the token information to the in-memory representation as real overhead.
The latency of both Unix IPC calls and serialization/deserialization of small amounts of JSON should be measured in single- to double-digit microseconds. A frame at 60fps is 16,000 microseconds, so if this is causing even a single dropped frame, something is seriously wrong.
The lexing itself might or might not be a performance worry, depending on the architecture of the whole system, but that's no different from the lexing built into the editor (except perhaps in that people writing editors tend to care more about latency)...
(Well, this seems academic, since according to other comments RLS doesn't aim to put syntax highlighting out of process.)
Syntax highlighting is (usually) done by the editor, not the server in the LS protocol. The changes sent from editor to server are pretty minimal, and so the whole thing is pretty quick. The overhead, never caused us any problems.
I was referring to the paragraph about Swift and Source kit. To my knowledge, the majority of editors do not do syntax highlighting outside of the main editor process.
I've wanted this for a long time. But I don't see why it has to be language-specific. Maybe some of the fancy parts do, but why can't make/cmake/ninja/etc. support this in a language independent way? Just basic queries/commands like:
1. run any build steps that directly depend on file X
2. run any build steps that directly or indirectly depend on file X
3. get whether the build of X succeeded or failed, and console output
These alone would go so far towards making Vim/Emacs/etc more user-friendly.
We've been planning this for a while, and it's largely been a joint effort between Jonathan and Nick; Jonathan _did_ work on TypeScript before coming to work on Rust, though.
Compilers that have a language service api are awesome. The Typescript compiler does and it makes for high quality low latency refactoring, code completion, show expression type etc across ediors and IDEs.
Since my other day to day language is Scala I was glad to find out that the upcoming Dotty Scala compiler will also feature a language service api. Yay.
107 comments
[ 6.7 ms ] story [ 181 ms ] threadI look forward to seeing support for this integrated with vim.
That isn't a glamorous task to work on, especially if you're already comfortable with the level of editor support you have, but it helps bring more people into the community, which has a multiplicative effect.
Refactoring tools would help address compiler concerns quicker, however.
filter is from the standard lib
apply is defined as
second comes fromhttps://github.com/Stebalien/tool-rs/blob/master/src/sequenc... which means you have to track down its implementation somewhere in macros
https://bitbucket.org/iopq/fizzbuzz-in-rust/src/920bd2bd1267...
this doesn't work, so I want you to tell me why
It would also work with a closure, but I wrote my code this way to avoid closures in the first place (I was going for point-free style).
I basically just have to write my own function:
https://bitbucket.org/iopq/fizzbuzz-in-rust/src/58bfa4c58c80...
this compiles, but it seems kind of pointless since I can only use this particular function in my case (because of the extra levels of references)
if I could use the tool.rs version, it would automatically generate all the implementations I need using macros
I had to change the `apply` function to dereference as well, that way it doesn't run into the lifetime issue. It has something to do with higher ranked lifetimes (the inability to express the higher ranked lifetime)
This is the working version:
https://bitbucket.org/iopq/fizzbuzz-in-rust/src/95451868e27f...
It sounds like you're returning closures, which is a very recent feature—it only became possible last month with the implementation of "impl Trait" IIRC. It takes time for features to mature.
Rust is a punishing language in the same way a notifying spell checker is a punishing tool. For someone used to writing without stopping and reviewing at the end, spell checking as you go might at first be annoying. After a while, you integrate it into your style, and choose the best time to deal with the errors when they come up yourself, whether that be immediately, after the paragraph/idea is expressed, or at the very end. In any case, having those handy notifications of what you need to fix is useful for a good 90% or more of the problems you might encounter.
(I don't remember similar experiences learning other languages even with strong static checks (e.g. haskell). Either my errors were 1 level deep there, or the propagation of errors (like changing a type somewhere) was easier to follow/predict to me)
I use Emacs as much as possible, but for Java dev use IntelliJ because the Java plugins I've tried were always relatively clunky. Having great language support regardless of editor sounds wonderful!
One really interesting thing about this effort is that the IntelliJ folks have their own stuff for autocomplete; they're not using racer.
Does Roslyn have the same support?
I guess JetBrains is heavily invested in ReSharper (and the underlying AST code) which would explain why they won't want to switch right away, if at all.
P.S. Regarding Java dev on Emacs, ENSIME (http://ensime.github.io) now works with java-mode rather than only Scala. I use it for Android development, and it's not perfect but works well enough.
So, yeah - the community inspired me, and I hope the RLS inspires other people to contribute to Rust and the tooling around it as well!
Within Mozilla, they can use servo as a selling point or turn to creators for help, but from the outside world, we've got so many languages available, while some might not be good when the software grow big, they are more beginner-friendly than Rust. Hence Rust needs all those tooling from the very start.
A foreknowledge of pointers helps, but for anyone for whom Rust is their first systems language, it's going to be Rust's task to teach them e.g. the stack/heap distinction, or what "aliasing" means, or why there are two string types (let's hope they're already versed in Unicode...), or how to design a system in the absence of pure OO.
C style pointer tricks are better left for level two, so to speak.
Probably a good approach teach Rust's OO way is to introduce traits as kind of extensible modules that can be composed and go from there.
It is not 100% correct, but I guess it is easier to teach than going full theory with traits being a kind of type classes and do on.
Python doesn't have complicated lifetime errors, confusing trait not being satisfied errors. Python function declarations don't need three lines of code.
I mean wtf is this?
there's about ten language features in these lines of code that Python doesn't haveI would think it's the latter, but I don't know, to be honest.
It could be T: Concatenable for all I care, but the rest of the function needs to look this way, more or less.
I only have one type parameter, that's very common. I'm abstracting over String and Cow<str> because I want to bench the performance of both. (Cow<str> is always faster)
(I don't disagree that Rust is more complicated than Python, just addressing GP's comment whether or not this is "good" rust.)
You will find examples of this mostly in libraries, but rarely its something a beginner would write. You can actually do a lot without ever using lifetimes. Eventually you'll need to learn to read this though, and its just a matter of getting used to it. I'd say none of those is any harder to understand then let say nested list comprehension in python.
I would argue that Rust has a higher learning curve overall than any dynamic language because of its type system.
That said, there are more tools to be built to make it even better. https://internals.rust-lang.org/t/borrow-visualizer-for-the-... is one extension I'm very interested in seeing.
Again, just want to mention that my point is simply that Rust needs more tooling to overcome its learning hurdles, and to me this is not a bad thing. Because once we have tools solving the learning curve problem, we can enjoy all the benefits Rust provides :)
It would help a lot in regards to language productivity.
edit: It looks like the error message changes may have been reverted (https://phabricator.haskell.org/rGHC92c924e07dfef327509555c2...) :( Sad that they reverted it instead of fixed it.
Compared to what ? Community designed languages like D, Nim, w/e - no dispute here. But corporately backed languages such as C#, F#, Swift, TypeScript, Kotlin - not really. I guess it's fair since Rust is middle ground - sponsored by Mozilla but not getting the level of investment as the others mentioned.
Rust is a community language (admittedly with the organisational backing of Mozilla), but is driving the whole ecosystem.
Swift also has a library[3] called SourceKit which does the same thing for Swift – it can run as an out-of-process daemon, and people have written integrations for Atom, Emacs, etc. to get autocomplete, syntax highlighting, def links, etc.
[1]https://github.com/Microsoft/language-server-protocol [2]https://github.com/Microsoft/vscode-languageserver-node [3]https://github.com/apple/swift/tree/master/tools/SourceKit
The Rust development experience in VS: Code is the only reason why I use it.
Kudos to everyone.
It's being built my MS and they are quick to respond to feedback and pull requests.
It's great that Rust will decode and encode the JSON quickly (I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project), but the project I work on cares about optimizations on the level of in-memory data layout and memory allocations. Moving syntax highlighting into a pluggable system would be a huge performance regression.
> I've been keeping an eye on Rust's MSVC progress before I consider using it for a real-world project
The latest builds of MSVC claim to have native Rust syntax highlighting and autocomplete, I have not yet tried it myself: https://www.visualstudio.com/news/releasenotes/vs15-relnotes
If you have the time, you should really consider helping to improve the Rust experience on Windows. There's a huge shortage of Windows developers in the Rust community.
Keep in mind std in the Rust distribution was built without debuginfo, so if you step into functions from that, things don't work too well.
(Also, you wouldn't necessarily need to transmit the whole editor state--just changes).
"My project depends on my editor being super fast."
"What are you writing that has that requirement?"
"I'm writing my editor."
The latency of both Unix IPC calls and serialization/deserialization of small amounts of JSON should be measured in single- to double-digit microseconds. A frame at 60fps is 16,000 microseconds, so if this is causing even a single dropped frame, something is seriously wrong.
The lexing itself might or might not be a performance worry, depending on the architecture of the whole system, but that's no different from the lexing built into the editor (except perhaps in that people writing editors tend to care more about latency)...
(Well, this seems academic, since according to other comments RLS doesn't aim to put syntax highlighting out of process.)
Side note: really looking forward to rolling this into Rust support for Sourcegraph (Sourcegraph team member here) to make any Rust library as easy to explore as https://sourcegraph.com/github.com/gorilla/websocket/-/blob/... and https://sourcegraph.com/github.com/staltz/xstream@master/-/b... are for Go and TypeScript. Thanks for making and open-sourcing this!