How does it find which tests are relevant? How big can the solutions and test suites be before it starts to...not work?
I have a decent sized project (100 projects, 10k+ tests, different suites with longer running smoke tests that take hours and so on). How does it know which tests it can run, and how does it know which will cover a certain line?
I can only speak for NCrunch, but once it's done a full coverage run, it knows which tests are likely to be affected by any changes, so it runs those tests first.
You can automatically re-run affected UI tests after any code change with UI tests recorded in Microsoft Test Manager, since 2012 if I remember correctly. So the technology is already there, I guess it was relatively simple to apply it to unit tests.
Test Professional is an interesting product, with recording manual tests, turning them into code, running them as a suite etc.
Hi Alkonaut, i'm dev on the Roslyn team, and i work closely with the group working on LiveUnitTesting (LUT). By default LUT just runs all your tests in realtime, as you make edits. It smartly prioritized tests based on seeing what code was previously hit by tests.
Now, that said, such an approach would definitely not scale up to projects with huge amount of tests, or tests that take hours and hours to run. As such, LUT provides ways to create 'test sets'. i.e. sets of tests scoped down to just the work you're doing. The intent is to have flexible way to create these sets and the individual sets can be set up to run automatically or manually depending on your preference.
A nice things is that all of these test run outside of code VS. So the perf of the tests is far less impactful on VS than it would be otherwise. i.e. you don't have a single GC engine trying to manage all the memory for core VS as well as whatever garbage the tests are generating. With this, and with tests running at lower pri, your editing experience should not be impacted (and we take perf impact very seriously with this feature).
There are various competencies you need to demonstrate to become a partner, but there's enough diversity between them that most businesses with Microsoft tech somewhere in the stack could become a partner.
If you are an ISV producing code that runs using Microsoft tech on Windows, it is not particularly difficult at all to qualify as at least a Silver-level partner. It's kind of a PITA to run your applications through the verification process every couple of years, but it's not hard, more tedious; or you can get a couple people to pass some of the certification tests.
Live Updated UnitTest OK/Failed/Not-Covered line indicators.
Better Exception Messages. NullReferenceException displays what variable caused it. Inner Exceptions are somehow displayed on top level.
Configurable Code Style suggestions/enforcement.
Intellisense filters in dropdown including shortcuts (Only show methods, properties, etc.)
Ctrl-T. "Goto All". Navigates to file or symbol, also with filters.
Find all References is now a nested, colorized tree.
All in all a nice update, cleary inspired by Resharper. We'll see how Resharper will evolve after this. I still depend a lot on it, because it really offers alot of features which are sub par in Visual Studio, for example like superior refactoring options, better 'find usages', File Structure Window, UnitTest Sessions. Things I use and depend on each and every day.
I dumped R# a few years back. It couldn't cope well with larger C# projects that I have to deal with and regularly hoses VS. Plus I spend a lot of time on Linux platforms so I don't want to get further embedded in that ecosystem as a mindset really.
We all come from different backgrounds and may never have had the chance to brush up on the best-practices of the updated code. I just bought a book that's about 1,600 pages long on the latest ASP.Net framework. Interesting stuff there, but it can be daunting to learn.
I love how Visual Studio is moving towards suggesting best practices for their languages with these new features. Not only are you going to write more solid code, but you'll also can improve and update your skill-set in C#.
I think C# is a fantastic language, and the more I learn about it the more I want to code in it. Its beginning to be less of a "means to an end" and more of a very solid tool for productivity code. I can't wait to use this new version of VS.
Having coded in C# with Resharper for more than 7 years, I feel like I can't code in any other language anymore. I mean anything that doesn't have that level of intellisense / navigation / refactoring is just so painful to work with for me.
I especially cannot live without the high quality static analysis possible on .NET anymore. So much so that any dynamic language just feels "shaky". I mean, when you are not entirely sure you covered all method calls with that refactoring, it feels uneasy.
Upon reflection, my code editing workflow follows surprisingly often the results of the static analyzer: I remove a code element and let the ReSharper figure where I need to go to make the required changes, be that variables in one function - up to solution wide elements. Alt+PgDown - Alt+Enter is a very powerful combination.
Edit: my goodness, it just struck me that I have used C# for 13 years already.
Visual Studio seems to keep incorporating features from ReSharper in every new version, and ReSharper subsequently keeps adding new features as a response. The ReSharper developers have the fun task of combining these features into a coherent UI.
New features in VS2017 that were already in ReSharper include configurable suggestions from static analysis, Ctrl+T navigation (it even uses the same shortcut), structured find references, intellisense filtering, multiple refactorings in given scope, live unit testing and coverage (in ReSharper Ultimate).
This is great and a bit amusing at the same time, but as a user, I like the improvements we get.
Every time I try to use Resharper, it takes only a few days before I have to uninstall it. Last time, I couldn't find a way to turn off ctrl+click navigating to a definition.
Jetbrains does still have a very nice decompiler, memory and performance profiler to go along with ReSharper. And then of course all of their other IntelliJ-based IDEs for other languages. It still makes a lot of sense for me to pay for my license, since it's less than a dollar a workday.
I'm kind of curious what they'll do with their C# IDE, Rider, and how much they can share between that and ReSharper. With Visual Studio itself starting to go cross-platform, I'm not sure how long that niche will stay open.
One additional selling point of ReSharper to me has been the embedded continuous learning enabled by it. Almost every time a new language feature is introduced, it is ReSharper that shows me how and when to use them, using suggestions in my existing codebase. I especially like how they are able to translate several nested foreachs and ifs into one line LINQ statement - and back again. This ability was instrumental for me to really become fluent in LINQ. This level of code analysis and refactoring is something we just don't see yet in Visual Studio.
I see this pattern in more Microsoft products.
1. Publish a default implementation with plugin support
2. Popularize and sell product
3. Add a small subset of popular plugin features to next version. Ensure to never implement the full set.
4. Release improved version of product
5. Plugin will enhance with newer features
6. Repeat steps 3-5
If done right, this will work for both product and plugin implementors.
It's amusing to watch a co-worker develop in the latest version of VS while still having ReSharper installed. There are so many intelligent hover-over tips, sidebar icons, and such. I can't imaging how they can read the code with all that. Plus, their IDE is slow.
I've tried ReSharper on and off for 15 years and I was never able to justify its features for just how slow it made everything. But I'm a C# developer. I understand the default C++ experience in VS historically want as good as the default C# experience, ReSharper bright then to parity, and C++ devs generally found the experience to be a universal improvement, rather than trade offs.
I agree that the tooling is about as heavy as you can get when you combine VS and R# with solution wide static analysis turned on, but it's nothing an equally heavy development machine can't fix. Then again, I may be old school in this, as I prefer massive desktop machines and too many screens. What kind of machines you use?
It is also worth noting that the structure of the code affects the static analysis and other features. It is faster to operate with lots of smaller isolated modules than large and coupled ones. The same techniques that kept C++ project compile times manageable, keep the static analysis real-time in C#. One more reason to decouple modules.
I ended up uninstalling ReSharper 2 years ago because of the amount of false positives it reported as well as having huge impedance mismatches with the performance requirements I had at the time.
One annoying moment was when it claimed a variable would always be false, and running in the debugger proved it could have both values. I also had to turn off a lot of rules because the refactoring suggestions I had would've made the code cleaner at the cost of adding indirections or allocations.
I was doing a 2D skeletal animation system for Unity3D at the time and wanted raw performance much more than anything else. We ended up pushing 10K sprites on mobile devices (thats about 200 dynamic meshes of 50 sprites each - the best existing solutions we could find had a performance cap of 10-20 meshes) but that was the least idiomatic C# I've ever written; structs and refs everywhere in tightly packed contiguous memory with components being dumb indices into these arrays.
Its a wonderful tool, but it requires some critical thinking to not naively apply everything it suggests, which in my case ended up being easier to work without ReSharper than with it.
Funny coincidence, I'm tweaking performance of manually managed contiguous memory using structs and refs as I'm writing this reply (the application is a combinatorial optimization system). There's nothing in ReSharper that currently suggests me to replace structs and refs with something else, or that I need indirections into my SoAs and AoSs.
But yeah, static analysis can yield false positives if you use for example reflection (one more reason to avoid it in most cases), but there's an option to turn a specific message off for particular line, class, etc. You can also edit ReSharper suggestions per solution basis, so if there are some special requirements, they can be incorporated into its behavior.
It probably improved a lot since the last time I used it. I haven't done any C# in a while :)
It wasn't complaining about structs or refs but wanted LINQ everywhere, anonymous lambdas and other high level constructs in place of high-performing but harder to read ones.
I wasn't using reflection either, it was something like this simplified pseudocode (off the top of my head, I haven't touched that codebase in over a year):
var completed = false
for each line in timeline do
completed = completed || tick(line)
if completed // here it complained its always false
There was more to it (to handle time offsets and multipliers, disabled lines and whatnot) but thats the gist of the algorithm.
After disabling or tweaking rules every day I just decided to uninstall it, and as I said its a wonderful tool - just not for that specific task at that specific point in time :)
I still say Resharper taught me Linq back when it first came out.
It'd generate some abhorrent Linq statements in place of very clean, readable code, but just trying to grok what those statements meant made me so much more comfortable with it.
And of course I'd always go back and undo the changes, but occasionally it'd do something that was the right balance of nifty and readable that I'd try to salvage out of the larger statement and reuse elsewhere.
It's true it takes some effort to tweak it to cut down on noise from its analysis (and IntelliJ has the same issue), but I think it manages to be worth it over all (assuming you have that time and effort to contribute, which it sounds like you didn't at the time)
The pseudocode is intriguing. I personally have been stumped 2 or 3 times by static code analysis, disagreeing firmly with the machine for few minutes before realizing my mistake, but it is of course possible that there have been bugs in the analysis system of R#, which might explain what you saw.
Adjusting the rules and settings of R# is a daunting task, I agree. However, usually if I have special requirements for a project, I just set Inspection Severity to none for the language features I don't want to use, and that solves the issue for that project.
Hey Anchor, I'm https://github.com/CyrusNajmabadi a developer on the Roslyn team, and a primary developer around many of the new features Kasey demo'ed here.
You're definitely correct that many of these things have already appeared in Resharper. However, that wasn't the point of us building them. We invested in the Roslyn platform many years ago so that not only could we share and leverage code across all our languages, but so that other people could plug in as well to add the sorts of enhancements they would like.
Currently, We've moved C#, VB, TypeScript, JavaScript, and now Xaml, onto the Roslyn platform. Because of this (note that some of this is bleeding edge), not only can we all share a common infrastructure for things like Find-References, IntelliSense, and Refactoring, but 3rd parties can now plug in as well and add their own capabilities.
For example, all the refactorings we've offered are ones that others could have as well. We just happen to think that, over time, we should continually be improving the core set we provide so that you can get a highly capable and powerful experience out of the box without necessarily having to go install a lot of extra plugins. Of course, if you do find plugins (including resharper) that you find valuable, then we want you to def go out and enhance your experience with them.
I hope you get a chance to try this stuff out. As always, please file feedback/issues on github.com/dotnet/Roslyn
Your account appears to be dead. I vouched for your comments in the thread so they would show up. Not sure why they were dead, but you might want to contact the staff to get that cleared up.
Hi and thanks for the work on Roslyn, I really like what you guys are doing with it.
Yeah, I know, many of the features of ReSharper are something VS should have had for a decade, regardless whether ReSharper has them or not. We - VS users - tried to tell VS dev teams in around 2005 that JBuilder (of all things, if you can remember that - it got eventually Eclipsed) had better refactoring and navigation tools than Visual Studio. The message from VS has consistently been that you are going to fix that, and when I heard of Roslyn, I though that this time it may be really happening. Only it has been ten years. Luckily .NET/C# was marginally better from developer perspective than JVM/Java from the beginning (the difference has grown ever since), and ReSharper (and other plugins) helped to keep Visual Studio afloat until today.
Now that Microsoft has embraced open source, community contributions, shorter release cycles, .NET Core, etc., I think you have a great foundation for the next ten years of .NET.
Oh, rest assured, I will continue to bug you guys through different channels.
Note: the Go to navigation is available since VS 2010, you only need to set the shortcut.
Integration of style-cop-like rules seems ok, my only worry is that all those features will make VS sluggish.
And that braces refactoring seems a poor example. Either you are a member of your team, and you don't write code without them, or it's some old code, that shouldn't be automatically refactored if the refactoring is the only goal.
> Integration of style-cop-like rules seems ok, my only worry is that all those features will make VS sluggish
They really shouldn't. We've actually spent the majority of our time in VS2017 on perf. So even with all the extra functionality, it's significant faster and more responsive across the board.
One thing that helps out here is that we don't run any of this analysis in the core VS process. As such, we are not contending for any resources that VS itself needs. i.e. a large source of hiccups in VS was due to the GC having to run right when a UI request was coming in. By moving this analysis out of proc we dramatically lighten the memory load in VS and we make it so that the GC runs less often, and takes less time. This clears up the UI thread more and makes things much more responsive.
> Either you are a member of your team, and you don't write code without them, or it's some old code, that shouldn't be automatically refactored if the refactoring is the only goal.
Or, you're a team that has decided on new, more stringent, coding requirements. And now you want to find and fix up all the violations. Or you're absorbing code by a different group with different coding patterns. Or, you just missed something during a code review and you want to go fix it up :)
If teams have a style preference, then it is definitely valuable to have automated tooling checking to ensure the style is maintained. Otherwise you're shifting the burden onto the team members, which is precisely what tooling like this is supposed to alleviate.
> And that braces refactoring seems a poor example.
NOte: we've added many style options in VS2017, and we're going to continue adding more. Options include:
1. Do you want field/property/methods/events qualified with "this." or not
2. Do you prefer predefined types (like "string") or do you prefer the .net fx types (like "System.String")
3. How and when do you use "var".
4. Do you want to use => for simple properties/methods/etc.
5. Would you prefer to use the more concise typechecking constructs over the previous approaches for type-checking.
6. Do you want to use ?? or ?. when applicable
57 comments
[ 8.1 ms ] story [ 1451 ms ] threadhttp://comealive.io/Code-Connect-Joins-Microsoft/
I have a decent sized project (100 projects, 10k+ tests, different suites with longer running smoke tests that take hours and so on). How does it know which tests it can run, and how does it know which will cover a certain line?
Test Professional is an interesting product, with recording manual tests, turning them into code, running them as a suite etc.
https://www.visualstudio.com/vs/test-professional/
Now, that said, such an approach would definitely not scale up to projects with huge amount of tests, or tests that take hours and hours to run. As such, LUT provides ways to create 'test sets'. i.e. sets of tests scoped down to just the work you're doing. The intent is to have flexible way to create these sets and the individual sets can be set up to run automatically or manually depending on your preference.
A nice things is that all of these test run outside of code VS. So the perf of the tests is far less impactful on VS than it would be otherwise. i.e. you don't have a single GC engine trying to manage all the memory for core VS as well as whatever garbage the tests are generating. With this, and with tests running at lower pri, your editing experience should not be impacted (and we take perf impact very seriously with this feature).
Cheers!
NCrunch is far from the only continuous testing runner. I've been using the one built into ReSharper lately.
https://www.visualstudio.com/en-us/news/releasenotes/vs2017-...
https://partner.microsoft.com/en-AU/membership/msdn-subscrip...
The registration fee is something like $2K.
There are various competencies you need to demonstrate to become a partner, but there's enough diversity between them that most businesses with Microsoft tech somewhere in the stack could become a partner.
Live Unit Testing https://channel9.msdn.com/Events/Connect/2016/171
I'm a dev on the Roslyn team. Alive is still being worked on. :)
But making something production quality and adding it to VS takes a lot of time.
Better Exception Messages. NullReferenceException displays what variable caused it. Inner Exceptions are somehow displayed on top level.
Configurable Code Style suggestions/enforcement.
Intellisense filters in dropdown including shortcuts (Only show methods, properties, etc.)
Ctrl-T. "Goto All". Navigates to file or symbol, also with filters.
Find all References is now a nested, colorized tree.
All in all a nice update, cleary inspired by Resharper. We'll see how Resharper will evolve after this. I still depend a lot on it, because it really offers alot of features which are sub par in Visual Studio, for example like superior refactoring options, better 'find usages', File Structure Window, UnitTest Sessions. Things I use and depend on each and every day.
Edit: Here's an alternative link on youtube with the same video, if someone prefers that: https://www.youtube.com/watch?v=TJu0hzuz9hE
I dumped R# a few years back. It couldn't cope well with larger C# projects that I have to deal with and regularly hoses VS. Plus I spend a lot of time on Linux platforms so I don't want to get further embedded in that ecosystem as a mindset really.
Visual Studio Development – Productivity Enhancements in Visual Studio 2017 RC https://msdn.microsoft.com/magazine/mt79
We all come from different backgrounds and may never have had the chance to brush up on the best-practices of the updated code. I just bought a book that's about 1,600 pages long on the latest ASP.Net framework. Interesting stuff there, but it can be daunting to learn.
I love how Visual Studio is moving towards suggesting best practices for their languages with these new features. Not only are you going to write more solid code, but you'll also can improve and update your skill-set in C#.
I think C# is a fantastic language, and the more I learn about it the more I want to code in it. Its beginning to be less of a "means to an end" and more of a very solid tool for productivity code. I can't wait to use this new version of VS.
Upon reflection, my code editing workflow follows surprisingly often the results of the static analyzer: I remove a code element and let the ReSharper figure where I need to go to make the required changes, be that variables in one function - up to solution wide elements. Alt+PgDown - Alt+Enter is a very powerful combination.
Edit: my goodness, it just struck me that I have used C# for 13 years already.
New features in VS2017 that were already in ReSharper include configurable suggestions from static analysis, Ctrl+T navigation (it even uses the same shortcut), structured find references, intellisense filtering, multiple refactorings in given scope, live unit testing and coverage (in ReSharper Ultimate).
This is great and a bit amusing at the same time, but as a user, I like the improvements we get.
I'm kind of curious what they'll do with their C# IDE, Rider, and how much they can share between that and ReSharper. With Visual Studio itself starting to go cross-platform, I'm not sure how long that niche will stay open.
If done right, this will work for both product and plugin implementors.
It is also worth noting that the structure of the code affects the static analysis and other features. It is faster to operate with lots of smaller isolated modules than large and coupled ones. The same techniques that kept C++ project compile times manageable, keep the static analysis real-time in C#. One more reason to decouple modules.
One annoying moment was when it claimed a variable would always be false, and running in the debugger proved it could have both values. I also had to turn off a lot of rules because the refactoring suggestions I had would've made the code cleaner at the cost of adding indirections or allocations.
I was doing a 2D skeletal animation system for Unity3D at the time and wanted raw performance much more than anything else. We ended up pushing 10K sprites on mobile devices (thats about 200 dynamic meshes of 50 sprites each - the best existing solutions we could find had a performance cap of 10-20 meshes) but that was the least idiomatic C# I've ever written; structs and refs everywhere in tightly packed contiguous memory with components being dumb indices into these arrays.
Its a wonderful tool, but it requires some critical thinking to not naively apply everything it suggests, which in my case ended up being easier to work without ReSharper than with it.
But yeah, static analysis can yield false positives if you use for example reflection (one more reason to avoid it in most cases), but there's an option to turn a specific message off for particular line, class, etc. You can also edit ReSharper suggestions per solution basis, so if there are some special requirements, they can be incorporated into its behavior.
It wasn't complaining about structs or refs but wanted LINQ everywhere, anonymous lambdas and other high level constructs in place of high-performing but harder to read ones.
I wasn't using reflection either, it was something like this simplified pseudocode (off the top of my head, I haven't touched that codebase in over a year):
var completed = false
for each line in timeline do completed = completed || tick(line)
if completed // here it complained its always false
There was more to it (to handle time offsets and multipliers, disabled lines and whatnot) but thats the gist of the algorithm.
After disabling or tweaking rules every day I just decided to uninstall it, and as I said its a wonderful tool - just not for that specific task at that specific point in time :)
It'd generate some abhorrent Linq statements in place of very clean, readable code, but just trying to grok what those statements meant made me so much more comfortable with it.
And of course I'd always go back and undo the changes, but occasionally it'd do something that was the right balance of nifty and readable that I'd try to salvage out of the larger statement and reuse elsewhere.
It's true it takes some effort to tweak it to cut down on noise from its analysis (and IntelliJ has the same issue), but I think it manages to be worth it over all (assuming you have that time and effort to contribute, which it sounds like you didn't at the time)
Adjusting the rules and settings of R# is a daunting task, I agree. However, usually if I have special requirements for a project, I just set Inspection Severity to none for the language features I don't want to use, and that solves the issue for that project.
You're definitely correct that many of these things have already appeared in Resharper. However, that wasn't the point of us building them. We invested in the Roslyn platform many years ago so that not only could we share and leverage code across all our languages, but so that other people could plug in as well to add the sorts of enhancements they would like.
Currently, We've moved C#, VB, TypeScript, JavaScript, and now Xaml, onto the Roslyn platform. Because of this (note that some of this is bleeding edge), not only can we all share a common infrastructure for things like Find-References, IntelliSense, and Refactoring, but 3rd parties can now plug in as well and add their own capabilities.
For example, all the refactorings we've offered are ones that others could have as well. We just happen to think that, over time, we should continually be improving the core set we provide so that you can get a highly capable and powerful experience out of the box without necessarily having to go install a lot of extra plugins. Of course, if you do find plugins (including resharper) that you find valuable, then we want you to def go out and enhance your experience with them.
I hope you get a chance to try this stuff out. As always, please file feedback/issues on github.com/dotnet/Roslyn
Thanks!
Yeah, I know, many of the features of ReSharper are something VS should have had for a decade, regardless whether ReSharper has them or not. We - VS users - tried to tell VS dev teams in around 2005 that JBuilder (of all things, if you can remember that - it got eventually Eclipsed) had better refactoring and navigation tools than Visual Studio. The message from VS has consistently been that you are going to fix that, and when I heard of Roslyn, I though that this time it may be really happening. Only it has been ten years. Luckily .NET/C# was marginally better from developer perspective than JVM/Java from the beginning (the difference has grown ever since), and ReSharper (and other plugins) helped to keep Visual Studio afloat until today.
Now that Microsoft has embraced open source, community contributions, shorter release cycles, .NET Core, etc., I think you have a great foundation for the next ten years of .NET.
Oh, rest assured, I will continue to bug you guys through different channels.
Integration of style-cop-like rules seems ok, my only worry is that all those features will make VS sluggish. And that braces refactoring seems a poor example. Either you are a member of your team, and you don't write code without them, or it's some old code, that shouldn't be automatically refactored if the refactoring is the only goal.
> Integration of style-cop-like rules seems ok, my only worry is that all those features will make VS sluggish
They really shouldn't. We've actually spent the majority of our time in VS2017 on perf. So even with all the extra functionality, it's significant faster and more responsive across the board.
One thing that helps out here is that we don't run any of this analysis in the core VS process. As such, we are not contending for any resources that VS itself needs. i.e. a large source of hiccups in VS was due to the GC having to run right when a UI request was coming in. By moving this analysis out of proc we dramatically lighten the memory load in VS and we make it so that the GC runs less often, and takes less time. This clears up the UI thread more and makes things much more responsive.
> Either you are a member of your team, and you don't write code without them, or it's some old code, that shouldn't be automatically refactored if the refactoring is the only goal.
Or, you're a team that has decided on new, more stringent, coding requirements. And now you want to find and fix up all the violations. Or you're absorbing code by a different group with different coding patterns. Or, you just missed something during a code review and you want to go fix it up :)
If teams have a style preference, then it is definitely valuable to have automated tooling checking to ensure the style is maintained. Otherwise you're shifting the burden onto the team members, which is precisely what tooling like this is supposed to alleviate.
> And that braces refactoring seems a poor example.
NOte: we've added many style options in VS2017, and we're going to continue adding more. Options include:
1. Do you want field/property/methods/events qualified with "this." or not 2. Do you prefer predefined types (like "string") or do you prefer the .net fx types (like "System.String") 3. How and when do you use "var". 4. Do you want to use => for simple properties/methods/etc. 5. Would you prefer to use the more concise typechecking constructs over the previous approaches for type-checking. 6. Do you want to use ?? or ?. when applicable
etc. etc.