While I certainly have had a few annoying issues myself, full credit to the team behind DevOps (nee VSTS) for doing an amazing job over the last >12 months to overhaul and improve the product. I have been impressed at the pace things are improving.
I have not had the same issues with build machines disappearing into the ether (we use Windows hosts however) but definitely agree that the (lack of) caching is maddening. Our apps are not huge but the NuGet & npm restore times now account for 25-30% of our build time.
I exchanged an email with the Product Manager of some Azure DevOps component that comprises the build pipelines. They’re well aware of the need for caching. Hopefully it comes soon!
Unfortunately, they've been aware of this problem for years - I'm an Azure DevOps fan, but I really just don't understand why they didn't fix this an age ago :/
I haven't had a build agent disappear, but lack of build caching is very frustrating. The hosted build agents are slow to begin with, let alone needing 5 minutes for every build to restore packages.
AFAIK there's no way to "only run changed tests" too.
EDIT: Forgot to add that, despite the above, I'm pretty happy with it. The whole VSTS experience (Wiki/code/PRs/etc) is pretty great, it's nice to have everything in a single place.
I've been using Azure DevOps for around 2 years, and TFS for several. Most of what this article says, I simply don't see in (almost) every day usage.
The single issue I agree with is the lack of caching - installing NPM packages can take up to 15 minutes in the worst case. MS have known about this for years, so I honestly don't understand why they haven't done something about it.
I also think the hosted build machines are underpowered, despite their claims of Xeon CPUs - build on even an Azure B1ms VM (a low-powered, burstable SKU) are up to 3 times faster. In fact this is how we solve the NPM caching and performance issues - by running the VSTS Build Agent (it's OSS!) on relatively cheap Azure VMs.
Aside from these niggles though, I actually think the Azure DevOps product is fantastic.
I'm interested to hear you have not had the same CI issues I have. One theory I have with no substantiating evidence is that Linux agents are least commonly used. The Linux agents on hosted builds run in a hypervisor but I've seemingly had the same issues running self hosted agents (as I mentioned AWS).
I can't find an actual issue and again as I mentioned in the post, replacing my AWS instance with a new AWS instance also eventually hit the same issue. Maybe it's some perfect storm of problems but that is just more speculation.
Regarding caching, with NPM you actually have a couple good options (best together):
1.) Make sure you are using `npm ci` instead of `npm install`. The `ci` install command is often much faster and skips a lot of stuff that `install` does. It's also better at sticking purely to your package-lock.json versions. I suggested on UserVoice at one point that Azure Pipelines should switch to `npm ci` as the npm task default. `npm ci` is a relatively recent npm command and I don't everyone has caught on to it yet.
2.) You can use Azure Artifacts for NPM packages. It acts as an alternate NPM feed and caches packages to your Azure DevOps account. Artifacts only supports NPM and NuGet, so it isn't a general caching solution, but in this specific case it helps. I've noticed the builds I have using Artifacts are a bit faster than going straight to NPM. (I started using Artifacts for private NPM packages, so I'm not using it in every build yet.) The current biggest caveat to using Artifacts as your main NPM feed for a project is that it doesn't support NPM audits currently. (I also made sure to report that on UserVoice.)
Why on earth would you use a different deployment stack in CI than you would in production? This seems to miss (at least) half the point of integration testing.
Making this decision is bad enough, but making it because you've chosen a bad CI/CD stack is even worse. Use the tools that work, instead of breaking things to match.
Are you talking about Azure Artifacts? Like I said, I'm using Azure Artifacts as a Private NPM repository for proprietary packages. It also providing caching benefits was a pleasant surprise.
For my projects using Artifacts as their NPM repository, they use an .npmrc at the repository top level to only use that feed (and its pass through capability to the public feed), so for those projects it is the same stack from dev through to production. As I said, the one caveat is that if I need to `npm audit` (which is generally a good idea) I currently have to redirect to the main public NPM feed. Other than that it is a solid replacement NPM feed.
I find it weird that in order to fix a problem Microsoft has created (slow builds due to lack of caching) is to give Microsoft more money. Artifacts isn't free.
Sorry, it's easy to forget what the commodity prices of some of the Azure DevOps tools are given that various combinations of MSDN licenses and Office 365 plans cover them. It's great that you can pick and choose now, though.
Like I said, caching is only a useful side effect of Artifacts for npm and NuGet users (that have access to Artifacts). I would believe/hope that Microsoft may still have interest in adding proper caching to Azure Pipelines directly.
I was completely unaware of `npm ci`, so I'll definitely check that out. We'll stick with hosted agents that do caching though, as the hosted build agents are really slow.
I've helped a few clients migrate from whatever ad-hoc process they had to CI/CD with Azure Devops.
Apart from one minor issue with a bug in node.js timing when running the build server on burstable VMs (which is probably more to do with AWS setup than node), I've not had any issues that I can think of.
I don't have the problem with cache though, as I always run my own build servers/agents.
The UI has changed a few times over the past year, mostly around the rebrand from VSTS to Azure DevOps. But once learned, it's pretty straightforward and I'm sure I've barely touched the surface of its capabilities.
I, for one, am very impressed.
EDIT: oh, one improvement I can think of, would be to accept web-hooks rather than polling Bitbucket. But honestly, that's only a delay of a minute or two, so it's not a big deal. You could also easily enough create your own serverless webhook listener that triggers the build, so it's a bit of a non-issue, but I was surprised at the lack.
Thanks for the feedback @davewasthere. I'm an engineer on Azure DevOps. Our bitbucket support is built around web hooks, not polling. If you are seeing delays email me your azure devops org name and we can look into it. My email is my HN name at microsoft.com.
Tangential: Reading this _really_ makes me appreciate the extremely stable system my team has build internally with GitLab and TeamCity. Azure DevOps sounds extremely frustrating (besides the point that I cannot stand the name they gave this product, and GitLab made that same mistake with the Auto DevOps feature name).
BTW We're planning to add per per minute for the CI runners and add shared MacOS and Windows runners so people that don't want to manage their own no longer need to.
That's awesome. We have so much customization around TeamCity that moving away from it doesn't make sense, but we're starting to look at the CI runners for other things, along with the k8s integration, all great stuff in GitLab. Yeah, the DevOps name though, it's unfortunate. DevOps is a culture, not a thing, product, tool, or process. Seems more like calling it Auto CI/CD is more appropriate, maybe? Anyways, thanks for the response!
We moved to Azure DevOps for Windows testing for multiple large projects (Next.js, Now Desktop, Hyper) and it's been an absolute pleasure. Much better than our previous Windows CI solutions.
We use Azure DevOps for a fairly large project and so far its working good (i.e., not great, but better than okay-ish). The point with the lost communication is something we also experienced more often than I would wish for, however, on a standard day I do not see the issue appearing at all.
Yes, the mobile page could be optimized (but this is not a deal breaker) and yes some pages do not subscribe for notifications, but usually all these things are not in the hot path.
TL;DR: Lots of improvements in the recent year and many things to come - it may not be perfect but it certainly is usable and provides value to our team.
The majority of the product indeed does work pretty well. We make heavy use of their Boards and Test Plan features and outside of specific but minor UI/UX complaints, it is a competent tool.
You are correct, I'm just an Android dev but the company I work for is a long time user of Microsoft products like Visual Studio and obviously Windows. Over time, if anything, we have become more invested in their products and I don't see that changing in the foreseeable future as they generally work well for our use cases.
PM for Azure DevOps here. We've been investing heavily in our user experience and our CI/CD experience, so I'm sad to see that we've disappointed here.
Some of these complaints I would agree with - in particularly, we're not (yet) caching build resources - though we're working on this now. But most of these complaints I was quite surprised to hear; not an experience I would want someone to have or what I see from the majority of our customers. So this is certainly a place where I'll follow up with the author to learn more.
Funny I got an email from an Azure PM after I stopped using the free tier asking if I ran into problems. I explained my issue and what I gave up on, and.... never heard a response. I thought that was kind of weird, why even ask?
They ask the question, but any response is prohibited unless they cycle it through a pr department. It's about public narrative, and how it changes sentiment. So unless that response pays them, it's better to avoid saying anything than saying the wrong thing and pissing someone off, especially the wrong influencer.
(Former Azure DevOps PM here) That's not quite true - did you hear that from someone? When I was there, I had pretty much full freedom to talk with customers, as long as they hadn't opted out of contact. Never had to talk to PR or have emails looked at.
I dunno if you guys have gotten feedback on this, but the difference between build artifacts, artifact feeds, and the output of some pipelines' tasks in terms of artifacts is really confusing.
I know it's probably getting translated to yaml from the old way which was a bunch of forms with build options, but we had some trouble with this. At the time, yaml releases were also still in development/preview mode.
One thing I'm going to miss is being able to discover things through the build wizard. Now it's just a blank yaml file and I feel that hurts the initial impression. I'd really like an option to build yaml files using the old way, but still get the yaml output for version control, that way I have all of the options and their potential values immediately available and I don't have to memorize and type everything.
There's a bunch I like about Azure DevOps though. Hopefully I haven't missed something really obvious.
Azure Pipelines PM here. Definitely hear you on the current YAML experience. We're working on updates to the editor now to allow you to quickly browse and insert task snippets. It's going to be a big improvement.
The old UI was better, and forcing people to this weird pretty UI doesn't make sense.
Several products aimed at tech people seem to make this mistake. We are power users, we don't want stuff hidden away. Stop having UX that's designed as if we were trying to browse twitter.
This is spot on, I've been using tfso or devops, brand of the month. I hate the New UI, its really clear that the people are focus on making something that doesn't need to be pretty, pretty at the expense of functionality.
For example, when you go to do a release and you of course want to do release notes, previously you could copy and paste the related issues output into any document.
Now its been CSSified to be big, bubbly, and utterly useless to do anything but look at, and there is no print or export feature.
The new UI was useful when it was opt-in and we were migrating from on-prem TFS to what was then called VSTS. It made it easy to tell which system you were using.
I also liked the old UI far better. I have trouble finding things and navigating to where I want to go in the new UI. In the old UI it was easier to get an overview and to see where I have to click.
The UI keeps changing, but doesn't fix underlying bugs that have been around for ages.
Multiple issues where the builds don't actually update in the browser when they go through the steps (f5 only fixes), the builds show out of order in the UI, and many many other wonky things.
Also releases aren't YAML yet. Builds are, and are a good feature.
Azure Devops is almost perfect, and with a bit of TLC would be excellent.
YAML build definitions are not supported for GitHub Enterprise, which is mystifying to me seeing as how they are supported for GitHub. This is a critical missing feature for us and will leave us having to manually create huge JSON blobs for submission to the API.
@thejosh I'm a PM on Azure Pipelines and I'm interested to hear the builds UI ordering and other wonkiness feedback if you want to share. You can email me at my firstname.lastname at microsoft.com and we can do whatever works from there.
The user experience leaves so much to be desired. The tools are not well integrated, the UI is really slow, there’s no dashboard view of active pull requests, builds, releases, etc for my favorite repos. Build/Deploy times are insanely slow for .NET Core micro service projects. We have a project with 4000 unit tests (not integration tests) and it takes over 7 minutes to do a full build + code coverage. The same operations locally take a fraction of the time.
> We've been investing heavily in our user experience and our CI/CD experience
I'm on the "new" UI and I'm not seeing it. You've just re-skinned the same buggy and broken cruft that was there before.
It reminds me of the Visual Studio 2019 blog posts about new icons, themes, and fonts as their big new features while ignoring the massive performance problems people have been asking about for years.
DevOps with or without the new branding/UI is the same poorly designed product that doesn't know what it wants to be and is super un-polished. It would be better if you made it really good for one specific user scenario rather than poor for dozens.
Honestly, while your criticisms are mostly fair, your approach is really unprofessional here. Yes, Microsoft is a massive company, but human beings are making this and I’ll guarantee you none of them are happy with where the product is. Please consider showing some compassion.
I’d also add that I use 2017 and 2019 daily, and build by build, I see massive improvements for performance.
Generally the whole area gets a continuous stream of small and big features. You can click through the release notes [1] and see the changes they've made. Over the past twelve or so months they've jettisoned XAML-based builds and added YAML builds.
As someone who's used it through VSO/VSTS/ADO days it's got foibles but yes, it's improving continuously.
I don't think that's a lie. They have certainly invested heavily in changing the experience --- but unfortunately, and like many other products seem to be these days, just not in the way that you want.
(It's not just you. The "why did they spend all that effort to change things that didn't need to be changed" question has come up for me countless times upon seeing redesigns of other products.)
I spend a few hours recently in Pipelines and the whole UI is confusing. One of the offender being the edit button being literally at the other side of the screen. Outlook.com UI was also recently destroyed so it is hard to trust Microsoft services, even if good, will stay usable in the future.
The whole pipeline debacle is confusing. I recently did a demo at a Microsoft DevDays showing Microsoft hosted Ubuntu 16.04 agents in my build and then a self-hosted vsts-agent container (provided by Microsoft which still has the old name, why?) and part of the point was to show how slow not having caching is, even for a simple Python build pipeline. On top of that Microsoft has some odd pricing in that they limit your self hosted pipeline concurrency to 1 unless you pay them. That seems like a bad decision because most people would probably go through the effort of spinning some VMs natively in Azure which would increase overall OpEx. But instead, like the OP of the article, people will self-host on prem or in another/cheaper cloud. Microsoft gives no incentives to use native build agents (they are unreliable and slow) and almost force users to go self hosted but also charge for concurrency.
I've also had very odd issues where agent pools will have an idle worker but the pipeline will sit saying there are no agents, waiting. Eventually it goes, but... It's just a bad impression when you're showing it and it's not consistent.
Also, the UI. Tis horrible. Take a nod from AWS guys and get rid of the widgety attempt at entertaining some exec who doesn't use Azure. AWS interfaces load consistently for the most part and do a much better job in being not annoying, comparatively.
Another bad user experience is if you write a pipeline, by hand, without the GUI and try to run your pipeline - well, good luck. Authorization for certain components of your pipeline will fail, even if they follow the pipeline documentation to the T. Why? Because the pipeline web GUI workflow does linking of authorization between components on first run and if you only kick your build off using, say a code check-in, the pipeline will just fail. And it will give you a non-descript authorization error until you pull all your hair out and randomly do a commit from the Repository GUI of which it will pop up a message asking you to authorize the thing that had been failing for the last hour. Either document this better or give users a descript error of the actual problem. This, again, is oft where I find Azure lacking.
I am from the engineering team in Azure Pipelines. We agree that the resource authorization experience needs improvement. We have been working on it. We would like to make sure that you can only include the reference to a resource (secure file, service connection, variable group, agent pool, etc) in your YAML file if you have permissions to access that resource. And, it becomes particularly hard to auto-validate this if you are pushing your YAML change directly to GitHub or another repository since we do not understand the identity of that user. If we are not sure, then the build fails with a resource auth error. You can do two things to rectify this - (a) with a recent change we rolled out, you can allow all pipelines to use a resource. This is a toggle that you will see on the resource (b) You can get someone with permissions to open the pipeline in the Pipelines hub of Azure Pipelines, and queue a build. You will get an option to correct the resource auth problem. We also made a recent change so that new resources that you create default to "usable in all pipelines". We are taking steps to improve this experience without compromising on the security of these resources for those that need that capability. More is planned in this area over the next couple of months. Thanks for understanding. I will also update the docs on this topic with all the updates that have been rolled out this week.
Thank you for the great explanation on changes you've made and will continue to make in this area! What is the best way to inform persons such as yourself about UX issues? Appreciate the thoughtful response.
We use Azure DevOps extensively at my work and, after having used GitHub, Gitlab, self hosted solutions, Jenkins, TeamCity... DevOps ranks dead last.
It's the entire experience. The UI is terribly clunky everywhere. The worst for me are pull requests. Incredibly tough to work with people on a pull request. I can't even point you to "a" particular problem - for us it's broken everywhere. From the weird file comparison with overlays to the missing support for (us) important file formats like ipython notebooks. The way notifications bubble up in a facebook like format on a pull request. The emails that have 20000 lines and never the exact information I need. The amount of clicks to go from comparing two files to open the current version. Other people mentioned the lacking caching support for builds, so I am not going to go into more detail here. Another missing piece for me: Where are my statistics. I'd love to see commit, PR, response, comment etc statistics per team member / teams / groups.
Using Azure DevOps board is a nightmare of its own...
btw: most of these issues are part of the whole "azure" experience. The Azure portal suffers from the exact same interface issues of horizontally stacking fields. One wrong click and your last edits have disappeared into the ether
Keyboard shortcuts across Silverlight applications.
I have no idea what became of Silverlight. But I liked it very much more when I understood Silverlight was the future.
(how anyone understood the tech road maps at the time Silverlight was introduced, I can't imagine)
I personally see the problem is that search engines are disinterested in the furtherance of any kind of consistency capable of reducing the utility of a layer of discovery that they can provide. If you explained to your senator, that whoever dominates the browser market, effectively controls how all the information contained in the www is displayed and arranged by publishers, and the standards bodies intent to introduce semantic markup and accessibility across the www potentially can eliminate the need for search engines for a variety of constituencies and professional workers, no less cutting spam most certainly in the process, maybe they'd see the inherent conflict to be as undesirable as I see it to be.
I keep thinking I need to at least spend a while considering how possibly a browser might technically introduce undo / redo for arbitrary websites. This is what I would be trying to accomplish, if I held the necessary influence with the Edge team. Even the ability to be informed whatever it is that I just did to the page, would be enormously valuable to me and everyone I know. I'm going to risk thinking that since Microsoft are contributing to the chrome codebase, and the default supposition is surely that Microsoft wouldn't want any advancement of any features that have desktop equivalence, maybe a diplomatic possibility exists for introducing the semantics and accessibility consistency we desperately need, by a little sacrifice. I would love to be able to hit a key combination inside a Azure page, that thanks to the aforementioned accessibility standards, had a (possibly expanded) command line consistent equivalent and, permitting a little browser specific markup, could, with a single further stroke, break the interaction out of the browser and into powershell. It seems to be the aim for Microsoft to be the universal developer interface for all. This is entirely consistent with the same idea. The number of developers is only likely to be increasingly close to the whole user base of PCs, as voice and other interfaces develop. Get VSC for VB6 on Android and the job's done, in one short generation ...
Same here.
Azure has a killer feature in India that neither AWS or Google have - multiple data centres. Because of Indian data residency guidelines, that becomes critical for business continuity planning.
But the Azure UI is so bad that people are willing to forego stuff like that just to avoid being on Azure.
The Azure product teams have asked for feedback from startup multiple times with consistent answers on UI - I'm no longer willing to believe that this isn't the consequence of some global UI mandate to make things look "windows-y"
Its a single geographic region. Its not allowed according to financial or healthcare data regulations in India.
P.S. i personally am working with the "Big 4" consulting guys to get a white paper to work with this. But big missed opportunity for Azure. All because of bad UI.
I just manage cdn and domains with this interface, it is super annoying, slow and complicated, already. I don't want to know how it is with more complex stuff...
One additional point... it would be GREAT if I could have a chat channel in MS Teams with just notifications of @user/group where I am a part of that user/group in VSTS. I get so many emails, I really can't/don't keep up with it. Generally, email for this type of thing sucks, and the email doesn't show the meat of what you need to see.
A dashboard interface for messages/notifications in VSTS would be helpful.
Aside, switching to a GraphQL type interface with something that updates your UI consistently would be nice. Some screens update fine, others just get lost/disconnected and you don't even know where you are at.
Overall, I haven't experienced the agent issues in TFA, but I've had plenty of other issues and continue to use the beast that I know. I do think the issue tracking is decent and the integration is as nice as other products I've tried imho. Sometimes simpler is easier though.
I feel like I must shoulder some of the responsibility here for not being louder about these issues. But must say, I'm disappointed to hear you're "surprised" about the UX issues. I've been telling your folks the UX is dreadful (e.g. as far back as pre-launch) and kept hearing back "we know, we're fixing it". I'll start formalizing the feedback and push it through the pipes, stay tuned. I'm also local (Bellevue), would love to come in and try to pipeline our relatively simple oss .net/wpf/uwp app. I suspect it'll be an eye opener for the both of us.
Some examples:
* You can't build a pipeline with a git repo. that contains submodules
* Found it impossible to edit the PATH for some custom tooling
* The New Pipeline experience just doesn't make a lot of sense, new users clicking around will eventually end up at the wrong Docs.
I believe their feedback mechanisms are placebo, pacifiers for loud people. Microsoft pesters you to give feedback and works hard to ignore them. And then goes ahead with what they initially planned. At the end, they'll announce - "We had delivered what you wanted. Based on feedback..."
And then you start wondering if you're in the minority and other people are simply crazy until you see multiple complaints about the same things.
Their music app, Windows phone, Windows 10, Skype and Edge are where I experienced this. In all cases, user experience kept getting worse with each "IMPROVEMENT."
The only time Microsoft listens is when people ignore official feedback channels and there are massive outcries on many fronts. Even then, Microsoft often retreats only temporary.
These large cut and paste answers with completely irrelevant instructions and a lot of begging for points. Totally useless and infuriating if you've spent a lot of time writing a good report.
Yes. Incidentally Connect used to be a placebo for us enterprise lot. This has been going on for a long time.
And I get told regularly by people who only use the Microsoft ecosystem that’s it is me who is the problem as well!? These people are just assuming the status quo is being punched in the dick all day.
And even worse when dealing with PlayReady. In the time i have been working with it, they have been switching between at least 4-5 different "Connect"-like portals for feedback and support. Each time dumping all the content from the previous that was no longer accessible.
I'm not in Redmond, but do send an email. I'm my HN name @microsoft.com. We should be able to sort out some of these issues for you. I'd like to hear more about the docs problem. But:
You should be able to recurse submodules in a pipeline. In the visual designer, select "Checkout Submodules" in the pipeline's get sources step. If you use YAML, set "submodules: true" or "submodules: recursive" in the checkout keyword.
You should also be able to specify environment variables (including PATH) with the env keyword. But do reach out and we can get to the bottom of this.
Hey, just one bit of feedback, I tried getting my github linked project to build late last year, but always failed to checkout sub modules.
The build log showed permission issues, and I did try workarounds suggested like changing the sub module URL and adding ssh keys as part of the checkout step, but in the end, my 8 hours allocated to “get vsts building” ended with this problem and I have not continued. The performance of builds was also poor.
It’s unfortunately burnt me enough that I’m just making an ec2 instance with teamcity.
It is likely that this is a variant of the issue with secure files. Secure files require a song and dance around changing the default branch or committing directly to a default branch to trigger a build after setup of the secure file. Coming from CI systems like Travis and Circle, this is an extremely confusing approach that I hope they consider changing.
Yes, we rolled out some improvements to how we authorize secure files (and other resources such as variable groups, agent pools, service connections) that you use in a YAML file. First, we introduced a toggle on each of these resources to mark them as "authorized in all pipelines". For new resources that you create, this will be on by default. Second, when you queue a build in the new YAML editor under Pipelines hub, then you get the option to correct the resource authorization issue. More work needs to be done here so that we can give you the right experience while maintaining the security of protected resources, and all of that is planned over the next month or two. Thanks.
I'm sorry that the product wasn't easier to use, and that it wasted so much of your time. For anyone else running into this, we do have docs up about how to access authenticated submodules: https://docs.microsoft.com/en-us/azure/devops/pipelines/repo... (that shows how to do it against another Azure Repos instance, but a variation will work for any Git provider with a PAT-style bearer token)
To your path issue... most of my own pipelines do an `npm run STEP` and I use JS scripts mostly... this covers me in window, linux and macos for the most part with Node. ymmv here though. That's just my $.02 and it's been relatively easier doing it that way.
I kind of wish $bash (git for window's bash version) was available as a shell option as that might make things easier as well.
Another issue is searching is painful... "TFS" vs "VSTS" vs "Azure DevOps" and half the time, the results are dated.
Sorry, I don't understand what you're seeing here - you're creating a new project in Azure DevOps? When you do this, you get the choice between Git (hosted in Azure Repos) or centralized version control (TFVC hosted in Azure Repos). We don't have an option to do anything with GitHub here.
So I think that I'm not understanding what you're creating, or where. Can you drop me an email with a screenshot? It's my HN username @microsoft.com. Thanks!
I will clarify. if you go to portal.azure.com --> Create a resource --> DevOps project --> Bring your own code --> In code repository you can choose between "GitHub" or "External Git". What I expected was that I could point it at VisualStudio.com - unless vsts has become the external git?
ps: happy azure devops / azure user here ;)
I was pretty excited to see steadily improving GitHub integration. Unlike the others here, DevOps has been my most positive experience compared to a few others like Travis and AppVeyor.
However, this confidence was seriously undermined when I noticed recently that my GitHub pull request validation build stopped working when I closed the DevOps tab.
Yes, seriously. I have to keep the DevOps tab open for my builds to actually trigger. My mind boggles at how such a design could even be possible.
I'm an engineer on Azure DevOps. GitHub PR build triggers are based on notifications our service gets from GitHub (purely service to service), and is unrelated to any state of the UI. Can you reach out to me on email so we can investigate? Use my HK name at microsoft.com.
There's a lot of comments about our UI and I thought I'd post a quick followup here instead of replying to each one individually. We are definitely reading these comments. UI itself is polarizing, and changing a UI moreso, but there's a lot of critical feedback here that we're going to investigate, try to understand and improve.
Really appreciate you engaging here - in addition I think there would be a lot of value in taking a closer look at the feedback that you're already getting on user voice, especially for issues that you think you've closed.
It looks like you did something that users didn't actually ask for in 2017, closed this, and then have been ignoring 1.5 years worth of comments being told what users actually want. From our end it's simple - the embedded burndown chart on the scrum board should be able to show story points, not just hours (like with any other scrum board tool). I shudder to think how many engineering hours went into solving the wrong thing here...
The old UX was not perfect, but I find that it worked better in so many ways. New UX does not seem finished, just pushed out in beta state. Like O365. Sad.
I've migrated my Boost builds from Travis and Appveyor to your CI offering recently.
It wasn't super smooth, but I got it to work quite fast and I like it a lot.
So sure, I'm missing caching that I've had on Travis. But honestly, it builds reliably fast anyway. On Travis, I always needed to wait a LONG time to get macOS agents, on Appveyor, we were limited to 2 parallel builds. Here, everything starts all at once, completes under 10 minutes (there's quite a lot to build and test).
If I could, I would love to have an sccache ( https://github.com/mozilla/sccache ) compatible distributed free caching solution for my C++ projects. That'd be a killer feature!
You can add as many additional parallel builds as you want to AppVeyor for $50/m each - is Azure CI pricing comparable to AppVeyor with only two concurrent builds?
Last time I checked, the free offering was 2 CI builds in parallel on AppVeyor, which is just too low and slow for feedback on my project.
Also, considering I make exactly $0 per month from it, it's either be patient or use another service that had more capacity and faster builders.
I believe that Travis-CI had a higher limit for parallel jobs, but your builds would most often end up queued because of a low capacity for a certain type of agents. Also, something quite noticeable was their lower limits on build time, which is why for some builds a cache was necessary. It got a bit hairy when you needed a full successful build to properly populate the cache but they would never finish in time.
And of course, the biggest drawback from Travis-CI back then was the lack of Windows support.
As for Azure CI, I believe there's something like a 10 parallel builds limit. And I've got exactly 10 configurations!
Often when I click Releases it is completly blank. All our releases are missing. I have to refresh 2-3 times before they show up.
The UI has so far changed five times in the past three months. It is insane. The current iteration is at least fine. Before it was a big blob of round circle that showed that the steps that succeeded succeeded. And then a big text saying it failed. It was worthless.
I’m considering migrating an appveyor setup to devops. The main driver is to get a pipeline setup with more parallelism and a more complex graph with resumable steps without fiddling with environment parameters, or really jumping around in a clunky ui maintaining a hundred “projects”
Haven’t looked to deeply what devops has to offer yet, but things I’d be looking for before migrating.
- I want to be able to develop, run and debug most, or all, of the steps in the graph on my development machine without roundtripping through long chains of triggers and pushing to code repos and pull requests
- I want to run as much as possible in parallel, preferably agnostic of wether it’s on the same machine or several
- I need to build and assemble many components from same repository (monorepo setup)
- only impacted components should build
- Sub-components may depend on different build tools (service fabric services with both webpack and msbuild based packages)
- Some steps may depend on configurations and secrets with a different lifecycle than the code branch being built
I imagine something clever could be built on docker to support both running things locally as well as customize build nodes.
I don’t care to much for scripting in yaml. In fact I spent the day converting some yaml (and cake) scripting to Invoke-Build. Invoke-Build is quite nice actually, the only thing I miss is the option to orchestrate tasks in parallel onto several build nodes. That plus something like ncrunch build/test engine for csharp projects
(really the entire vs/msbuild things needs to be replace with something truly distributed, incremental and repeatable)
I can recommend https://concourse-ci.org/ for pipelines that are difficult to reason about or orchestrate in, e.g. GitLab and other CI tools, due to external actors or lifecycles beyond a single codebase/repo.
Builds are 100% stateless within the CI cluster and rely on tracking/publishing just about anything you can imagine or plug as a resource; out-of-the-box inclusions being docker images (registry), S3 (e.g. minio, AWS) and git.
It's a bit strange to reason about in the beginning but I have yet to try anything that comes remotely close to Concourse when it comes to orchestrating across multiple sources of input/output and non-linear pipelines.
I forgot to mention the semver resource which is a bit weird to use for a single repo pipeline but comes in handy for out-of-band jobs like running an external automation test suite asynchronously.
I think title should be updated to "Microsoft's Azure DevOps CI: An Unsatisfying Adventure", as all the complaints here are about builds but this service supports lots of other features.
My experience with using DevOps nee VSTS was mostly satisfying. For various reasons unrelated to the post, my team ended up installing the build agent on our own bare metal hardware, and other than needing to restart the agent, I didn't have any mission critical issues. Totally agree with all the UI complaints.
I've been using this day in and day out for several years now on a ton of projects. We mostly use hosted git, and pipelines. The lack of NPM cache is a giant pain, and that's why we run our own build agents. Other than that, I can't say enough about it. Compared to some of the monsters I've seen built with Jenkins, I love the simplicity of the builds and releases we have created in Azure DevOps.
This article is nonsense. Azure DevOps is a delight to work with. PR system is probably the best I've worked with so far. Build pipeline can improve but it's a far cry from a "horrifically broken".
IMHO, Microsoft has been getting a lot of things right lately. Keep up the good work.
Horrifically broken was an exaggeration, you are correct. Much of this was written after a stressful day compounded by dealing with our CI failing to do the thing it is supposed to do. I also agree with you that Microsoft has been making great strides in the direction of "getting a lot of things right lately" as you said.
We really hate to hear you are having such a bad experience. The engineering manager has reached out on Twitter and included me. I would love to see what I can do to help
In the opening paragraphs I stated it was my hope that this gives me a gateway to communicate with someone that isn't support and isn't the void that is their robot response issue tracker.
That said, they generously reached out to me and appear to genuinely be concerned by my experience. I plan to keep posting as I run into issues and also as they correct my mistakes and also solve my problems.
It just strikes me as an odd thing to put on a luxury tier like that. Reiterates that quality and testing is often an afterthought rather than something everyone should include. If MS wanted the revenue they would put something the most senior higher paid developers wanted rather than the testing team.
True, but I thought the goal of Azure DevOps was to be a complete package solution. And so while all the pipelines and code and work items are stored there, it would be a pain to keep test cases in something else. And why no integration with Excel?
(Azure DevOps PM here) Test Manager comes with VS Enterprise, VS Test Professional, MSDN Subscriptions and is also available for direct purchase through https://marketplace.visualstudio.com/items?itemName=ms.vss-t.... You are welcome to take your pick from what suits your needs.
As a subscription, it's more per month than Visual Studio Professional. As part of VS Test Professional it's almost twice as much for the first year. And you can't really do any test plans without it so its a huge part of DevOps that is just unused for us, unfortunately.
I've yet to look, let alone try, Azure DevOps. There would have to be some compelling reasons to choose that over TeamCity, which I have had nothing but great and positive experiences with.
Any big reasons DevOps is something to consider, or is it mostly ideal for new projects?
We use paid Azure DevOps for Source Control (although we're watching GitHub closely).
We tried to also use Azure Boards (Work Items, Boards, Backlogs, etc). Ouch. It is a complete UI mess of disjointed ideas. Instead of implementing one thing well, they implemented two dozen things terribly.
We actually migrated from Azure Boards to Office 365's Planner (KanBan) just because the UI wasn't so horrible. It is simple and gets out of your way. In fact O365's Planner, Teams, and OneNote Online products might be superior to anything Azure DevOps has produced.
The only nice thing I have to say about Azure DevOps Boards is that it integrates into Visual Studio, but frankly even that is barely passable. I'm predicting they scrap DevOps completely and just use GitHub as a clean slate.
I don't even use VS if I can avoid it... web dev is so much smoother in VS Code, and the laginess of VS is unbearable to me. I do have a couple .Net Core projects I mostly use in VS Code and one MS SQL DB project that I have to use VS for.
We are using the on-premise solution with our own build agents to build iOS. Looking to move actually into Azure Devops to reduce the effort of maintenance on our own build agents. I've been using AppCenter for hobbyist projects and I feel its been a good replacement for BuddyBuild. Seems to be pretty reliable additionally we've thrown a lot of builds at Azure Devops to validate how well those hosted-agents can handle a large project which also worked out well. So all in all, we are excited about the hosted-Mac agents as well as the flexibility to BYO-agent (helps in the transition).
I have seen a lot of these same problems noted in this link as well, just didn't see to prevent from getting our work done as much. So YMMV depending on your product/devops use case.
Personally have been using their APIs with some custom CLI scripts to get around some of these issues.
We migrated from TeamCity and OctopusDeploy a couple of months ago and we are already regretting it. The reliability of this system is just horrendous. The documentation is pretty much outdated, and quite a few plugins do not work. I have to write a bunch of powershell script to get it to work the way I want. And today, all hell broke loose.
Since this morning, we are stuck with an error message "TF400893: Unable to contact the server. This is most likely caused by a network error. Please check your connection and try again.". This has cause a couple of release pipelines to be get stuck, with cancel option not working either. The paid "parallel jobs" are being consume in this stuck pipeline and no other project from my organisation can be built or deployed because of it.
Pretty much regretting having spent so much time configuring the pipelines. We might switch it doesn't get resolved soon or if it happens even one more time.
At my company we use a mixture of Azure DevOps, Jenkins, CircleCI, Github, Jira, Artifactory, and a host of AWS and Azure, and my view is that Azure DevOps (formerly VSTS, formerly VSO, formerly TFService, formerly TFS, formerly VSS---i kid i kid) is a semi-strong product that just can't get out of its own way.
The New UI is a clear step backward...and the Github integration leaves a lot to be desired. If you have Github Enterprise...why use Azure Repos...why are MSFT teams moving from Azure Repos to Github...and others moving from Github to Azure Repos. The SCM platform and strategy obviously hasn't settled yet...so Gitlab is just as viable an offering that's growing more compelling by the day.
The Azure DevOps hosted build system hasn't been viable in my opinion since it launched, so on-prem agents are a much better option. Take a look at CloudBees offerings, they set the poll position.
The key thing that continues to be a pain point is the reporting and project organization data structure limitations with Azure DevOps: is it 1 TPC per business unit, 1 TPC for the whole company, 1 team project for the whole company. What happens when you have 2...how do you consolidate...what's the path forward if you have to move things around. The answer for the last 8 years has been silence.
The request for a clean solution has been asked for since 2011. Yes 2011, look at the link below. 8 long years...and basically silence...and "it's a hard problem"...and multiple updates saying "We will provide an update once we start planning for the second half of 20XX."
Eventually, the promises just ring hollow, the proof is in shipped code, and in the meantime other platforms and competitors stacks keep building out better enterprise stories for migrating from Azure DevOps (here's to look at Github, Gitlab, Atlassian's platforms, Jenkins/CircleCI, TeamCity, Artifactory, Xebia Labs etc).
Honestly, the world is a MUCH more competitive place with lots of best in class point solutions that aren't that hard to stitch together and get better performance/features.
Hope springs eternal...that with the new vertical offerings the DevOps team can deliver rapid value...but 8 years is a long time to only hear crickets.
If you're starting out and you need a one-size fits all solution and you're a SMB company with two doze devs...go hog wild with Azure DevOps...that's not to say it doesn't work for large companies like Shell and Microsoft itself...but talk with teams about the limitations they run into and make your own decision on your DevOps transformation adventure...oh and never create more than 1 team project in Azure Devops...seriously...if you do, game over.
> For a few hours, builds ran smoothly. As expected, eventually the agent died during a build. Builds sat queued, my frustration grew, and AWS notified my instance needed a doctor. After an hour I gave up hope for the unresponsive server and commanded it to reboot.
Sounds a lot like some process on the build server is leaking memory. We have a couple of older services that have this problem, and if we don't catch it early enough the server will become unresponsive and unable to even handle ssh connections.
This mimics the mac vs windows argument I had with my brother-in-law where it boiled down to the fact that you could run windows on something cheaper than a macbook. Even though in order to remap caps lock to something reasonable you have to edit the registry. In 2019. The best answer I think involves windows server 2003 resource kit and its UI, which is so old it could get a driver's license.
It's not even about the price. I have no problem tweaking my Linux desktop to use an international Dvorak layout (useful for those European languages with ä's and é's) and switching the caps and ctrl keys. Say what you want about Gnome 3, but this works just fine.
On Windows.. I get a US only bare bone Dvorak layout and few ways to remap the caps/ctrl keys. Messing with the registry is at best annoying on my gaming (only) PC but impossible on a corporate locked down desktop. Then I'm lucky if the corporate package repo comes with AutoHotkey that can remap keys in a very hacky way.
A lot of Windows 98 still shines through in the basic usability suite even on the newest versions of Windows. Even the on-screen keyboard on a (Android) phone is more fun than the HID mess in Windows.
I would love to find an .xinitrc in some dark corner of windows 10 where there are a bunch of options commented out and you have to pick the right ones or else your monitor explodes.
This would be an upgrade as at least I would kinda trust that a file I found on my own system from the vendor can probably be trusted. I trust the options would be at least mostly self documenting (more self documenting than a random list of hex digits). And, if I'm going to use software written in the 90s, I'm going to go with good old vim.
I love vim, but that .xinitrc stuff is really out of date by now. Gnome 3 now uses Wayland fairly well and the setup is done with the GUI app Tweaks and the system settings.
I'm only one person, and my team is only one team, but we have been completely satisfied with Azure and DevOps so far. But then again, we are a small team and tend towards many smaller projects rather than fewer large projects, so some of the issues mentioned here w/ caching and missing stats, for instance, may just not apply for us. We don't really need to see stats of how many pull requests each dev has made when there's only 4 of us doing backend work.
Additionally, we are a non-profit, and MS works with us in a moderately generous way with pricing, whereas AWS wouldn't.
I can only second that. Team of 10 with about 30 projects and Azure DevOps was the best that could happen to us. The only problem I saw in here is that the build ui does not automatically update sometimes. Another thing that annoyes me is that the tasks and stories do not update in realtime when you work on them from multiple machines.
I can’t even fathom it myself. I begrudgingly spent a month on azure as I worked for an MSFT partner. This was to establish if it was the way to go. It wasn’t. It was a friction filled mess.
AWS slowly crept in because there was no entry barrier, no magic Harry Potter spells to get anything working and it actually worked and stayed working.
Now I went back to azure to see what has changed Q3 last year and it was the same “eating sawdust” experience. Literally everything is clunky.
When it comes to these providers, time is money, possibly more so than worrying about the instances. Azure costs a lot more in human attention.
AWS is not perfect though. CloudFormation quite frankly sucks balls but there’s enough interest and so little friction that there are lots of third party things that work well with it.
1. I don't use Google products if there is any other choice.
2. I think Azure is better overall than AWS. It's way easier to grasp the different features, while I haven't used Azure Devops, I've had little issues with the rest.
I don’t see the relation between Azure DevOps (an issue management and build system competing with Jira, Gitlab and others) and GCP/AWS?
Even if I run on some other cloud than Azure, I still need the software. I could even run Azure DevOps on Amazon’s cloud I guess (If I choose the self hosted option) but the UX complaints would apply there as well...
My current job has various .NET/Windows-based software products that can run as PaaS projects in Azure - from what I've seen it wouldn't be practical to run them on AWS/GCP without managing VMs.
The silver lining here is that Azure DevOps's code review tool is lightyears ahead of all other repos. I would like to see this support for github repos
VM Agent based builds are an obsolete model. A docker/container-based pipeline is the current state of the art and allows for very fast builds with caching built in and isolated stages with narrow responsibilities.
There is an official Linux image for a VSTS Build Agent container - but, frustratingly, not for a Windows one! People have been begging for such an image for a long time now, but Microsoft do nothing... I just don't understand why they treat Windows as a 2nd class citizen!
196 comments
[ 7.8 ms ] story [ 266 ms ] threadI have not had the same issues with build machines disappearing into the ether (we use Windows hosts however) but definitely agree that the (lack of) caching is maddening. Our apps are not huge but the NuGet & npm restore times now account for 25-30% of our build time.
I'm on a team at building a C++ component and just installing boost can take over 20 minutes for a build.
I haven't had a build agent disappear, but lack of build caching is very frustrating. The hosted build agents are slow to begin with, let alone needing 5 minutes for every build to restore packages.
AFAIK there's no way to "only run changed tests" too.
EDIT: Forgot to add that, despite the above, I'm pretty happy with it. The whole VSTS experience (Wiki/code/PRs/etc) is pretty great, it's nice to have everything in a single place.
https://docs.microsoft.com/en-us/azure/devops/pipelines/test...
The single issue I agree with is the lack of caching - installing NPM packages can take up to 15 minutes in the worst case. MS have known about this for years, so I honestly don't understand why they haven't done something about it.
I also think the hosted build machines are underpowered, despite their claims of Xeon CPUs - build on even an Azure B1ms VM (a low-powered, burstable SKU) are up to 3 times faster. In fact this is how we solve the NPM caching and performance issues - by running the VSTS Build Agent (it's OSS!) on relatively cheap Azure VMs.
Aside from these niggles though, I actually think the Azure DevOps product is fantastic.
I can't find an actual issue and again as I mentioned in the post, replacing my AWS instance with a new AWS instance also eventually hit the same issue. Maybe it's some perfect storm of problems but that is just more speculation.
1.) Make sure you are using `npm ci` instead of `npm install`. The `ci` install command is often much faster and skips a lot of stuff that `install` does. It's also better at sticking purely to your package-lock.json versions. I suggested on UserVoice at one point that Azure Pipelines should switch to `npm ci` as the npm task default. `npm ci` is a relatively recent npm command and I don't everyone has caught on to it yet.
2.) You can use Azure Artifacts for NPM packages. It acts as an alternate NPM feed and caches packages to your Azure DevOps account. Artifacts only supports NPM and NuGet, so it isn't a general caching solution, but in this specific case it helps. I've noticed the builds I have using Artifacts are a bit faster than going straight to NPM. (I started using Artifacts for private NPM packages, so I'm not using it in every build yet.) The current biggest caveat to using Artifacts as your main NPM feed for a project is that it doesn't support NPM audits currently. (I also made sure to report that on UserVoice.)
Making this decision is bad enough, but making it because you've chosen a bad CI/CD stack is even worse. Use the tools that work, instead of breaking things to match.
Are you talking about Azure Artifacts? Like I said, I'm using Azure Artifacts as a Private NPM repository for proprietary packages. It also providing caching benefits was a pleasant surprise.
For my projects using Artifacts as their NPM repository, they use an .npmrc at the repository top level to only use that feed (and its pass through capability to the public feed), so for those projects it is the same stack from dev through to production. As I said, the one caveat is that if I need to `npm audit` (which is generally a good idea) I currently have to redirect to the main public NPM feed. Other than that it is a solid replacement NPM feed.
Like I said, caching is only a useful side effect of Artifacts for npm and NuGet users (that have access to Artifacts). I would believe/hope that Microsoft may still have interest in adding proper caching to Azure Pipelines directly.
Apart from one minor issue with a bug in node.js timing when running the build server on burstable VMs (which is probably more to do with AWS setup than node), I've not had any issues that I can think of.
I don't have the problem with cache though, as I always run my own build servers/agents.
The UI has changed a few times over the past year, mostly around the rebrand from VSTS to Azure DevOps. But once learned, it's pretty straightforward and I'm sure I've barely touched the surface of its capabilities.
I, for one, am very impressed.
EDIT: oh, one improvement I can think of, would be to accept web-hooks rather than polling Bitbucket. But honestly, that's only a delay of a minute or two, so it's not a big deal. You could also easily enough create your own serverless webhook listener that triggers the build, so it's a bit of a non-issue, but I was surprised at the lack.
We're open to a better name for Auto DevOps https://docs.gitlab.com/ee/topics/autodevops/ but we haven't been able to find one. Suggestions are very welcome.
BTW We're planning to add per per minute for the CI runners and add shared MacOS and Windows runners so people that don't want to manage their own no longer need to.
I agree DevOps is a culture foremost. But at the same time it is starting to turn into a product category (DevOps tools).
Hope you are able to adopt GitLab with ease.
Yes, the mobile page could be optimized (but this is not a deal breaker) and yes some pages do not subscribe for notifications, but usually all these things are not in the hot path.
TL;DR: Lots of improvements in the recent year and many things to come - it may not be perfect but it certainly is usable and provides value to our team.
I could find no evidence, looks like the guy is an android developer of some sort.
Some of these complaints I would agree with - in particularly, we're not (yet) caching build resources - though we're working on this now. But most of these complaints I was quite surprised to hear; not an experience I would want someone to have or what I see from the majority of our customers. So this is certainly a place where I'll follow up with the author to learn more.
Sorry that you didn't hear back though :(
You should reach out to some PM's on Twitter! I know many are fairly active, and I'm sure would love suggestions or feedback.
I know it's probably getting translated to yaml from the old way which was a bunch of forms with build options, but we had some trouble with this. At the time, yaml releases were also still in development/preview mode.
One thing I'm going to miss is being able to discover things through the build wizard. Now it's just a blank yaml file and I feel that hurts the initial impression. I'd really like an option to build yaml files using the old way, but still get the yaml output for version control, that way I have all of the options and their potential values immediately available and I don't have to memorize and type everything.
There's a bunch I like about Azure DevOps though. Hopefully I haven't missed something really obvious.
Several products aimed at tech people seem to make this mistake. We are power users, we don't want stuff hidden away. Stop having UX that's designed as if we were trying to browse twitter.
For example, when you go to do a release and you of course want to do release notes, previously you could copy and paste the related issues output into any document.
Now its been CSSified to be big, bubbly, and utterly useless to do anything but look at, and there is no print or export feature.
I'm more used to the old UI but some features are not easily discoverable. I wrote some posts on how to build branches and pull requests as it's not obvious: https://unop.uk/build-and-release-all-pull-request-merge-res...
The UI keeps changing, but doesn't fix underlying bugs that have been around for ages.
Multiple issues where the builds don't actually update in the browser when they go through the steps (f5 only fixes), the builds show out of order in the UI, and many many other wonky things.
Also releases aren't YAML yet. Builds are, and are a good feature.
Azure Devops is almost perfect, and with a bit of TLC would be excellent.
I'm on the "new" UI and I'm not seeing it. You've just re-skinned the same buggy and broken cruft that was there before.
It reminds me of the Visual Studio 2019 blog posts about new icons, themes, and fonts as their big new features while ignoring the massive performance problems people have been asking about for years.
DevOps with or without the new branding/UI is the same poorly designed product that doesn't know what it wants to be and is super un-polished. It would be better if you made it really good for one specific user scenario rather than poor for dozens.
I’d also add that I use 2017 and 2019 daily, and build by build, I see massive improvements for performance.
As someone who's used it through VSO/VSTS/ADO days it's got foibles but yes, it's improving continuously.
[1] https://docs.microsoft.com/en-us/azure/devops/release-notes/
(It's not just you. The "why did they spend all that effort to change things that didn't need to be changed" question has come up for me countless times upon seeing redesigns of other products.)
I've also had very odd issues where agent pools will have an idle worker but the pipeline will sit saying there are no agents, waiting. Eventually it goes, but... It's just a bad impression when you're showing it and it's not consistent.
Also, the UI. Tis horrible. Take a nod from AWS guys and get rid of the widgety attempt at entertaining some exec who doesn't use Azure. AWS interfaces load consistently for the most part and do a much better job in being not annoying, comparatively.
Another bad user experience is if you write a pipeline, by hand, without the GUI and try to run your pipeline - well, good luck. Authorization for certain components of your pipeline will fail, even if they follow the pipeline documentation to the T. Why? Because the pipeline web GUI workflow does linking of authorization between components on first run and if you only kick your build off using, say a code check-in, the pipeline will just fail. And it will give you a non-descript authorization error until you pull all your hair out and randomly do a commit from the Repository GUI of which it will pop up a message asking you to authorize the thing that had been failing for the last hour. Either document this better or give users a descript error of the actual problem. This, again, is oft where I find Azure lacking.
It's the entire experience. The UI is terribly clunky everywhere. The worst for me are pull requests. Incredibly tough to work with people on a pull request. I can't even point you to "a" particular problem - for us it's broken everywhere. From the weird file comparison with overlays to the missing support for (us) important file formats like ipython notebooks. The way notifications bubble up in a facebook like format on a pull request. The emails that have 20000 lines and never the exact information I need. The amount of clicks to go from comparing two files to open the current version. Other people mentioned the lacking caching support for builds, so I am not going to go into more detail here. Another missing piece for me: Where are my statistics. I'd love to see commit, PR, response, comment etc statistics per team member / teams / groups.
Using Azure DevOps board is a nightmare of its own...
btw: most of these issues are part of the whole "azure" experience. The Azure portal suffers from the exact same interface issues of horizontally stacking fields. One wrong click and your last edits have disappeared into the ether
I have managed to forget the search terms I entered, but the first thing I have stumbled on, is this :
https://www.w3.org/TR/WCAG20-TECHS/SL15.html
Keyboard shortcuts across Silverlight applications.
I have no idea what became of Silverlight. But I liked it very much more when I understood Silverlight was the future.
(how anyone understood the tech road maps at the time Silverlight was introduced, I can't imagine)
I personally see the problem is that search engines are disinterested in the furtherance of any kind of consistency capable of reducing the utility of a layer of discovery that they can provide. If you explained to your senator, that whoever dominates the browser market, effectively controls how all the information contained in the www is displayed and arranged by publishers, and the standards bodies intent to introduce semantic markup and accessibility across the www potentially can eliminate the need for search engines for a variety of constituencies and professional workers, no less cutting spam most certainly in the process, maybe they'd see the inherent conflict to be as undesirable as I see it to be.
I keep thinking I need to at least spend a while considering how possibly a browser might technically introduce undo / redo for arbitrary websites. This is what I would be trying to accomplish, if I held the necessary influence with the Edge team. Even the ability to be informed whatever it is that I just did to the page, would be enormously valuable to me and everyone I know. I'm going to risk thinking that since Microsoft are contributing to the chrome codebase, and the default supposition is surely that Microsoft wouldn't want any advancement of any features that have desktop equivalence, maybe a diplomatic possibility exists for introducing the semantics and accessibility consistency we desperately need, by a little sacrifice. I would love to be able to hit a key combination inside a Azure page, that thanks to the aforementioned accessibility standards, had a (possibly expanded) command line consistent equivalent and, permitting a little browser specific markup, could, with a single further stroke, break the interaction out of the browser and into powershell. It seems to be the aim for Microsoft to be the universal developer interface for all. This is entirely consistent with the same idea. The number of developers is only likely to be increasingly close to the whole user base of PCs, as voice and other interfaces develop. Get VSC for VB6 on Android and the job's done, in one short generation ...
But the Azure UI is so bad that people are willing to forego stuff like that just to avoid being on Azure.
The Azure product teams have asked for feedback from startup multiple times with consistent answers on UI - I'm no longer willing to believe that this isn't the consequence of some global UI mandate to make things look "windows-y"
Its a single geographic region. Its not allowed according to financial or healthcare data regulations in India.
P.S. i personally am working with the "Big 4" consulting guys to get a white paper to work with this. But big missed opportunity for Azure. All because of bad UI.
The API is probably your only chance..
A dashboard interface for messages/notifications in VSTS would be helpful.
Aside, switching to a GraphQL type interface with something that updates your UI consistently would be nice. Some screens update fine, others just get lost/disconnected and you don't even know where you are at.
Overall, I haven't experienced the agent issues in TFA, but I've had plenty of other issues and continue to use the beast that I know. I do think the issue tracking is decent and the integration is as nice as other products I've tried imho. Sometimes simpler is easier though.
I feel like I must shoulder some of the responsibility here for not being louder about these issues. But must say, I'm disappointed to hear you're "surprised" about the UX issues. I've been telling your folks the UX is dreadful (e.g. as far back as pre-launch) and kept hearing back "we know, we're fixing it". I'll start formalizing the feedback and push it through the pipes, stay tuned. I'm also local (Bellevue), would love to come in and try to pipeline our relatively simple oss .net/wpf/uwp app. I suspect it'll be an eye opener for the both of us.
Some examples:
* You can't build a pipeline with a git repo. that contains submodules
* Found it impossible to edit the PATH for some custom tooling
* The New Pipeline experience just doesn't make a lot of sense, new users clicking around will eventually end up at the wrong Docs.
I believe their feedback mechanisms are placebo, pacifiers for loud people. Microsoft pesters you to give feedback and works hard to ignore them. And then goes ahead with what they initially planned. At the end, they'll announce - "We had delivered what you wanted. Based on feedback..."
And then you start wondering if you're in the minority and other people are simply crazy until you see multiple complaints about the same things.
Their music app, Windows phone, Windows 10, Skype and Edge are where I experienced this. In all cases, user experience kept getting worse with each "IMPROVEMENT."
The only time Microsoft listens is when people ignore official feedback channels and there are massive outcries on many fronts. Even then, Microsoft often retreats only temporary.
It'll cost less, answer faster and wouldn't be any worse than the current outsourced indian community support staff.
And I get told regularly by people who only use the Microsoft ecosystem that’s it is me who is the problem as well!? These people are just assuming the status quo is being punched in the dick all day.
You should be able to recurse submodules in a pipeline. In the visual designer, select "Checkout Submodules" in the pipeline's get sources step. If you use YAML, set "submodules: true" or "submodules: recursive" in the checkout keyword.
You should also be able to specify environment variables (including PATH) with the env keyword. But do reach out and we can get to the bottom of this.
The build log showed permission issues, and I did try workarounds suggested like changing the sub module URL and adding ssh keys as part of the checkout step, but in the end, my 8 hours allocated to “get vsts building” ended with this problem and I have not continued. The performance of builds was also poor.
It’s unfortunately burnt me enough that I’m just making an ec2 instance with teamcity.
I kind of wish $bash (git for window's bash version) was available as a shell option as that might make things easier as well.
Another issue is searching is painful... "TFS" vs "VSTS" vs "Azure DevOps" and half the time, the results are dated.
beetlejuice,beetlejuice,BEETLEJUICE!
So I think that I'm not understanding what you're creating, or where. Can you drop me an email with a screenshot? It's my HN username @microsoft.com. Thanks!
Maybe you should start investing smartly.
However, this confidence was seriously undermined when I noticed recently that my GitHub pull request validation build stopped working when I closed the DevOps tab.
Yes, seriously. I have to keep the DevOps tab open for my builds to actually trigger. My mind boggles at how such a design could even be possible.
Great example here: https://visualstudio.uservoice.com/forums/330519-team-servic...
It looks like you did something that users didn't actually ask for in 2017, closed this, and then have been ignoring 1.5 years worth of comments being told what users actually want. From our end it's simple - the embedded burndown chart on the scrum board should be able to show story points, not just hours (like with any other scrum board tool). I shudder to think how many engineering hours went into solving the wrong thing here...
I have a 4K laptop, evan on high res, the lack of horizontal scrolling in Azure portal and other Azure tools is a dailly irritant.
Stop trying to badly emulate Android phone look & feel ... for desktop usage !!!
We are professionals, we can handle scrollbars without screwing up.
So sure, I'm missing caching that I've had on Travis. But honestly, it builds reliably fast anyway. On Travis, I always needed to wait a LONG time to get macOS agents, on Appveyor, we were limited to 2 parallel builds. Here, everything starts all at once, completes under 10 minutes (there's quite a lot to build and test).
If I could, I would love to have an sccache ( https://github.com/mozilla/sccache ) compatible distributed free caching solution for my C++ projects. That'd be a killer feature!
Also, considering I make exactly $0 per month from it, it's either be patient or use another service that had more capacity and faster builders.
I believe that Travis-CI had a higher limit for parallel jobs, but your builds would most often end up queued because of a low capacity for a certain type of agents. Also, something quite noticeable was their lower limits on build time, which is why for some builds a cache was necessary. It got a bit hairy when you needed a full successful build to properly populate the cache but they would never finish in time. And of course, the biggest drawback from Travis-CI back then was the lack of Windows support.
As for Azure CI, I believe there's something like a 10 parallel builds limit. And I've got exactly 10 configurations!
The UI has so far changed five times in the past three months. It is insane. The current iteration is at least fine. Before it was a big blob of round circle that showed that the steps that succeeded succeeded. And then a big text saying it failed. It was worthless.
Haven’t looked to deeply what devops has to offer yet, but things I’d be looking for before migrating.
- I want to be able to develop, run and debug most, or all, of the steps in the graph on my development machine without roundtripping through long chains of triggers and pushing to code repos and pull requests
- I want to run as much as possible in parallel, preferably agnostic of wether it’s on the same machine or several
- I need to build and assemble many components from same repository (monorepo setup)
- only impacted components should build
- Sub-components may depend on different build tools (service fabric services with both webpack and msbuild based packages)
- Some steps may depend on configurations and secrets with a different lifecycle than the code branch being built
I imagine something clever could be built on docker to support both running things locally as well as customize build nodes.
I don’t care to much for scripting in yaml. In fact I spent the day converting some yaml (and cake) scripting to Invoke-Build. Invoke-Build is quite nice actually, the only thing I miss is the option to orchestrate tasks in parallel onto several build nodes. That plus something like ncrunch build/test engine for csharp projects
(really the entire vs/msbuild things needs to be replace with something truly distributed, incremental and repeatable)
Builds are 100% stateless within the CI cluster and rely on tracking/publishing just about anything you can imagine or plug as a resource; out-of-the-box inclusions being docker images (registry), S3 (e.g. minio, AWS) and git.
It's a bit strange to reason about in the beginning but I have yet to try anything that comes remotely close to Concourse when it comes to orchestrating across multiple sources of input/output and non-linear pipelines.
Thanks!
My experience with using DevOps nee VSTS was mostly satisfying. For various reasons unrelated to the post, my team ended up installing the build agent on our own bare metal hardware, and other than needing to restart the agent, I didn't have any mission critical issues. Totally agree with all the UI complaints.
IMHO, Microsoft has been getting a lot of things right lately. Keep up the good work.
In this situation I feel like the product has already wasted my time and now people are contacting me to waste more of my time.
The blog is really detailed and includes screenshots. You could fix the obvious issues and then follow up with an update once it's released.
That said, they generously reached out to me and appear to genuinely be concerned by my experience. I plan to keep posting as I run into issues and also as they correct my mistakes and also solve my problems.
Microsoft also has operating expenses. They need to bring home some income to keep the lights on.
Any big reasons DevOps is something to consider, or is it mostly ideal for new projects?
We tried to also use Azure Boards (Work Items, Boards, Backlogs, etc). Ouch. It is a complete UI mess of disjointed ideas. Instead of implementing one thing well, they implemented two dozen things terribly.
We actually migrated from Azure Boards to Office 365's Planner (KanBan) just because the UI wasn't so horrible. It is simple and gets out of your way. In fact O365's Planner, Teams, and OneNote Online products might be superior to anything Azure DevOps has produced.
The only nice thing I have to say about Azure DevOps Boards is that it integrates into Visual Studio, but frankly even that is barely passable. I'm predicting they scrap DevOps completely and just use GitHub as a clean slate.
May want to checkout GitLab as an option as well.
I have seen a lot of these same problems noted in this link as well, just didn't see to prevent from getting our work done as much. So YMMV depending on your product/devops use case.
Personally have been using their APIs with some custom CLI scripts to get around some of these issues.
Since this morning, we are stuck with an error message "TF400893: Unable to contact the server. This is most likely caused by a network error. Please check your connection and try again.". This has cause a couple of release pipelines to be get stuck, with cancel option not working either. The paid "parallel jobs" are being consume in this stuck pipeline and no other project from my organisation can be built or deployed because of it.
Pretty much regretting having spent so much time configuring the pipelines. We might switch it doesn't get resolved soon or if it happens even one more time.
The New UI is a clear step backward...and the Github integration leaves a lot to be desired. If you have Github Enterprise...why use Azure Repos...why are MSFT teams moving from Azure Repos to Github...and others moving from Github to Azure Repos. The SCM platform and strategy obviously hasn't settled yet...so Gitlab is just as viable an offering that's growing more compelling by the day.
The Azure DevOps hosted build system hasn't been viable in my opinion since it launched, so on-prem agents are a much better option. Take a look at CloudBees offerings, they set the poll position.
The key thing that continues to be a pain point is the reporting and project organization data structure limitations with Azure DevOps: is it 1 TPC per business unit, 1 TPC for the whole company, 1 team project for the whole company. What happens when you have 2...how do you consolidate...what's the path forward if you have to move things around. The answer for the last 8 years has been silence.
The request for a clean solution has been asked for since 2011. Yes 2011, look at the link below. 8 long years...and basically silence...and "it's a hard problem"...and multiple updates saying "We will provide an update once we start planning for the second half of 20XX."
https://visualstudio.uservoice.com/forums/330519-azure-devop...
Eventually, the promises just ring hollow, the proof is in shipped code, and in the meantime other platforms and competitors stacks keep building out better enterprise stories for migrating from Azure DevOps (here's to look at Github, Gitlab, Atlassian's platforms, Jenkins/CircleCI, TeamCity, Artifactory, Xebia Labs etc).
Honestly, the world is a MUCH more competitive place with lots of best in class point solutions that aren't that hard to stitch together and get better performance/features.
Hope springs eternal...that with the new vertical offerings the DevOps team can deliver rapid value...but 8 years is a long time to only hear crickets.
If you're starting out and you need a one-size fits all solution and you're a SMB company with two doze devs...go hog wild with Azure DevOps...that's not to say it doesn't work for large companies like Shell and Microsoft itself...but talk with teams about the limitations they run into and make your own decision on your DevOps transformation adventure...oh and never create more than 1 team project in Azure Devops...seriously...if you do, game over.
Sounds a lot like some process on the build server is leaking memory. We have a couple of older services that have this problem, and if we don't catch it early enough the server will become unresponsive and unable to even handle ssh connections.
You save money but what kind of life is that?
On Windows.. I get a US only bare bone Dvorak layout and few ways to remap the caps/ctrl keys. Messing with the registry is at best annoying on my gaming (only) PC but impossible on a corporate locked down desktop. Then I'm lucky if the corporate package repo comes with AutoHotkey that can remap keys in a very hacky way.
A lot of Windows 98 still shines through in the basic usability suite even on the newest versions of Windows. Even the on-screen keyboard on a (Android) phone is more fun than the HID mess in Windows.
As I'm not typing them frequently I'm happy with a Compose key binded to Scroll Lock.
This would be an upgrade as at least I would kinda trust that a file I found on my own system from the vendor can probably be trusted. I trust the options would be at least mostly self documenting (more self documenting than a random list of hex digits). And, if I'm going to use software written in the 90s, I'm going to go with good old vim.
Additionally, we are a non-profit, and MS works with us in a moderately generous way with pricing, whereas AWS wouldn't.
AWS slowly crept in because there was no entry barrier, no magic Harry Potter spells to get anything working and it actually worked and stayed working.
Now I went back to azure to see what has changed Q3 last year and it was the same “eating sawdust” experience. Literally everything is clunky.
When it comes to these providers, time is money, possibly more so than worrying about the instances. Azure costs a lot more in human attention.
AWS is not perfect though. CloudFormation quite frankly sucks balls but there’s enough interest and so little friction that there are lots of third party things that work well with it.
Azure, not so much. It’s a leper colony.
2. I think Azure is better overall than AWS. It's way easier to grasp the different features, while I haven't used Azure Devops, I've had little issues with the rest.
Even if I run on some other cloud than Azure, I still need the software. I could even run Azure DevOps on Amazon’s cloud I guess (If I choose the self hosted option) but the UX complaints would apply there as well...
[1] https://docs.microsoft.com/azure/devops/pipelines/process/co...
[2] https://docs.microsoft.com/azure/devops/pipelines/process/se...