HN Explorer

Ask HN: Flexible, lightweight build automation/CI tool?

50 points by jakobegger ↗ HN
I‘ve recently set up Jenkins to continuously build and test a Mac app (Postico), and it‘s not been a pleasant experience. Everything is poorly documented, you constantly need to switch between different paradigms (point and click interface, procedural pipelines, declarative pipelines), the API is atrocious, and I haven‘t figured out yet how to configure the dashboard.

So I‘ve started to look for more lightweight and flexible alternatives. My build process consists of a couple of shell scripts (bash), some ruby scripts, and even a python script. The individual parts work well, I only need something to tie it all together and display status.

It can‘t run on docker because the app is built with Xcode, but I do use docker to start up test servers that our unit tests run against.

I‘ve been looking at buildbot, and it looks like it does what I want. Does anybody have any experience with it?

I‘ve also considered using a general purpose task scheduling system, but I don‘t know anything about that space. Does anybody have good recommendations?

43 comments

[ 4.7 ms ] story [ 102 ms ] thread
baronofcheese
You don't mention whether this needs to be on prem or if it is OK if it runs somewhere else. Visual Studio Team Services work great for me. They also have hosted Mac instances that you can run on. It is very flexible. You can also install agents on your own machines to run on instead.

I've also used Bitrise in the past, it also has Mac OS and Linux machines where you can run your builds on.

taspeotis
+1 VSTS. You also get 240 minutes of build agent time free per month. Their hosted Mac agents are in preview right now.
vletrmx
Check out http://git.liw.fi/ick2/
dozzie
From their very own README:

> Ick2 will become a CI system. It is not there yet. It is not even ALPHA level yet.

It doesn't even have a website, just a repository browser.

jitl
This website lacks all information about the project. What does it look like? How to use it? Googling returns only irrelevant results. No findable docs.

This looks like a toy or side project.

mkesper
You don't need to click to configure jenkins jobs, they're yaml files containing "builder" steps.
Androider
Jenkins setups invariably accrue a bouillabaisse of plugins and stews in a "just so" environmental soup, that simply moves the "works on my machine" syndrome to "works on our Jenkins (please, please don't break or we'll be down for a week reverse engineering this Ubuntu/RHEL snowflake from 2012)".
maxxxxx
I have heard the best way to use Jenkins (or another CI server) is to avoid any plugins and just have it kick off a few shell scripts. Seems also easier to test the different steps that way.
ctpide
I’ve been using Concourse CI and it’s fairly light weight and has great declarative pipelines. Learning curve is not too steep and it has a command-line tool to update your pipelines. Maybe it’s a stupid suggestion, but you could connect the worker in that setup through ssh to a Mac for building, but of course that takes away some of the comfort, but you will have to run a dedicated build machine anyway.

Also, doesn’t Travis CI have macOS support ?

mmanulis
+1 to Concourse. Though if the OP needs OS X build support, there is no escaping needing a Mac box to run it on.

One can still run Concourse on a Mac Mini if one chooses, not that much extra work. This is the simplest solution but you will still need XCode Server last time I checked (been a while). I'm sure there are ways to hack around it, just a matter of time vs money question.

Both Travis CI and Circle CI offer Mac builds, so if one is willing to spend some money, that's an option.

Having had to run my own CI infrastructure, I'd rather pay someone else to deal with it. Especially if your deploys are tied to it, as once it breaks, and it will do it, your entire pipeline is down.

I ended up spending a good chunk of my own time dealing with failing builds or having a couple of people on my team be the point-persons for CI/CD infra.

jakobegger
I need to build and run tests on macOS, so I‘d need to circumvent the core principle of Concourse (containerized builds). I really don‘t want to work against my CI system...
rc_kas
bash scripts are the most light and most flexible :D
ctrlc-root
I know you said lightweight, but have you given TeamCity (https://www.jetbrains.com/teamcity/) a look? It's much easier to configure than Jenkins in my opinion and very powerful. I've used it at my last two jobs and so far there's nothing it hasn't been able to handle, including some complicated inter-project dependencies, and it's been relatively easy to set up and use.
jakobegger
Their screenshots scared me away initially — it looked too similar to Jenkins.
bluehatbrit
Circle-CI (https://circleci.com/) have a macos system for building things like xcode projects. I've not tried it myself but it's cloud based which saves self hosting. Unless you want to self-host in which case this may not be a great option.
Androider
CircleCI 2.0 has full support for macOS, sounds like a perfect fit for you. It's also inexpensive (if you don't consider your time worthless, since it's $39/mo, which is ridiculously cheap compared to time we spent babysitting Jenkins previously), and just one of the best CI systems there exists in my experience.

Just a happy customer.

circular_logic
Circle might be nice for a small project, but I would advise away from Circle Ci for larger projects. Their feature set is immature if you want to do anything that requires a moderately complex workflow or need triggered deploys. Your builds are in a queue that can take ages sometimes. Their UI is pretty poor. For example, branches show up forever on a project even if you delete them from the repo. You actually have to ask their support to remove them.
busterarm
Circle CI 2.0 and their triggered deploys work pretty well, actually. We've been using 1.0 in production for over 2 years and we're about 3 months into using 2.0 and it's working out great.

Our builds are pretty complex and we have tear down scripts that we always want to run even if a build fails and CircleCI does that (on 2.0, finally).

The only thing I wouldn't advise CircleCI for is if you want to distribute your tests across many machines -- and only because it's very expensive!

perlgeek
Not the most light weight, but consistent and cross-platform: https://gocd.org/

It has pipelines as a first class citizen. The config is rather concise XML, and can be hand-written or generated/modified through a web UI. There is also the option to store pipeline config as XML/YAML/JSON in a git repo.

cygned
Did they add key based authentication with SSH yet? That is basically the missing feature that renders it unusable for me.
perlgeek
You can give the go system user an ssh key, and then you can use ssh git URLs.

Is that what you want?

jakobegger
Thanks! This looks interesting, especially since I can host it locally, and the fan out/fan in stuff looks just like what I want for testing across multiple configurations.
SeanA208
Buildkite (https://buildkite.com/) is so underrated and really nice! Much simpler interface than most other providers like Circle CI or TeamCity and works really well.
jakobegger
Thanks, this looks interesting, since it lets me build on my own machines (I really need to control the environment, eg. build and test on specific versions of macOS)
exidy
Buildkite is my go-to CI in my professional work as well. The hybrid model (the master is SaaS, but you bring your own agents) works so well both from a cost-efficiency perspective, and also when working with enterprises who don't like builds occurring on machines they don't control.
atsaloli
GitLab CI is lightweight compared to Jenkins. I recently taught CI/CD using both, on small instances, and the instances running Jenkins would occasionally grind to a halt due to load while the ones running GitLab kept ticking. Same CI/CD load, same instances. Just a different CI app.
assafmo
Seconded. I'm using gitlab-ci offline (bitnami instance + runner) and on the gitlab website and it's awesome.

Also travisci with github is good but I don't know if it supports mac.

jakobegger
It looks like Gitlab CI requires the pipelines to be inside your repo. I‘d like to keep them in a separate repo, otherwise making changes eg. to a test server configuration requires merging the new config into every branch.
jeremija
You can define build (secret) variables per project, which means you can change the configuration without committing every branch.
vorpalhex
I use some Jenkins and some Travis-CI(https://travis-ci.org/) with good luck. Travis seems capable of most things that Jenkins can do, while still being a pleasure to configure and work with.
jakobegger
I‘ve tried Travis once, but the build didn‘t work and I just had no idea how to debug it. Also, the build environment I wanted wasn‘t available, so I really want to go with something that allows me to use my own machines for building/testing.
alexjarvis
bitrise (https://www.bitrise.io) is the best that I've used so far for building iOS apps. Its elite package is also really fast if you can afford it. I've used travis (slow, 45 min build times) and gitlab-ci with a dedicated mac pro runner and bitrise wins hands down for speed and simplicity. Also don't forget fastlane (https://fastlane.tools). Most of my build configuration is in fastlane (for iOS and Android) and bitrise just triggers fastlane via the script step. Good luck!
jakobegger
Doesn‘t look like bitrise supports macOS?
robertop
I have used buildbot for about 5 years on a cross-platform (mac, linux, windows) project. It is definitely lightweight and flexible.

Pros: - The schedulers and pollers is very stable. You don't really have to worry about uptime. - Good enough documentation.

Cons: - Configuration is done in python. If you don't know python well then be prepared spend time to learn it.

The biggest pain point, at least for me, has been that buildbot has no support for caching of dependencies. If your project has many dependencies you will spend quite a bit of time either installing dependencies on the CI builders or having long build times because every build will download dependencies. This is not really a buildbot issue itself, but it will definitely bug you if you have used hosted CI products like TravisCI or Circle CI.

Rondom
My usage of buildbot is only very anecdotal (small hobby project). I consider it very easy to get started with. Through its use of Python it allows for a lot of flexibility, but I don't know how well that scales to very big complicated organisations. But then again, Jenkins is not easy to work with, either and you might not really need the complexity and feature-set that Jenkins provides.
bm1362
Drone is an open source CI/CD alternative.