This article turns into the creation of a small web app, so while not exactly relevant, I'll ask here: are there any good form validation libraries for Go?
I thought about how you could design a library like this in Go, but couldn't think of a way that did not end up using reflection or plain maps, which defeats the purpose of using static typing.
Is this lack of validation libraries due to the lack of people writing web apps with the language or is it a problem of expressiveness with the language itself?
I ran into this issue several times when trying to write a web app in Go. A lot of the basic infrastructure is there, but the niceties are missing. The template language, for example, is great but lacks something as basic as inheritance without kludges. There are other templating languages, but they don't feel quite there. Got the same feel from the ORMs. Hats off to those who are doing work in it in spite of this, but it seems there's a way to go before the language is really usable for web apps.
schema is not a validation library, it just turns HTTP forms into Go structs. Validate uses tags or maps (not a huge issue, granted). The Revel one does look nice, though. Thanks.
>couldn't think of a way that did not end up using reflection
The builtin decoders (eg, json) use reflection. You don't lose any of the benefits of static typing as you are still decoding into your statically-typed object.
I would separate form parsing from object validation.
Use something like this: https://github.com/ajg/form to populate statically-typed go structs from form data, and then validate your objects.
There is a speed hit to reflection, though, right?
Also, you do lose the benefit of static typing, for example, the following typo isn't picked up at compile time:
type User struct {
Name string `form:"nam"`
}
Tags are rather ugly and don't scale to many options well. What happens when your ORM, form validation library, xml library, and json libraries each need a separate tag? That's a slight exaggeration but you could see how the apparent simplicity in the form example can quickly become anything but.
Heavy use of tags (and reflection) just makes me feel like you have to work around the language too much and I question whether you may as well just use Python, hence my original question.
I've done minimal work with Go, though, so perhaps I'm misinformed.
There is a speed hit to reflection. As for the tag issue: writing your own marshaler isn't actually a huge problem. I have been meaning to write my own json marshaler for a while. I personally find the solution much more paletable than many other ways of marshaling in and out of json.
I've attacked this from a few angles myself—previously using reflection and struct tags - `min:"2", max:"50"` - but found it fairly inelegant (although practical). I wrote some good tests around it, that's for sure.
Now I'm a fan of https://github.com/katco-/vala which has a nice API for doing this for you. I've ripped out one of the examples from the README (which I think illustrates it well), but you could also wrap this in a method that you call on the instance of the struct you get back from (i.e.) gorilla/schema:
func ClearValidation(a, b, c MyType) (err error) {
err = BeginValidation().Validate(
IsNotNil(a, "a"),
IsNotNil(b, "b"),
IsNotNil(c, "c"),
).CheckAndPanic().Validate( // Panic will occur here if a, b, or c are nil.
HasLen(a.Items, 50, "a.Items"),
GreaterThan(b.UserCount, 0, "b.UserCount"),
Equals(c.Name, "Vala", "c.name"),
Not(Equals(c.FriendlyName, "Foo", "c.FriendlyName")),
).Check()
if err != nil {
return err
}
// ...
}
Note that for any reasonable amount of validation w/ structs (i.e. not nil, ranging over it automatically, etc) you will need reflection. There are no ifs or buts about that. But the whole 'reflection is slow' mantra is premature optimisation: if you need reflection, you need it. Your template rendering uses reflection, your database connection introduces latency, etc - all of that dwarfs any cost associated with validating a struct or two.
I like the idea of this series, because I think it's more useful than Hello World type examples. There's an implicit trust that whoever is writing the article knows what they're doing, which could be a problem for someone new to a community. I skimmed over the Ruby one (I know Ruby better than Go), and it seemed pretty solid, and was written by a Rails core contributor, so it should be. Hopefully they'll get more of these out.
The whole point of this series is to pick people who are knowledgeable in their respective language. Jose wrote Elixir and Fred is a known personality in the Erlang community.
It is so much more powerful when it is people with knowledge who writes these!
Why do it in a Vagrant? If you download the go tarball from golang.org, it will work just fine with no dependencies on any desktop linux. You can have multiple versions, and switch between them by changing your GOROOT.
Don't use println(), use fmt.Println() or other equivalent functions instead.
Current implementations provide several built-in functions
useful during bootstrapping. These functions are documented
for completeness but are not guaranteed to stay in the
language.
Don't decode into maps. If you don't want to create a struct, use an anonymous struct.
The only case I can think of making sense is when you deal with JSON that can change shape under your feet (like when dealing with shitty APIs that sometimes returns {}, sometimes []).
It's actually annoying Go is shown as a web language in many tutorials. It's simply not. If you use Go for systems programming and other stuff like creating daemons, services, you will fall in love with it more and will not go back to any other language. That's why I believe those Go start tutorials should not be showing the net/http side of things.
Go definitely shines as a server language, so I think some kind of server is indeed the best way to illustrate Go's capabilities. It's true that net/http is kind of... boring, and maybe not the best choice for a beginner tutorial due to some of its quirks, but it has the benefit of being a well-understood reference point.
In reality, this tutorial is about making the API calls and decoding their JSON responses, and then introducing some concurrency to do multiple calls in parallel. The HTTP portion of this tutorial functions as some familiar ground for newbies, a nice way to get a grasp of the Go style in a context that many people understand.
Nice post there. Easy to understand and it gives us a solid entry point.
I've been building my own backend for a couple of weeks (yes, very slowly) and I'm still wondering how to write proper test. Unit testing is a charm and there is nothing complex there, but mocking my dao package for example is a little harder.
I'd like to see how you write beautiful unit tests (w/ or w/o mock) in Go.
Am I right in my understanding that the end of the last code snippet shows that the app will error out for the user every time any single one of the many APIs used is unavailable, rather than making an average of the remaining services (1 of them)?
Isn't this approach making the service's downtime be the sum of the downtime (minus overlap) from all the APIs being called?
You're quite right. The mutliWeatherProvider should probably collect the errors and only error out if it doesn't get a single good response. Easy to do.
35 comments
[ 4.8 ms ] story [ 72.2 ms ] threadI ended up using this approach:
http://www.alexedwards.net/blog/form-validation-and-processi...
This is straightforward but also manual and low-level.
In comparison is Python's formencode library which is declarative and high-level:
I thought about how you could design a library like this in Go, but couldn't think of a way that did not end up using reflection or plain maps, which defeats the purpose of using static typing.Is this lack of validation libraries due to the lack of people writing web apps with the language or is it a problem of expressiveness with the language itself?
I ran into this issue several times when trying to write a web app in Go. A lot of the basic infrastructure is there, but the niceties are missing. The template language, for example, is great but lacks something as basic as inheritance without kludges. There are other templating languages, but they don't feel quite there. Got the same feel from the ORMs. Hats off to those who are doing work in it in spite of this, but it seems there's a way to go before the language is really usable for web apps.
Here are a couple:
http://www.gorillatoolkit.org/pkg/schema
http://godoc.org/github.com/mccoyst/validate
There are also some bigger web framework projects that include validation, Revel is one: http://revel.github.io/manual/validation.html
The builtin decoders (eg, json) use reflection. You don't lose any of the benefits of static typing as you are still decoding into your statically-typed object.
I would separate form parsing from object validation.
Use something like this: https://github.com/ajg/form to populate statically-typed go structs from form data, and then validate your objects.
Also, you do lose the benefit of static typing, for example, the following typo isn't picked up at compile time:
Tags are rather ugly and don't scale to many options well. What happens when your ORM, form validation library, xml library, and json libraries each need a separate tag? That's a slight exaggeration but you could see how the apparent simplicity in the form example can quickly become anything but.Heavy use of tags (and reflection) just makes me feel like you have to work around the language too much and I question whether you may as well just use Python, hence my original question.
I've done minimal work with Go, though, so perhaps I'm misinformed.
I think the issue is more of a DRY thing, which could maybe be fixed with something like:
This hypothetical syntax would instruct the library to look for the form fields "first-name" and "last-name".Now I'm a fan of https://github.com/katco-/vala which has a nice API for doing this for you. I've ripped out one of the examples from the README (which I think illustrates it well), but you could also wrap this in a method that you call on the instance of the struct you get back from (i.e.) gorilla/schema:
Note that for any reasonable amount of validation w/ structs (i.e. not nil, ranging over it automatically, etc) you will need reflection. There are no ifs or buts about that. But the whole 'reflection is slow' mantra is premature optimisation: if you need reflection, you need it. Your template rendering uses reflection, your database connection introduces latency, etc - all of that dwarfs any cost associated with validating a struct or two.I'm going to do a Rust one around the 1.0 timeline too, hopefully.
It is so much more powerful when it is people with knowledge who writes these!
Maybe just chill your beans and enjoy Peter's writeup.
Plus, it'll be faster.
And for beginners, it could be interesting to know that decoders also support simple maps in addition to tagged struct.
Nice series of articles.
May as well start learning the language with best practices, even if it does involve an extra line to 'import "fmt"'.
The only case I can think of making sense is when you deal with JSON that can change shape under your feet (like when dealing with shitty APIs that sometimes returns {}, sometimes []).
Would you be able to explain why you iterate over the range of providers instead of iterating over the channel in your concurrency example?
Since the channel is created as buffered, you could do a normal for-loop over cap(chan), which would be the same as len(providers).
I've been building my own backend for a couple of weeks (yes, very slowly) and I'm still wondering how to write proper test. Unit testing is a charm and there is nothing complex there, but mocking my dao package for example is a little harder.
I'd like to see how you write beautiful unit tests (w/ or w/o mock) in Go.
https://github.com/andrewgleave/react-weather
It's very simple, but I'd be interested to hear feedback on making it more idiomatic Go.
Isn't this approach making the service's downtime be the sum of the downtime (minus overlap) from all the APIs being called?