Ask HN: What technologies did you learn in 2018?
What technologies have you invested time in learning that you are going to keep using/experimenting/learning about in the next year?
I'm a professional Android engineer, and as for me: I learned Kotlin and use it at work every day, and so far I like it a lot. I also learned a little bit of iOS development with Swift. I also experimented with Flutter developing non-trivial "Hello world" apps, and I feel that it reduces a lot of friction that a lot of native mobile developers encounter.
What's your favorite tech in 2018?
619 comments
[ 4.5 ms ] story [ 249 ms ] threadI discovered that git had bash installed quite early on, so my initial automation and reporting was based on UNIX pipelines. As the data volumes became bigger and the customer requirements became more demanding, I eventually bit the bullet and started learning perl one-liners to do some gymnastics that just weren't possible or practical with just bash (first big win was adding the MD5 of the line so that I could dedupe the obfuscated data).
This lead to eventually writing full scripts in perl and now I have basically all of my reporting automated, even able to take up some 'value add' reporting that I didn't think was going to be possible.
So yeah, not a new technology but I'm now a perl convert and will almost certainly use it in the future if I'm faced with a similar situation.
[0]https://perlmonks.org/
And I'm not a new, disgruntled user. I like the site a lot and am quite active on the IT Security stackexchange site. From my semi-moderator's point of view, the only place for discussion is the chat function (which only a few regulars use).
I'm not quite sure what is going to fill this gap. I don't always have a place to post things. Blogging doesn't exactly work without an audience. The mainstream things these days appear to be just chats. There are still forums, mailing lists... heck, there are BBSes but it's niche and hard to find anything that isn't already on the decline.
It was a nice forum and I really wanted it to succeed and but it seems to have faded out.
Reddit?
The second and more pivotal one liner was when I needed to create a primary key to ensure I wasn't getting any dupes later down the pipeline (a requirement that came about as some of the other contractors on the project were messing with my exports and then blaming my files for their conflicting numbers). My first approach was to use the MD5 tool included in GIT bash, but this was prohibitively slow, taking 45 mins to loop through each line of a 4mb file. Not sure if there was a performance penalty on the box I was using, but the perl one liner did the job in a couple of seconds for all the files I had.
After that it mostly became a question of maintainability, with the performance being a boosting factor. I found my perl deduping one liner was faster than either 'sort -u' or 'sort | uniq'. I was able to add new features and account for weird issues more easily. Regex was better than sed, syntax was more flexible than awk, there were more filtering options than grep.
Plus it was good fun. Perl has so many things that just feel so nice and tidy compared to bash.
Having said that, I still use Unix pipelines extensively, they just tend to be for ad hoc queries rather than for my automated reports.
There's an old quip that "perl is portable sed" - in my experience this is pretty true.
Where are y'all based? I'm reasonably happy with my current job, but I'm definitely curious about what else is out there (especially if it's not in the Bay Area).
https://careers-interactivebrokers.icims.com/jobs/1777/progr...
* Perl is set up for really terse one-liners. You can basically emulate or surpass most Unix tools with only a cursory knowledge of the syntax
* Perl syntax is highly geared towards text processing and regular expressions. File handling and regexes are so much quicker and easier to use than Python.
If I were in a situation where I had a bunch of files I needed to process and the task didn't easily fall into something that could be done with a bash command, I'd probably reach for perl over python for a quick and dirty disposable script.
This definitely interests me. How does perl surpass Unix tools? I guess perl has the benefit of handling the equivalent of sed and awk.
Sed:
perl -pe 's/foo/bar/' file1
Grep:
perl -pe '/foo/' file1
uniq:
perl -ne 'print unless $seen{$_}++' file1
But then you can combine conditions:
perl -ne 's/foo/bar/ if /baz/ && /yot/; print "Line number $. changed to $_"' file1
And do some awk-ish line splitting:
perl -F, -ane '$sum += $F[2]; END{ print "The sum of column 3 is $sum" }' file1
It absolutely flies in the face of the unix philosophy and is much more complicated than picking up each tool individually, but once you get to grips with 'em they're very handy.
https://unix.stackexchange.com/a/79656
perl -e 'for(<*>){((stat)[9]<(unlink))}'
seems to be the fastest possible way to delete enormous amounts of files
One may also want to learn Perl5 for dealing with legacy stuff (either keeping or getting rid of it). There is a lot of it out there.
A lot of forced indentation and absence of braces, and absence of sigils makes it harder to read Python scripts. However that's not the only problem with Python. If you are scripting you want the regular `unix_command` command thingy to work fine. More-so you want to read the output of that thing using a syntax like my @command_output = `unix_command`;
You also want heredocs. And you want multiline anonymous functions. I could go on and on.
But Python seems to be more like a tool trying to compete with Java than Perl.
http://www.smashcompany.com/technology/docker-is-a-dangerous...
Unfortunately I never finished getting everything I was aiming to done in that repo, but may come back to it and update it at some point...
Disclaimer: I used to work at HashiCorp on both Terraform and Packer.
[1]: https://www.youtube.com/watch?v=8ZRa0lLq8OU [2]: https://github.com/jen20/hashidays-nyc
(edited to change "best-practice" to "good-practice", since it's more accurate...)
with habitat you now have the reproducibility and isolation of containers without being forced to use them. (also supervising and config management)
* Python * Common Lisp
For Work:
* Pandas (Learning Python resulted in me being asked to help our 'data team' get started with Pandas)
* Go (I moved to that Team)
* Protobuf (google protocol buffers)
I've played around with some other technologies along the way as I enjoy learning various languages. But I wouldn't say I really become good with them, some other's that I've played with this year:
* OpenDylan
* Erlang
* Django
EDIT: Formatting
My favorite tech in 2018 has been Parcel bundler, though.
On my own time I've been playing with Golang, Bigquery and Image Stacking.
It has probably replaced Java as my go-to language. Which is amazing. I thought I was a die hard Java fan.
https://github.com/dotnet/cli/issues/3093
OData - A standard for rest APIs.
Did some playing with .Net Core, but still not really satisfied that it's ready for serious prime time.
For example, if you use core, you can't use entity framework 6, you have to use EF core, which has most features and if you are building a new app you'll probably find a way to do most things you want, but if you are porting something you might struggle. And that means you can't use anything that is built on EF 6 either. The dependacy chains seem to be muddled. One library will seem to be fine on core, but one of it's dependencies will only work on the full .net fx.
Another problem I hit was with doing some dynamic code generation. Roslyn is supported, but code dom isn't. That would be fine if I was starting from scratch, but I had a bunch of useful existing that that all depended on code dom that would have been a lot of extra effort if I couldn't reuse.
I would say if you are starting a new project and it's a fairly standard website you'll almost certainly be fine. But be ready for a lot of pain if you have any code to port.
Dev: Continued working on our full stack JS (Node, React, React native, Flowtype, Jest). I am planning on moving to Typescript because of the poor developer experience of flow, but appart from that, I am loving the state of JS at the moment!
Databases: AWS Aurora with PostgreSQL and PostGIS, with Prisma to create a GraphQL API. It is amazing and feels like the future.
CI/CD: I am getting deeper in the GitLab stack to integrate all the elements above.
In 2018 I went from "zero to sixty", so to speak. At the beginning of the year I knew nothing about ML, and my math skills had gotten so rusty I couldn't even remember how to do simple derivatives or multiply two matrices.
In the first half of 2018 I relearned the college math that I had forgotten by working through MIT OCW's Linear Algebra, Calculus, and Probability courses.
In the Fall, I enrolled in Carnegie Mellon's 10-601 course, graduate "Introduction to Machine Learning". It was hard, it was a ton of work, but I did well and it was completely worth it as I now have a solid understanding of the basics of the field and can continue studying this upcoming year and into the future.
This kind of courses from UPenn are really good if you need to refresh single variable, though I did an older version were all the courses were given together than I can't find in the current Coursera: https://www.coursera.org/learn/single-variable-calculus
There are several CMU courses, however, where the full collection of lectures are available to watch on YouTube. The Fall 2017 offering of Mathematics for Machine Learning lectures are all on YouTube.
But I motivate myself by focusing on the long-term impact for my career, and thus the benefits for myself and my family:
Coupling my expertise in software engineering with deep knowledge in an adjacent field such as ML I believe can open up new opportunities and give me the freedom to take my career along paths that weren't possible before. That is what I remind myself when I sit down to work homework problems late in the evening.
> college math
Those should be high school on most places
I was never taught how to do either of those things. Am I missing out on something?
If you want to do coding in general, I like to say that you can have a career in code without being good at math. But, math is how great coders do what seems like magic to other coders.
As a grown up who has already been working as a software developer for ~10 years, do you have any recommendations on books to read, courses to do etc if I want to improve my maths? I’m not terrible, but I know I could be a lot better!
1. No bullshit guide to Maths and Physics [1]
2. No bullshit guide to Linear Algebra [1]
3. A programmer’s introduction to mathematics [2]
I haven’t gone through the books completely but I’m halfway through the ‘no bullshit..’ books and I liked them based on how easy they were for me to approach.
May be they also fit your learning style as well? Hope this helps and happy learning.
[1] : https://minireference.com
[2] : https://www.amazon.com/dp/1727125452
- I know it's said over and over again but KhanAcademy is pretty good for high-school math - For probability, I'm also going over this website[1] at the moment. Maybe someone else has other recommendations too - Linear Algebra is next on my list but others have given some reference for it already - Andrew Ng's course on Machine Learning from Coursera is also another frequently recommended if you are trying to learn math for it
[1] https://www.probabilitycourse.com/
[0] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Except for Tasks; the fact that such a bare-bones language has first-class support for concurrent execution makes it clear how ahead of its time it was.
Another technology was optics and specifically fiber optics. I went into the field with knowledge of microwave engineering and I am surprised by how similar it is. I thought it would be a completely foreign, but the high level concepts are actually relatable to microwave tech.
AWS - At the beginning of the year I knew nothing about AWS. I thought it was just a way to host VMs. Since then I’ve done projects involving networking, Devops, and development and I have four certifications. My company paid for them and it forced me to learn AWS inside and out except for the Big Data, IOT, and mobile.
Linux - I’ve been developing on and deploying to Windows servers for 20 years. But the Windows tax becomes real once you start using AWS. I’ve learned just enough to be almost competent.
I personally have 3 AWS certifications (All of the associate level).
Their use to me was 1) When studying for them, I learned a ton more about the platform and 2) Recruiters and employers love it - I get even more job/interview offers now.
Yes since you have practical experience it can’t hurt. You’ll learn things that you didn’t already know and it will help you get your foot in the door slightly.
But, don’t become a “paper tiger” with a lot of certs and no practical experience. I’ve also come across people who have claimed that “they have use AWS” but when you dig deep they mean - they’ve hosted a few VMs and used AWS as nothing more than overpriced colo.
I don’t pay much credence on certification as proving competence. I take them as a slightly negative signal when I am interviewing someone.
Certifications for me are partially forcing functions to make me get a broad overview of a subject matter. At the level I’m at now, my “whiteboard interviews” are not about leetCode they are about how would I architect systems, meaning I don’t need to know the details of everything but I do need to know how to talk the talk.
When it’s actually time to implement the solutions, I’m either then going through the SDK and figuring out the details or assisting other developers in coming up with a proof of concept.
Except for the Architect Associate which I studied for just to get an overview of AWS, I’ve had practical experience on most of the other areas that the certifications covered before I took them. The other three are the developer, Devops and Architect Pro.
My company is paying for them, and even though my official title is “Senior Developer”, when talking to our clients (B2B) I am the “certified AWS infrastructure architect”, so yeah it helps give customers a little bit of confidence.
On the other hand, whenever the day comes that I decide to make my next career move, it will probably be as an overpriced “implementation consultant”. I found at my last job where I was the dev lead, that locally, that’s where the money is.
Amazon Certified Partners have to have a certain number of certified employees.
Two of the three certifications I am working on next year will take me way out of my comfort zone - advanced networking and Big Data.
I will have to really study for those two and do some side projects. Unlike the first four where I could learn on the job.
And looking forward to migrate to Ionic 4 and make both desktop and web versions of my apps!