According to the OpenNTPD web page, "The portable version is outdated and in need of a maintainer."
Why another from-scratch rewrote, instead of helping with that project, especially when it was also motivated by an aversion to the "100,000 KLOC" in the ntp.org ntpd?
That sounds like you're implying that safety has no value, the only "overhead" worth considering is CPU cycles (what about the programmer's time/effort/sanity?), and that abstractions have negative value.
I don't think that's how you're supposed to software.
but latency, and jitter, are important in this application. Two things that discount most of the "safer" languages. After ignoring inmature ones, the field is pretty barren.
Yes, but if you're building an app for the future, then what excuse is there for not using Rust, which is both very low latency and very close to a 1.0 release?
I get that sometimes you just want to hack in a language that's familiar to you, but then call your project a fun side project and not the "NTPD replacement" (and yes, I know who phk is).
Please start by showing us the PLL in Rust (details are available at phk's blog) and reason about how those changes affects the workings of the program. Is it more readable? Less? More exact? That I would be interested in.
It's a little bit sad that at -3 votes, this is the lowest voted comment I've ever had in my ~4 years on HN, lower than a number of controversial political comments.
To clarify, I'm not claiming that everyone should start using Rust. Now was I claiming that PHK is somehow unqualified for the task, far from it. I'm not even saying C is a bad language (I like C), or that people should stop using C altogether. But if you're building an security-critical application designed to be used in the future across millions of servers, and one that requires low-latency, I simply don't understand why you wouldn't strongly consider Rust, which offers strong guarantees about memory and type safety along with low latency and a very modern and complete standard library. Or Ada. Or OCaml.
Bold assumption. It uses 0.0006% on my slow 6 years old box (over 13 days uptime). Let's be generous and say it's 0.001%, then you get 1.1 ton of CO2, which is the equivalent of the production of about 32.8 Kg of beef. I don't think such dramatic conclusions can be made from those facts.
First, I doubt your kernels statistics actually account correctly at that level of precision. (If it does, that alone would waste a lot of CPU cycles!)
Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles.
And don't forget: That number was one quarters purchase of servers, running for half a year. All the servers bought in 2010, 2011, 2012, 2013 and the other half of the year were not included.
I think you missed something in your calculations.
Removing 0.1% load from the machine will not result in 0.1% of energy savings, very very far from it if in any at all. And that's not even what we want to calculate. We want the difference with hypothetical alternative in a safer language, which is hard to guess, but is unlikely to be in the same order or two with your calculations.
It's a core operating system component not some web app.
Overhead is important since Linux is designed to run on a whole spectrum of devices. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. And frankly nobody could care less about programmer's time/effort/sanity in this case. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
Overhead is important since Linux is designed to run on a whole spectrum of devices.
Like the difference between a ntpd.c vs ntpd.go is going to matter for anyone...
If you really find yourself stuck with an embedded device small enough that a few megabytes of RAM matter then you can still use the old C implementations, it's not like they are going away.
Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead.
In this case abstractions mean you can probably write the whole thing in under 2 kloc, get memory safety for free, and base it on a network stack that other people actually use and debug independently.
It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
Umm. Yea. Right. Tell that to the ntpd guys with their 100kloc codebase...
The jitter introduced by garbage collection in a go implementation could well be significant, even with the new "never pause the mutator for more than 1ms" guarantee.
How much of the code is actually sensitive to interruptions?
Why not write that part in C and leave the rest to the more suitable language?
I would guess that's a rather small portion of the codebase, essentially the callouts to adjtime().
I don't buy into the low-latency FUD. C isn't a magical realtime language either; what happens when the system is under high load? What about delays in the kernel/network stack?
As a layman I would think incoming frames will have to be stamped with a hardware high resolution timestamp at arrival anyway, regardless of the language. And yes, this internal timestamp will have to be compared immediately before setting the hwclock. I would argue this is the only part that really needs to be written in a GC free language.
I might be wrong on this, but I'd like to see a stronger argument than "could well be significant" for writing yet another generation of a baseline system daemon in an unsafe language...
And speaking of latency: If this really is such an issue, why hasn't NTP long been moved into the kernel?
Abstractions are what make it possible to reason about complex systems. Being able to ignore parts of your codebase is the only way to make changes to it once it gets beyond a trivial size.
The whole of the Unix philosophy is enforcing this policy of strong abstractions by making each part of a complex overall system its own binary, such that they can only communicate using the file I/O abstraction, possibly via pipes. C was therefore designed for a system which promotes abstractions.
Oh, I'm still bumming around with a copy of C99. Now that I'm here, I'll add the definition of streams in the previous section is probably the more interesting part, since that's the term used for the idea of abstracting tapes and terminals.
Even if your comment both made sense and was factually accurate, it wouldn't have much relevance to the kind of C programming I'm talking about: For example, mmap(2) isn't defined in the C standard but in POSIX, but it's universal in the Unix and Unix-like OS world to the extent programs can safely rely on it.
Standards are good and useful things, but they're only fully written down in retrospect.
Lifting up constants into defines/consts and DRYing up C code makes for shorter, less error-prone and easier-to-understand codebases. Eg there's no reason to continue writing code like it's 1960's because we can steal good software eng and maintainability lessons from everything that has come since. (If one must maintain absolute portability.)
PS: I would like to see a toolchain for Go that emits portable C instead of statically linking in a giant runtime... Lightweight camping vs. YAGNI hoarding. Seriously, the weight of runtimes would be the biggest barrier to serious systems deployment, outshining the slowdown, productivity and safety tradeoffs. (Eg Go would need to play nice with embedded constraints including memory usage as well.).
That's taking things literally too far. Common sense says take some giant unsigned long long constant used in multiple places that cannot be vertically aligned and give it the semantic meaning it deserves. The goal of reducing LoC is to not obscure meaning but to write less code that states its behavior more clearly. Abstractions are garbage if they are confusing / hide meaning, but abstractions are immensely valuable if they perform complex operations in a clear manner (say an well-designed http client lib in C.)
Rust. Some of it's design goals (as stated on their webpage) are among the exact things you mentioned: zero-cost abstractions, minimal runtime, guaranteed memory safety, threads without data races.
On the other hand, this program is going to take a year or so before anybody seriously considers using it, simply due to the fact that few people are going to do something other than their distro default for such a small part of their system.
The point is simply that if one started a project now with the idea that it would probably only be a serious competitor in a year, Rust will most likely be ready by then - it doesn't really matter if Rust's ready right now if your project's not going to be used by anybody right now.
Zero-cost to whom? The developer? no as now the implementation details have to be known, the machine? no as time will be spent in the compiler churning away removing junk or if the compiler isn't smart enough including it. Also the C++ / rust definition of zero-cost is only if you don't use the abstraction. Minimal runtime is relative.
Can you explain why abstraction is bad? Abstraction does _not_ mean your program will be slower. In fact, the opposite can happen, because the compiler has more context to work with, and can get rid of stuff.
The problem with abstraction is that in order to deterministically control performance, you have to break down all the walls of the abstraction anyway. If I'm not 100% confident that my compiler will optimize a vector op with SIMD, and I absolutely require SIMD performance, then I end up having to know the entire layer cake to be able to verify that it is doing exactly that.
In most cases it isn't a bad thing at all. But a use case like a timekeeping daemon, where a single unexpected garbage collection pause could cause a world of problems, it most definitely is more pain than gain.
Abstractions are certainly not "bad", there are many abstractions in Ntimed, it's also pretty object-oriented if you care to take a look.
This is more a matter of "the right language for the job" and for timing-critical systems programming running as root, that language is C.
And there are plenty of common abstractions I would love see added to the C language: basic linked lists, byte-endianess and packing for struct members, validity intervals for integers and FP variables (like Ada!)
Unfortunately my taste seems to be the direct opposite of ISO-C which have instead wasted time giving us another thread-API.
I'd wager that the lower level nature would be useful for something like accurate timekeeping, which is actually an intricate task if you intend on doing it fully.
The appropriate systems language for use on UNIX and Linux is C. The entire interface is based on C conventions. Sometimes, when all you have is nails, you really ought to reach for a hammer.
I seriously think the C-hater crowd on HN is just disappointed that they can't reason well about pointers. They can't write good C so no one can! It needs, like, more javascript, man.
(Preemptive response to downvoters: please have a sense of humor about yourselves. :P)
You realize that the most vociferous of those decrying this language choice come from the proponents of a language that has 3 different types of pointers, right?
You should check out Rust and Go. Their users are most definitely among those who fail to understand C pointers.
> Go is also a better alternative to C in many instances.
Go isn't in the same ballpark nor do the authors (venerable as they are) seem to have considered that many people need consistent power over their run time execution when microseconds matter. stop the world is not a viable memory management trade off. Also managing memory in c++ these days is pretty "easy".
I've leived for years with people spending hundreds of man hours a year trying to get around issues caused by STW pauses in high throughput java applications. Makes no sense.
Rust seems interesting, though when I read about it I often think "sounds good, but why not just use C or C++?" These are, after all, already working well for a lot of people, and working just about everywhere. And you don't have issues that come with an experimental language: like on day x GC is a good idea, and on day x + 1 reference counted pointers are the way to go.
The benefits range from overstated [bounds checking is a thing, but amongst experienced practitioners using modern, safe styles and conventions, lack of memory safety is not really the huge issue people say it is] to sometimes irrelevant [I read that the concurrency model aims to eliminate the possibility of race conditions entirely - who cares? In many domains races are a fact of life, benign, or totally manageable, so it appears to solve the wrong problem].
As for Go, I do acknowledge the background of its creators [venerable, as another commenter said], but most stuff I see written by "rank and file" members of that community is very clearly written by people who mostly deal in higher-level languages.
Memory safety is a huge problem: every large C++ has vulnerabilities caused by memory safety problems. E.g. web-browsers get pwned because of memory unsafety. Apparently, all (or close to all) critical security bugs in Firefox were problems that Rust compilers will catch. I believe pcwalton went through Mozilla's bugzilla to check this.
In any case, bounds checking is a tiny side street in how Rust ensures memory safety: ownership and lifetimes are the important novel things (lifetimes especially are novel, only research languages have had anything similar until this point, and I don't know if any have had exactly the same system).
In Rust the story is totally settled: in order from most to least preferred/recommended: use references, no pointers, owning pointers and then start thinking about reference counting/other shared ownership.
Rust doesn't solve race conditions: it solves data races, which are also problematic (they cause memory unsafety). It is impossible/very hard to avoid arbitrary race conditions, but data races are the only ones that can cause memory unsafety directly.
> ownership and lifetimes are the important novel things
The first is very clearly a borrowed C++ feature (from RAII and smart pointers). I'm unfamiliar with the 2nd but it sounds like just making the compiler yell at you more when you break scope rules (like C++ will happily let you do with dangling pointers - I will give you that dangling pointers are bad).
> Rust doesn't solve race conditions: it solves data races, which are also problematic (they cause memory unsafety). It is impossible/very hard to avoid arbitrary race conditions, but data races are the only ones that can cause memory unsafety directly.
This sounds like gibberish to me. When I said "race conditions" please read it as "data races". Many of which will not "cause memory unsafety directly". Many of which low-level code such as a kernel will need to not be abstracted from. (Imagine writing a page fault handler in a language that tries to hide data races from you... Doesn't sound like a great idea.)
Even the Linux kernel has very, very few intentional data races. Most of them rely on undefined behavior and Linus has complained before about kernel code breaking when GCC actually tries to exploit that behavior. In those rare cases, it is perfectly reasonable to use unsafe code, along with copious comments explaining why it's safe.
I think you misunderstand me. Data races happen a lot in kernel development, yes Linux too, and you need to handle them and keep behavior sane. Take my example of a page fault handler. You need a strategy to account for the cases where another proc faults on the same page being processed. Or if a page is being unmapped and accessed simultaneously, you may let the access through or you may deliver a signal depending on timing. Handling these is a real task and if you are in lala land pretending data races don't happen, good luck.
This, in my opinion, suggests that what I read about Rust makes it sound unsuitable for certain domains where concurrent writes and data races are an inherent fact of the universe, a reality that the language seems to deny and be working against.
(Didn't even mention lock free algorithms, for example I was reading one day about [I think it was] the dentry cache using lock free algorithms... You can bet there is part of that code that yes, expects data races to happen and handles them.)
Those aren't data races. Page faults are actually synchronized (they block) and the switch to kernel mode is a synchronization point. They are examples of races, but they're not examples of data races.
Just to be completely clear, since we didn't define what we were talking about: a data race exists when there are two or more unsynchronized accesses to the same memory location, at least one of which is a write. What happens when that occurs is undefined behavior in C, whether or not you are writing a kernel. On some hardware the result is totally unusable (the write is corrupted). Additionally, the compiler and (on some architectures) the processor are free to reorder operations such that other threads will see your writes in the wrong order, or not at all.
This is all hardware level, the kernel has to deal with these problems too. The "relaxed" ordering in the C++11 memory model is essentially the lowest guarantee you can do anything useful with (other than have something like a racy counter, where you literally don't care at all what the value is, which is basically the only place the kernel has intentional data races at all). For some architectures and operations it resolves to a noop (relaxed stores and loads on x86, for example, never require additional synchronization beyond what the compiler provides).
The Linux kernel takes advantage of these semantics for lock-free or mostly-lock-free algorithms like RCU. There's a lot of code in those implementations specifically to ensure the correct barriers and orderings are in place so that there can't be data races. There's no reason they couldn't be implemented in Rust, and it would probably be easier since they could rely on LLVM's implementation of fences and so on rather than reimplementing it for each architecture. And if LLVM didn't have what they needed, they could just implement it in inline assembly (which is what kernel developers already do anyway).
My parenthetical remark which started this tangent is that I see people suggesting rust because "it solves race conditions" and when I looked into that claim it looked basically like the compiler enforcing no data sharing. The scheme is unworkable for lots of domains.
If you define data races to not include race conditions that your code is handling (by placement of atomics, fences, locking etc.) then maybe your points make some sense but it is exactly the code handling these that I am saying does not fit with what I have seen rust advocates claim. You have to reason about these things at some level, and what I hear people advocate on hn is to sidestep it all by preventing sharing, which may be a fine idea but doesn't work for everything.
Lastly, in my mind, if your lock free algorithm works by letting the races happen but using the result of an atomic op and/or fences to ensure consistent behavior, I would still call that a (benign) race. That you needed atomics, fences, or compiler hints to get what you want seems orthogonal.
Rust definitely allows data sharing. It just doesn't allow data races.
> If you define data races to not include race conditions that your code is handling (by placement of atomics, fences, locking etc.) then maybe your points make some sense but it is exactly the code handling these that I am saying does not fit with what I have seen rust advocates claim.
Its actual guarantees for atomics are not very strong and let you handle all of those things in safe code. What's interesting is that it can guarantee correct behavior for types that have more interesting semantics than "a bytestring," like mutexes or smart pointers. The implementations of those things are still unsafe, but you can generally provide a safe API to them, which you can't do in most other languages. This is because Rust has the notion of thread safety built into the language and enforced by the type system. For example, Rust's `shared_ptr` equivalent, `Arc<T>` is defined to be thread safe to share and send to other threads if and only if `T` is also thread safe to share and send to other threads. That doesn't mean you can't build up a safe type from the lower level atomics, or Rust isn't suited for this domain. It means you can write safe APIs in Rust that you can't in C++.
> You have to reason about these things at some level, and what I hear people advocate on hn is to sidestep it all by preventing sharing, which may be a fine idea but doesn't work for everything.
Rust just forces you to do just enough synchronization to not have undefined behavior. It's not a dogmatic language and doesn't believe there is one right way to do concurrency. Channels were just removed from the prelude for precisely that reason--they're not particularly favored over other concurrency mechanisms.
> Lastly, in my mind, if your lock free algorithm works by letting the races happen but using the result of an atomic op and/or fences to ensure consistent behavior, I would still call that a (benign) race. That you needed atomics, fences, or compiler hints to get what you want seems orthogonal.
Data races are well-defined, and what you're describing aren't data races, benign or otherwise. You basically never want data races, just like you basically never to dereference a dangling pointer. Rust prevents things you basically never want, it doesn't prevent writing useful things like lock-free data structures.
> Data races are well-defined, and what you're describing aren't data races, benign or otherwise. You basically never want data races, just like you basically never to dereference a dangling pointer
Lock free algorithms, or even a lock implementation, does not jive with this. You let the race happen, and you safely detect when you lost or won the race, and then you do stuff accordingly. To use your pointer analogy, it's OK to have a stale pointer in RAM or in a register at a moment in time. It's a violation of the invariants to dereference it.
As for the rest, perhaps I have been misinformed or led to some outdated info on rust when I saw things about no sharing. I will be sure to take a look.
Lock free algorithms have to be careful to avoid dataraces by using the appropriate atomic instructions on the CPU. Using non-atomic instructions is a data race and may mean you incorrectly detect that you won the race (seeing stale data in a CPU cache or somesuch).
A data race is essentially defined as a race condition between reads and writes where at least one of those is non-atomic. Rust's type system allows one to enforce atomicity by default (e.g. using atomic instructions, or wrapping data in a mutex).
Are you assuming that I haven't written these things in real life or something?
I said "safely" detect, so yes, the final observation that you have won or lost the race must come from an atomic op. However, a "data race" susceptible read can totally be part of the process. The most common idiom I have seen for a lock free atomic read-modify-write has been to do an "unsafe" read, then act like it was OK, then issue compare-and-swap to determine if it really was OK and do the write if successful. (A failed compare and swap means you need to re-fetch and try again.) Yes you need to make sure fences are OK and the compiler is not caching/reordering the read and you need to account for the famous "ABA problem", blah blah blah, but it's what people do and I am not making it up for the purposes of an HN discussion.
The typical RISC approach to atomics, load-link/store-conditional, also encourages what I will call "my kind of thinking" on this issue, though it does so with specialized instructions. You do a special read, then you use ordinary non-atomic register operations, then you do the special store which fails when you lose the race. I will give you that these are specialized instructions and not ordinary C assignments but I would suggest looking into them if you have not already, the semantics are very educational. (And they may just convince you that disallowing data races is an overly restrictive thing to do in some scenarios.)
I was assuming you were talking about the race & detection as e.g.
// let the "race" happen:
old = compare_and_swap(some_shared_memory, 1);
if old == 0 {
// we won!
}
This is perfectly valid data-race-free code if the CAS is atomic, but is invalid if the CAS is not (assuming no other memory fences).
But sure, if you're manually fencing then you can have data races that are benign; however, just because something has some relatively rare uses doesn't mean it's wildly unsafe for the general case, and so disallowing by default it helps the correctness of the vast majority of concurrent code.
In any case, Rust allows one to opt-in to that sort of behaviour using `unsafe` locally, e.g. one would use `unsafe` deep in the internals of the implementation of the lock-free data structure and with careful vetting to ensure it's correct, and then all users can benefit with the compiler ensuring concurrency-safety by default.
Rust tries not to completely disallow behaviour, just make memory safety the default. The programmer can override the compiler via `unsafe` if they truly know better.
Other than the address compare and swap takes two operands: expected value and new value. It does an equality check and (if equal) the swap atomically. Therefore the equality check is handy to atomically check if a previous potentially unsafe read is still valid, and if so, set a new value.
Too late to edit the post but maybe some code sample will help illustrate the pattern. I am typing code into a web form so I make no guarantees that it is perfect.
// Performs some modification on the input.
extern int f(int);
volatile int global_var = /* ... */;
int expected;
int new; // assuming C and not C++, where "new" is reserved
do
{
// Do a speculative read
expected = global_var;
// Make some (non-atomic) modification
new = f(expected);
// loop until the speculative read was OK. (won the race)
} while (compare_and_swap(&global_var, expected, new) != expected);
Whoops, I got the signature of CAS slightly wrong. In any case, I already understood the pattern and understand why it is OK. However, as I said, it is the exception that a data race is OK, which is why Rust disallows them by default (it does not disallow them completely).
This is an accurate parody of my opinion. I'm unable to say "good C" with a straight face.
> I seriously think the C-hater crowd on HN is just disappointed that they can't reason well about pointers.
Let's be (hopefully) pessimistic and assume I'm a bottom 10th percentile programmer, and assume (I believe optimistically) that every programmer half an iota better than me is physically incapable of writing a single solitary buffer overflow, double free, use-after-free, or other pointer usage related bug. Perhaps they are robots!
I'm of the opinion that the remaining bottom 10th percentile remains a significant problem and security hazard. My continued employment with C++ seems to indicate we lack either the technology to detect, or the will to fire, this bottom 10th percentile. I feel that a hatred of C is a pragmatic step in helping reduce this problem, by reducing the usage of C by my fellow incompetents, by reducing the usage of C in general. Do you disagree? (I extend this stance to C++, despite it being my day job.)
Continuing the assumption - I'm disappointed that I can't reason well about pointers. I'm a bit more bummed about being in the bottom 10th percentile. I'm significantly more distressed that people can't reason well about people who can reason well about pointers - that, by virtue of being human, even they will make mistakes.
Being slightly less pessimistic: I'm fucking awesome. C isn't. What Dunning–Kruger effect?
I take slight issue with talking too cavalierly about ranking programmers [what are we, the HR department at a big company? ;-)], but let's reverse it and say the top 10th percentile of people working in C write very few overflows and pointer bugs (but never zero, since we're all human). Should you therefore mistrust by default the software written by those folks, due to being burned by output from the lower 10th working in the same language? And all this despite the fact that we have some mitigations these days, such as address randomization, fewer pages being executable, privilege separation, safer coding styles and practices being pretty well known in some circles, etc.?
And where would that lower 90th be if it weren't for the fact that these days people are primarily being taught in memory safe languages? A key difference here, and one of the reasons I would resist percentile ranking for this discussion, is simple exposure and practice.
> I take slight issue with talking too cavalierly about ranking programmers
Reasonable. Programming skill is not a 1 dimensional trait... and it's a bit bloody minded, even if it were. I threw myself in the fire with the hypotheticals because it made me uncomfortable, even as a hypothetical for the sake of argument, to be doing such a reduction.
> Should you therefore mistrust by default the software written by those folks, due to being burned by output from the lower 10th working in the same language?
Sure, absolutely. This manifests as a willingness to review their commits for mistakes, and to support their efforts in trying to reduce the chances of making certain types of mistakes, and an insistence they do the same for me.
And once I've gotten a better handle on their skill, assuming they're good, we'll both invariably regret it when I start to get complacent with my reviews and they check something in with a problem we've now both missed. It wouldn't surprise me if the best programmers I've worked with sneak more bugs past me than the worst as a direct result of this...
> And all this despite the fact that we have some mitigations these days, such as address randomization, fewer pages being executable, privilege separation, safer coding styles and practices being pretty well known in some circles, etc.?
Yes, absolutely. Why even indulge in any of these mitigations if you trust yourself and your fellow programmers with unsafe pointers? Each of these is vitally important specifically because we can't. And unfortunately, these mitigations are imperfect solutions. A horrifying amount of software disables them outright - and even when enabled, exploits still show up in the wild - they just get more convoluted and end up with lower success rates.
Better language choice is yet another potential mitigation. For me that usually means C#. This too is imperfect: I'm still more than able to write an unchecked buffer overflow with an unsafe block, or a marshaling API call, etc. ad infinitum - but it's an improvement.
> And where would that lower 90th be if it weren't for the fact that these days people are primarily being taught in memory safe languages?
Spending more time learning about and fighting memory bugs instead of shipping features, a sad waste of productivity. (I don't think that's the answer you were hinting at, but I'm not sure what is. An aside: My field still worships and is primarily taught memory unsafe languages.)
C programs are still much smaller than something written in many of the newer languages. This still matters for embedded software.
The portability of C is also huge because you want this running on a large number of hardware platforms. Everything from tiny home routers to the biggest server imaginable.
For something like an ntpd daemon you also want a language that many developers read and understand. C is used by more developers than pretty much any of the newer safer languages.
It might be old and weird, but C is still really hard to beat.
"The main reason I didn't start from OpenBSD's NTP implementation is that it is not aimed at being part of a larger family of time-keeping programs, like I intend to deliver, so it wouldn't save me any time in the end."
If Ntimed separates out the client from the daemon, so I can install just the client package only, IMO that will be a big win for computer time synchronization.
One thing that has always bothered me with the NTP project, is the lack of a separate daemon and client. If you want the NTP project's client, then you must also install the daemon, even if it is listening only on localhost, or if you opt to not start it up on boot. It still must be installed though.
On my Debian system, I only have the following "clients" available:
* libnet-ntp-perl
* openntpd (which also installs a listening daemon)
* python-ntplib
In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running, nor do I want to manage firewalling and configuring the daemon to listen on localhost to prevent remote queries. The only other two options are programming libraries.
As such, I'm looking forward to systemd-timesyncd. It implements a simple SNTP client to synchronize your clock. As such, the full NTP complexity is not a concern. This is very likely an attractive scenario for most GNU/Linux installations, as most users won't be interested in running an NTP daemon synchronizing other NTP client clocks.
That simply won't do for OP as he seems to want solely client only mode implementation of NTP. chrony supports both. He seems to think the authors of these daemons don't have his best interests at heart by adding functionality he doesn't want personally yet many other people will use.
> One thing that has always bothered me with the NTP project, is the lack of a separate daemon and client.
> In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running
Pretty confused. Why would you care about a code path that isn't executed? Are upset about losing a few kb of disk space? What is the alternative to "a full blown daemon". Can you demonstrate how that is an issue for anyone?
Your comment also shows a fundamental misunderstanding of NTP. It's a peer-to-peer protocol in many use cases, as is PTP so instances have to run as client & server.
Well, in the case of SSDs, every byte counts. But no, that's not my concern. My concern is installing a binary I have no intention of using. It's solid security best practices to only install what you need, and leave the rest out.
Also, I don't have a "fundamental misunderstanding of NTP". I've blogged about it quite a bit:
> It's solid security best practices to only install what you need, and leave the rest out.
You are installing only what you need. And running it in the mode you suggested, client only, will lead to no philosophically different a state than a all new client that does the same thing. Once you set it running as a privileged user it cannot be modified by other users. It will have been tested in your environment and signed off by your infosec team if that isn't you. How is that different from any other piece of software that has options you should never use?
Are you saying if your users use git that the 'bisect' command should be removed because they never need to use it?
Why the hell would you write this in C, when so many other programming languages exist now. I mean, you could write a time sync program in Python today. But if you still want to be maximally performant since you intend to run on millions of computers, Go and Rust are new and fresh and memory safe, and modern C++ is great too. A lot better than malloc and pointer arithmetic.
I get that the original ntpd is almost 20 years old, and opentpd is about a decade old, and back then writing every in C was forgivable, because the alternates were not as far ahead in every metric possible as they are today. Now it just looks like someone trying to use Cobol or Fortran in new production products.
While I agree with the motivation behind your statement, I don't think Rust has proven itself yet to the degree that it should be a default choice for anything.
Remember that compilers and runtime libraries can have bugs too, and new ones tend to have more bugs than those maintained over some time.
Yes, runtimes and libraries can have bugs, too, agreed, but at least you will have to just recompile with the patched version vs having to fix and test your code as fast as possible. It's more efficient as well as fixing the core will benefit many (well, of course, that would have compromised many as well... to start with).
it's easy to criticize someone's choice of language or coding style with generalities--and i don't necessarily even disagree with the substance of your criticism--but it's also important to note that building free software for freedom is much, much harder than any two-paragraph comment you or i might write.
imperfect as it may be, i'm grateful that phk was willing to build something that people can freely choose while contributing mutual aid and solidarity to the free software community--instead of snark.
Yeah. I think we've hit the point where we need to start discouraging the proliferation of C and C++ (in my opinion). It seems silly that we cling to languages from the 80's and earlier when modern languages have advanced so far.
I'm not saying a language must die because its old. Modern C++ is not bad - but you have to restrict your development into the "good" subset, and avoid the bad stuff from before. IE, use references and smart pointers rather than news and deletes.
With C, there is no alternative. It is still malloc and free, still buffer overflows and integer overflows galore, and the C language standards group is not moving to make the kinds of radical changes to modernize it the C++ one is.
And most C++ is written insecurely and poorly, especially since good C++ is literally a new invention in the last three years.
It is why Rust and Go are better in general, because they force you to be safe, whereas C++ gives you all the tools you want to stab yourself in the foot in the same toolbox as all the stuff that makes sense.
Are you speaking from experience writing system components in these languages? Lots of very high-quality software is written in C. Go and especially Rust are still moving targets. Depending on a separate compiler or runtime can hinder adoption and introduce errors when versions are out of sync.
Yes, I am. I have worked on pulseaudio and gstreamer in the past (because of fundamentally broken parts I had to patch and merge upstream to get it to work) and GObject or just no type system at all is a nightmare in a huge project. I'm also contributing to frameworks 5 from KDE, and those libraries are a hell of a lot saner and they are still using Qt functions over std ones that have arisen in the last three years with better syntax, but it sure as hell beats having to spend days manually tracing all your allocations and having to write out every version of a function by hand because you can't template anything.
> separate compiler
Go has its own compiler and runtime, yeah. Rust compiles to strict native and uses LLVM. C++ uses the same compilers as C (GCC, etc). In any case, both Rust and C++ can be distributed as final binaries with no runtime dependencies besides potentially dynamically linked Rust std libs (libgreen, etc) and a C++ stdlib. Feel free to bundle those, in the same way you would have to bundle or trust the end users libc implementation.
NTP is a surprisingly large piece of software considering it's narrow task... and at this point rather crufty, ... e.g. in spite of having a ton of monitoring code that keeps turning up vulnerabilities, it's rather hard to monitor.
It has complex filtering, but still can perform rather poorly (esp with broken servers available). etc.
And accurate and (increasingly-) precise time keeping is mission (and, in some cases, security) critical to many applications.
Time really should be a solved problem but sadly it's not. Though fairly accurate temperature compensated oscillators are not terribly expensive, they don't find their way into typical server clocks. Many public ntp servers give bogus data. etc.
Meanwhile, kernel timekeeping has become more sophisticated. I suppose one of the principles in varnish was letting the kernel do its job, so there may be some opportunity to apply that principle here too.
So I think there is plenty that PHK could do interesting here while keeping the codebase quite focused and compact.
I do wonder how he would compare this effort to some of the other NTP alternatives (in particular chrony).
The biggest annoyance with NTPD is that it will give up synching if the time delta is too large. It just shows that the software was primarily coded to be a server. If you just use NTPD to keep your machine times in sync `chrony` is a better solution for now.
Whats more likely? your system clock sped up by a factor of 2 and everything kept working, or some guy on your network is trying to dork around with your time so he can bypass some time limited authentication?
Of course, the software needs to correctly handle cases like suspend where the time may honestly need to step. If it doesn't thats a short coming of the software, not a reason to eagerly make "presumed impossible" leaps just because some unauthenticated packets on the network tell you to.
Oh, yeah, and PHK hates OpenNTPD. I'm surprised he apparently has recently used such nice words about OpenNTPD! One of the main reasons he was against OpenBSD's sensors framework going into FreeBSD was the timedelta sensor support, a type of sensor which basically tells you the offset of your system clock compared to some other clock source (e.g. http://mdoc.su/o/nmea.4 / http://bxr.su/OpenBSD/sys/kern/tty_nmea.c), and these timedelta hardware sensors are used by OpenBSD's ntpd to set the correct time, which seems to do the job just fine.
108 comments
[ 3.7 ms ] story [ 155 ms ] threadWhy another from-scratch rewrote, instead of helping with that project, especially when it was also motivated by an aversion to the "100,000 KLOC" in the ntp.org ntpd?
And why oh why in C with so many safer options available today ..
I don't think that's how you're supposed to software.
I get that sometimes you just want to hack in a language that's familiar to you, but then call your project a fun side project and not the "NTPD replacement" (and yes, I know who phk is).
To clarify, I'm not claiming that everyone should start using Rust. Now was I claiming that PHK is somehow unqualified for the task, far from it. I'm not even saying C is a bad language (I like C), or that people should stop using C altogether. But if you're building an security-critical application designed to be used in the future across millions of servers, and one that requires low-latency, I simply don't understand why you wouldn't strongly consider Rust, which offers strong guarantees about memory and type safety along with low latency and a very modern and complete standard library. Or Ada. Or OCaml.
About 2 million servers were sold world-wide in 2014Q2
Assume 25% runs UNIX and NTPD -> half a million servers
Assume NTPD uses 0.1% of machine resources -> 500 fully loaded servers.
Assume 100W/server -> 50 kW
50kW for half a year -> 220,000 kWh
Assume 500g CO2/kWh -> 110 tons of CO2
QED: I think CPU overhead is a very relevant concern for time synching programs.
Bold assumption. It uses 0.0006% on my slow 6 years old box (over 13 days uptime). Let's be generous and say it's 0.001%, then you get 1.1 ton of CO2, which is the equivalent of the production of about 32.8 Kg of beef. I don't think such dramatic conclusions can be made from those facts.
Second, have you checked the memory footprint of NTPD ? "machine resources" is more than CPU cycles.
And don't forget: That number was one quarters purchase of servers, running for half a year. All the servers bought in 2010, 2011, 2012, 2013 and the other half of the year were not included.
Things add up.
284K, which is a whopping 0.04% of available memory. I don't see how this increases CO2 output though.
> Things add up
To a still very insignificant number, dwarfed by the beef consumption of the people reading this.
I'd say there are better arguments in favour of using dated languages than this.
I can change how their computers synchronize time.
Removing 0.1% load from the machine will not result in 0.1% of energy savings, very very far from it if in any at all. And that's not even what we want to calculate. We want the difference with hypothetical alternative in a safer language, which is hard to guess, but is unlikely to be in the same order or two with your calculations.
Overhead is important since Linux is designed to run on a whole spectrum of devices. Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead. And frankly nobody could care less about programmer's time/effort/sanity in this case. It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
Like the difference between a ntpd.c vs ntpd.go is going to matter for anyone...
If you really find yourself stuck with an embedded device small enough that a few megabytes of RAM matter then you can still use the old C implementations, it's not like they are going away.
Abstractions mean you are bringing in more code i.e. greater complexity, space, overhead.
In this case abstractions mean you can probably write the whole thing in under 2 kloc, get memory safety for free, and base it on a network stack that other people actually use and debug independently.
It is a tightly focused, specific utility that shouldn't see too many changes after a certain point.
Umm. Yea. Right. Tell that to the ntpd guys with their 100kloc codebase...
Why not write that part in C and leave the rest to the more suitable language?
I would guess that's a rather small portion of the codebase, essentially the callouts to adjtime().
I don't buy into the low-latency FUD. C isn't a magical realtime language either; what happens when the system is under high load? What about delays in the kernel/network stack?
As a layman I would think incoming frames will have to be stamped with a hardware high resolution timestamp at arrival anyway, regardless of the language. And yes, this internal timestamp will have to be compared immediately before setting the hwclock. I would argue this is the only part that really needs to be written in a GC free language.
I might be wrong on this, but I'd like to see a stronger argument than "could well be significant" for writing yet another generation of a baseline system daemon in an unsafe language...
And speaking of latency: If this really is such an issue, why hasn't NTP long been moved into the kernel?
The whole of the Unix philosophy is enforcing this policy of strong abstractions by making each part of a complex overall system its own binary, such that they can only communicate using the file I/O abstraction, possibly via pipes. C was therefore designed for a system which promotes abstractions.
[1] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf
Standards are good and useful things, but they're only fully written down in retrospect.
PS: I would like to see a toolchain for Go that emits portable C instead of statically linking in a giant runtime... Lightweight camping vs. YAGNI hoarding. Seriously, the weight of runtimes would be the biggest barrier to serious systems deployment, outshining the slowdown, productivity and safety tradeoffs. (Eg Go would need to play nice with embedded constraints including memory usage as well.).
System time is far, far too critical to take chances with.
Haskell + stream fusion gives better results than well tuned C in some very typical situations http://research.microsoft.com/en-us/um/people/simonpj/papers...
Even just comparing C++/C, C++'s templating mechanisms mean that generic containers can be optimised on a per-type basis.
Abstraction and overhead are not as tightly coupled as they used to be, and not introducing Heartbleed-style bugs is worth a lot in my opinion
In most cases it isn't a bad thing at all. But a use case like a timekeeping daemon, where a single unexpected garbage collection pause could cause a world of problems, it most definitely is more pain than gain.
This is more a matter of "the right language for the job" and for timing-critical systems programming running as root, that language is C.
And there are plenty of common abstractions I would love see added to the C language: basic linked lists, byte-endianess and packing for struct members, validity intervals for integers and FP variables (like Ada!)
Unfortunately my taste seems to be the direct opposite of ISO-C which have instead wasted time giving us another thread-API.
(Preemptive response to downvoters: please have a sense of humor about yourselves. :P)
You should check out Rust and Go. Their users are most definitely among those who fail to understand C pointers.
My point was, Rust supporters have no problems understanding pointers.
Go isn't in the same ballpark nor do the authors (venerable as they are) seem to have considered that many people need consistent power over their run time execution when microseconds matter. stop the world is not a viable memory management trade off. Also managing memory in c++ these days is pretty "easy".
I've leived for years with people spending hundreds of man hours a year trying to get around issues caused by STW pauses in high throughput java applications. Makes no sense.
The benefits range from overstated [bounds checking is a thing, but amongst experienced practitioners using modern, safe styles and conventions, lack of memory safety is not really the huge issue people say it is] to sometimes irrelevant [I read that the concurrency model aims to eliminate the possibility of race conditions entirely - who cares? In many domains races are a fact of life, benign, or totally manageable, so it appears to solve the wrong problem].
As for Go, I do acknowledge the background of its creators [venerable, as another commenter said], but most stuff I see written by "rank and file" members of that community is very clearly written by people who mostly deal in higher-level languages.
In any case, bounds checking is a tiny side street in how Rust ensures memory safety: ownership and lifetimes are the important novel things (lifetimes especially are novel, only research languages have had anything similar until this point, and I don't know if any have had exactly the same system).
In Rust the story is totally settled: in order from most to least preferred/recommended: use references, no pointers, owning pointers and then start thinking about reference counting/other shared ownership.
Rust doesn't solve race conditions: it solves data races, which are also problematic (they cause memory unsafety). It is impossible/very hard to avoid arbitrary race conditions, but data races are the only ones that can cause memory unsafety directly.
The first is very clearly a borrowed C++ feature (from RAII and smart pointers). I'm unfamiliar with the 2nd but it sounds like just making the compiler yell at you more when you break scope rules (like C++ will happily let you do with dangling pointers - I will give you that dangling pointers are bad).
> Rust doesn't solve race conditions: it solves data races, which are also problematic (they cause memory unsafety). It is impossible/very hard to avoid arbitrary race conditions, but data races are the only ones that can cause memory unsafety directly.
This sounds like gibberish to me. When I said "race conditions" please read it as "data races". Many of which will not "cause memory unsafety directly". Many of which low-level code such as a kernel will need to not be abstracted from. (Imagine writing a page fault handler in a language that tries to hide data races from you... Doesn't sound like a great idea.)
Even the Linux kernel has very, very few intentional data races. Most of them rely on undefined behavior and Linus has complained before about kernel code breaking when GCC actually tries to exploit that behavior. In those rare cases, it is perfectly reasonable to use unsafe code, along with copious comments explaining why it's safe.
You may want to read this article: https://software.intel.com/en-us/blogs/2013/01/06/benign-dat...
This, in my opinion, suggests that what I read about Rust makes it sound unsuitable for certain domains where concurrent writes and data races are an inherent fact of the universe, a reality that the language seems to deny and be working against.
(Didn't even mention lock free algorithms, for example I was reading one day about [I think it was] the dentry cache using lock free algorithms... You can bet there is part of that code that yes, expects data races to happen and handles them.)
Just to be completely clear, since we didn't define what we were talking about: a data race exists when there are two or more unsynchronized accesses to the same memory location, at least one of which is a write. What happens when that occurs is undefined behavior in C, whether or not you are writing a kernel. On some hardware the result is totally unusable (the write is corrupted). Additionally, the compiler and (on some architectures) the processor are free to reorder operations such that other threads will see your writes in the wrong order, or not at all.
This is all hardware level, the kernel has to deal with these problems too. The "relaxed" ordering in the C++11 memory model is essentially the lowest guarantee you can do anything useful with (other than have something like a racy counter, where you literally don't care at all what the value is, which is basically the only place the kernel has intentional data races at all). For some architectures and operations it resolves to a noop (relaxed stores and loads on x86, for example, never require additional synchronization beyond what the compiler provides).
The Linux kernel takes advantage of these semantics for lock-free or mostly-lock-free algorithms like RCU. There's a lot of code in those implementations specifically to ensure the correct barriers and orderings are in place so that there can't be data races. There's no reason they couldn't be implemented in Rust, and it would probably be easier since they could rely on LLVM's implementation of fences and so on rather than reimplementing it for each architecture. And if LLVM didn't have what they needed, they could just implement it in inline assembly (which is what kernel developers already do anyway).
If you define data races to not include race conditions that your code is handling (by placement of atomics, fences, locking etc.) then maybe your points make some sense but it is exactly the code handling these that I am saying does not fit with what I have seen rust advocates claim. You have to reason about these things at some level, and what I hear people advocate on hn is to sidestep it all by preventing sharing, which may be a fine idea but doesn't work for everything.
Lastly, in my mind, if your lock free algorithm works by letting the races happen but using the result of an atomic op and/or fences to ensure consistent behavior, I would still call that a (benign) race. That you needed atomics, fences, or compiler hints to get what you want seems orthogonal.
> If you define data races to not include race conditions that your code is handling (by placement of atomics, fences, locking etc.) then maybe your points make some sense but it is exactly the code handling these that I am saying does not fit with what I have seen rust advocates claim.
Its actual guarantees for atomics are not very strong and let you handle all of those things in safe code. What's interesting is that it can guarantee correct behavior for types that have more interesting semantics than "a bytestring," like mutexes or smart pointers. The implementations of those things are still unsafe, but you can generally provide a safe API to them, which you can't do in most other languages. This is because Rust has the notion of thread safety built into the language and enforced by the type system. For example, Rust's `shared_ptr` equivalent, `Arc<T>` is defined to be thread safe to share and send to other threads if and only if `T` is also thread safe to share and send to other threads. That doesn't mean you can't build up a safe type from the lower level atomics, or Rust isn't suited for this domain. It means you can write safe APIs in Rust that you can't in C++.
> You have to reason about these things at some level, and what I hear people advocate on hn is to sidestep it all by preventing sharing, which may be a fine idea but doesn't work for everything.
Rust just forces you to do just enough synchronization to not have undefined behavior. It's not a dogmatic language and doesn't believe there is one right way to do concurrency. Channels were just removed from the prelude for precisely that reason--they're not particularly favored over other concurrency mechanisms.
> Lastly, in my mind, if your lock free algorithm works by letting the races happen but using the result of an atomic op and/or fences to ensure consistent behavior, I would still call that a (benign) race. That you needed atomics, fences, or compiler hints to get what you want seems orthogonal.
Data races are well-defined, and what you're describing aren't data races, benign or otherwise. You basically never want data races, just like you basically never to dereference a dangling pointer. Rust prevents things you basically never want, it doesn't prevent writing useful things like lock-free data structures.
Lock free algorithms, or even a lock implementation, does not jive with this. You let the race happen, and you safely detect when you lost or won the race, and then you do stuff accordingly. To use your pointer analogy, it's OK to have a stale pointer in RAM or in a register at a moment in time. It's a violation of the invariants to dereference it.
As for the rest, perhaps I have been misinformed or led to some outdated info on rust when I saw things about no sharing. I will be sure to take a look.
A data race is essentially defined as a race condition between reads and writes where at least one of those is non-atomic. Rust's type system allows one to enforce atomicity by default (e.g. using atomic instructions, or wrapping data in a mutex).
I said "safely" detect, so yes, the final observation that you have won or lost the race must come from an atomic op. However, a "data race" susceptible read can totally be part of the process. The most common idiom I have seen for a lock free atomic read-modify-write has been to do an "unsafe" read, then act like it was OK, then issue compare-and-swap to determine if it really was OK and do the write if successful. (A failed compare and swap means you need to re-fetch and try again.) Yes you need to make sure fences are OK and the compiler is not caching/reordering the read and you need to account for the famous "ABA problem", blah blah blah, but it's what people do and I am not making it up for the purposes of an HN discussion.
The typical RISC approach to atomics, load-link/store-conditional, also encourages what I will call "my kind of thinking" on this issue, though it does so with specialized instructions. You do a special read, then you use ordinary non-atomic register operations, then you do the special store which fails when you lose the race. I will give you that these are specialized instructions and not ordinary C assignments but I would suggest looking into them if you have not already, the semantics are very educational. (And they may just convince you that disallowing data races is an overly restrictive thing to do in some scenarios.)
But sure, if you're manually fencing then you can have data races that are benign; however, just because something has some relatively rare uses doesn't mean it's wildly unsafe for the general case, and so disallowing by default it helps the correctness of the vast majority of concurrent code.
In any case, Rust allows one to opt-in to that sort of behaviour using `unsafe` locally, e.g. one would use `unsafe` deep in the internals of the implementation of the lock-free data structure and with careful vetting to ensure it's correct, and then all users can benefit with the compiler ensuring concurrency-safety by default.
Rust tries not to completely disallow behaviour, just make memory safety the default. The programmer can override the compiler via `unsafe` if they truly know better.
This is an accurate parody of my opinion. I'm unable to say "good C" with a straight face.
> I seriously think the C-hater crowd on HN is just disappointed that they can't reason well about pointers.
Let's be (hopefully) pessimistic and assume I'm a bottom 10th percentile programmer, and assume (I believe optimistically) that every programmer half an iota better than me is physically incapable of writing a single solitary buffer overflow, double free, use-after-free, or other pointer usage related bug. Perhaps they are robots!
I'm of the opinion that the remaining bottom 10th percentile remains a significant problem and security hazard. My continued employment with C++ seems to indicate we lack either the technology to detect, or the will to fire, this bottom 10th percentile. I feel that a hatred of C is a pragmatic step in helping reduce this problem, by reducing the usage of C by my fellow incompetents, by reducing the usage of C in general. Do you disagree? (I extend this stance to C++, despite it being my day job.)
http://www.cvedetails.com/vulnerabilities-by-types.php
Continuing the assumption - I'm disappointed that I can't reason well about pointers. I'm a bit more bummed about being in the bottom 10th percentile. I'm significantly more distressed that people can't reason well about people who can reason well about pointers - that, by virtue of being human, even they will make mistakes.
Being slightly less pessimistic: I'm fucking awesome. C isn't. What Dunning–Kruger effect?
And where would that lower 90th be if it weren't for the fact that these days people are primarily being taught in memory safe languages? A key difference here, and one of the reasons I would resist percentile ranking for this discussion, is simple exposure and practice.
Reasonable. Programming skill is not a 1 dimensional trait... and it's a bit bloody minded, even if it were. I threw myself in the fire with the hypotheticals because it made me uncomfortable, even as a hypothetical for the sake of argument, to be doing such a reduction.
> Should you therefore mistrust by default the software written by those folks, due to being burned by output from the lower 10th working in the same language?
Sure, absolutely. This manifests as a willingness to review their commits for mistakes, and to support their efforts in trying to reduce the chances of making certain types of mistakes, and an insistence they do the same for me.
And once I've gotten a better handle on their skill, assuming they're good, we'll both invariably regret it when I start to get complacent with my reviews and they check something in with a problem we've now both missed. It wouldn't surprise me if the best programmers I've worked with sneak more bugs past me than the worst as a direct result of this...
> And all this despite the fact that we have some mitigations these days, such as address randomization, fewer pages being executable, privilege separation, safer coding styles and practices being pretty well known in some circles, etc.?
Yes, absolutely. Why even indulge in any of these mitigations if you trust yourself and your fellow programmers with unsafe pointers? Each of these is vitally important specifically because we can't. And unfortunately, these mitigations are imperfect solutions. A horrifying amount of software disables them outright - and even when enabled, exploits still show up in the wild - they just get more convoluted and end up with lower success rates.
Better language choice is yet another potential mitigation. For me that usually means C#. This too is imperfect: I'm still more than able to write an unchecked buffer overflow with an unsafe block, or a marshaling API call, etc. ad infinitum - but it's an improvement.
> And where would that lower 90th be if it weren't for the fact that these days people are primarily being taught in memory safe languages?
Spending more time learning about and fighting memory bugs instead of shipping features, a sad waste of productivity. (I don't think that's the answer you were hinting at, but I'm not sure what is. An aside: My field still worships and is primarily taught memory unsafe languages.)
I think it's safe to say that he's forgotten more about C/kernel/low-level/... programming than HN'ers will ever know.
The portability of C is also huge because you want this running on a large number of hardware platforms. Everything from tiny home routers to the biggest server imaginable.
For something like an ntpd daemon you also want a language that many developers read and understand. C is used by more developers than pretty much any of the newer safer languages.
It might be old and weird, but C is still really hard to beat.
2. Because this is exactly the kind of task C was intended for: systems programming.
3. Because I want this to be a light-weight and portable program.
4. Because I have 30+ years of C-experience and happen to like the language.
However beyond that speculation, I don't know. It would be interesting to know.
(TL;DR: Yes, he thought things through and are ntimed is implementing a few new tricks.)
"The main reason I didn't start from OpenBSD's NTP implementation is that it is not aimed at being part of a larger family of time-keeping programs, like I intend to deliver, so it wouldn't save me any time in the end."
https://news.ycombinator.com/item?id=8776155
One thing that has always bothered me with the NTP project, is the lack of a separate daemon and client. If you want the NTP project's client, then you must also install the daemon, even if it is listening only on localhost, or if you opt to not start it up on boot. It still must be installed though.
On my Debian system, I only have the following "clients" available:
* libnet-ntp-perl
* openntpd (which also installs a listening daemon)
* python-ntplib
In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running, nor do I want to manage firewalling and configuring the daemon to listen on localhost to prevent remote queries. The only other two options are programming libraries.
As such, I'm looking forward to systemd-timesyncd. It implements a simple SNTP client to synchronize your clock. As such, the full NTP complexity is not a concern. This is very likely an attractive scenario for most GNU/Linux installations, as most users won't be interested in running an NTP daemon synchronizing other NTP client clocks.
Q: Does openntpd hook into PLL infrastructure?
> In most cases, such as on my laptop, workstations, and embedded systems, I don't want the complexities of a full blown daemon running
Pretty confused. Why would you care about a code path that isn't executed? Are upset about losing a few kb of disk space? What is the alternative to "a full blown daemon". Can you demonstrate how that is an issue for anyone?
Your comment also shows a fundamental misunderstanding of NTP. It's a peer-to-peer protocol in many use cases, as is PTP so instances have to run as client & server.
Even if the code path is never intended to be executed, there are bugs in code that could lead to it.
Yes, it's a P2P protocol, but when you're managing a bunch of servers, you don't use it as such.
You set up one server in your environment (time.example.com) and have all your boxes sync to that.
Why would appserv.example.com EVER need to be able to accept NTP connections from anyone else?
That said, a slightly configured ntp doesn't ever accept ntp connections from anyone else. Thats not like a requirement or anything.
Also, I don't have a "fundamental misunderstanding of NTP". I've blogged about it quite a bit:
https://pthree.org/2013/11/05/real-life-ntp/
https://pthree.org/2013/10/13/ntp-drift-file/
https://pthree.org/2013/10/12/new-public-ntp-server/
http://transmission.xmission.com/2014/05/28/building-a-strat...
You are installing only what you need. And running it in the mode you suggested, client only, will lead to no philosophically different a state than a all new client that does the same thing. Once you set it running as a privileged user it cannot be modified by other users. It will have been tested in your environment and signed off by your infosec team if that isn't you. How is that different from any other piece of software that has options you should never use?
Are you saying if your users use git that the 'bisect' command should be removed because they never need to use it?
I get that the original ntpd is almost 20 years old, and opentpd is about a decade old, and back then writing every in C was forgivable, because the alternates were not as far ahead in every metric possible as they are today. Now it just looks like someone trying to use Cobol or Fortran in new production products.
Remember that compilers and runtime libraries can have bugs too, and new ones tend to have more bugs than those maintained over some time.
while i'm here, compare the tone of your comment with that of the following (lifted from somebody's comment in another HN thread):
Personally I wish this was done with $otherlang.. But then again I'm not writing it myself, so I can't really complain.
imperfect as it may be, i'm grateful that phk was willing to build something that people can freely choose while contributing mutual aid and solidarity to the free software community--instead of snark.
With C, there is no alternative. It is still malloc and free, still buffer overflows and integer overflows galore, and the C language standards group is not moving to make the kinds of radical changes to modernize it the C++ one is.
It is why Rust and Go are better in general, because they force you to be safe, whereas C++ gives you all the tools you want to stab yourself in the foot in the same toolbox as all the stuff that makes sense.
> separate compiler
Go has its own compiler and runtime, yeah. Rust compiles to strict native and uses LLVM. C++ uses the same compilers as C (GCC, etc). In any case, both Rust and C++ can be distributed as final binaries with no runtime dependencies besides potentially dynamically linked Rust std libs (libgreen, etc) and a C++ stdlib. Feel free to bundle those, in the same way you would have to bundle or trust the end users libc implementation.
NTP is a surprisingly large piece of software considering it's narrow task... and at this point rather crufty, ... e.g. in spite of having a ton of monitoring code that keeps turning up vulnerabilities, it's rather hard to monitor.
It has complex filtering, but still can perform rather poorly (esp with broken servers available). etc.
And accurate and (increasingly-) precise time keeping is mission (and, in some cases, security) critical to many applications.
Time really should be a solved problem but sadly it's not. Though fairly accurate temperature compensated oscillators are not terribly expensive, they don't find their way into typical server clocks. Many public ntp servers give bogus data. etc.
Meanwhile, kernel timekeeping has become more sophisticated. I suppose one of the principles in varnish was letting the kernel do its job, so there may be some opportunity to apply that principle here too.
So I think there is plenty that PHK could do interesting here while keeping the codebase quite focused and compact.
I do wonder how he would compare this effort to some of the other NTP alternatives (in particular chrony).
But obviously: if I had throught it was just the thing, I wouldn't have started writing Ntimed.
I also don't expect Ntimed to out-compete Chrony, and if it did, I'd have to start a fourth alternative myself, to keep competition healty.
A very large part of NTPDs problem was that there were no competition, which meant that everything got crammed into NTPD, come hell or 300KLOC.
We saw this also with GCC, which had become a stagnated arrogant monopoly, and suddenly LLVM forced them to care about users again.
Having competing projects is good thing, and I hope Chrony sees things that way too.
Whats more likely? your system clock sped up by a factor of 2 and everything kept working, or some guy on your network is trying to dork around with your time so he can bypass some time limited authentication?
Of course, the software needs to correctly handle cases like suspend where the time may honestly need to step. If it doesn't thats a short coming of the software, not a reason to eagerly make "presumed impossible" leaps just because some unauthenticated packets on the network tell you to.
Apart from OpenBSD's ntpd (OpenNTPD) -- http://bxr.su/OpenBSD/usr.sbin/ntpd/ -- which is a compete ntpd client/server solution done the OpenBSD way since some 2004, there's also DragonFly's dntpd -- http://bxr.su/DragonFly/usr.sbin/dntpd/ -- a client-only ntpd, originally written by Matt Dillon in 2005 -- http://lists.dragonflybsd.org/pipermail/commits/2005-April/2... -- e.g. nearly concurrently with OpenNTPD development and polish.
Oh, yeah, and PHK hates OpenNTPD. I'm surprised he apparently has recently used such nice words about OpenNTPD! One of the main reasons he was against OpenBSD's sensors framework going into FreeBSD was the timedelta sensor support, a type of sensor which basically tells you the offset of your system clock compared to some other clock source (e.g. http://mdoc.su/o/nmea.4 / http://bxr.su/OpenBSD/sys/kern/tty_nmea.c), and these timedelta hardware sensors are used by OpenBSD's ntpd to set the correct time, which seems to do the job just fine.