55 comments

[ 6.2 ms ] story [ 102 ms ] thread
"To an approximation, asking when a non-trivial program will be done is just like asking when a theorem will be proven."

Thank you for this. ;)

First comment: the link to recursiveuniver.se is broken as far as I can tell. Simple typo it looks like. [Edit: this has been fixed.]

Second comment: the more I work with mathematicians and computer programmers, the better I get a sense for how long something will take. If the problem is in the normal range of difficultly, I can usually estimate within a few hours of how long it will take me to make something from scratch.

I think the regime you are referring to one filled with the hard problems that fall outside the normal range. The ones were you have no idea how to make something work. There are no books that give you a rough recipe for what to do. It's just you and the computer trying to make sweet sweet music with your ideas.

Whenever I run up into a problem like that, I switch my mindset over from mathematician or programmer and turn into a explorer and pirate with laughably low standards. I copy and try every idea I can think of. Oh, that one guy did something that looked like it might work? I steal the idea whole sale and see if it solves my problem. No such luck? Time to explore everything. There is no nook or cranny that gets left unexplored. Every other line is a printf or console.log. I need to see everything that the computer is doing. I will try anything once and forgive anything twice if it shows promise.

Normal projects are just you doing some programming. Hard problems are you doing the footwork and experiments to understand what exactly is going on and then doing some programming at the very end.

Oh absolutely! If programming is like architecture, you aren’t always building the first suspension bridge to connect South Africa to Antarctica, most of the time you’re designing a collection of single family dwellings for a tract builder.

I was just riffing off the observation that software development is a behavioural science, not a formal or empirical science. I hoped that this example, while clearly ridiculous (which client uses terms like “super-linear” but simultaneously demands that the unknowable be known by Friday?) would provide a chuckle while making the same point.

Love the description of your approach. Speaking of sweet music, rock on!

As I learn C++ I have a more than once written little toy programs (class Derived : public Base) to verify my understanding and test that what I plan to do will actually work.
I've been working on splines all day.

The client's requirement: to move the control point of the front of a series of splines, in such a way that it interpolates between the N nearest splines, where the nearest splines are filtered at the opposite end of the point the user is manipulating. Basically, you move your control point where you want, but the other end of the curve moves as little as possible. When you stop moving it, it interpolates smoothly to the spline whose equivalent control point is closest to the one you were moving.

I had no idea how long this would take, but I wasn't at all surprised that it's taken me all day just to write an implementation. Tomorrow I need to test it. It's so complicated that it's actually impossible to tell if it's working just by looking at it. My brain feels fried.

It's times like these when you hope your project manager will be sympathetic :)

I think I'm a good developer then I read a post like this and think, "It would take me a week just to fully understand the problem. He's unhappy with an implementation -if not perfect- in a day?"
Ah I didn't start the project from scratch today. I've been working on it for about a week so far. This is a new problem that means changing a lot of the code base, but it's not a new application at this point (thank god!).
Where do you find work like that? Does it pay well?
I work for a small games company. It pays reasonable. I took a pay cut to work here from working in enterprise software. It was worth it :)
I don't have much to add except that my toungue-in-cheek proper-nouning of your subject makes your post difficult for me to read.
“Design a two-dimensional cellular automata such that we can create a self-replicating pattern.”

What clients might require cellular automata, and automata such as these?

http://en.wikipedia.org/wiki/Rule_30 and http://en.wikipedia.org/wiki/Cellular_automaton tell me they have applications in cryptography and random number generation. Are there any other areas where cellular automata are applied?

I think this was made up; just to prove a point.
Axiom: A client or employer's reasonableness of his or her expectation of performance is DIRECTLY (oops) proportional to his or her ability to accomplish the task.

Corollary: If a client or employer's expectation requires you to bend spacetime to accomplish the task in his or her set timeframe, yes, he or she is dumb as a brick.

2nd Corollary: Need coffee.

Don't you mean directly proportional? More ability => more reasonable expectation
(comment deleted)
Thank you. (I was thinking of amount of time to accomplish task, not reasonableness.)
Why do you let your client talk to you like that?
Any client that talked to me like that better be paying serious bucks. Like "early retirement" amounts of cash. Sometimes programmers need a push and need to be motivated, but unless this was paraphrased for dramatic effect, I found the tone very insulting.
(comment deleted)
“Design a two-dimensional cellular automata such that we can create a self-replicating pattern.” I’m a smart guy, maybe not as smart as John von Neumann, but smart enough to come up with a 500 by 500 pattern in a 29-state automaton that works.

Hm, what about "light goes on if a light is on in a neighboring cell"? Not the most interesting, but it does self-replicate, if at least one light is on in generation zero.

Great question!

This is analogous to asking, “How does life as we know it differ from rust? They both grow..."

There is such a thing as trivial replication, the trick is non-trivial replication. I don’t have the background to give you the formal definition, but my layman’s understanding is that non-trivial replication works from a description of a thing and a general-purpose factory that makes things from descriptions.

I think the formal definition says something about the information encoded in the thing that is replicating.

A fundamental questions mathematicians and computer scientists asked themselves in the first half of the 20th century was whether it was possible for a factory+description to be encoded in the description. 70 years later, we know the answer. but at that moment, it wasn’t obvious how this could be done: If the factory required space F for itself and the description of the factory required space d(F), F+d(F) seemed to be larger than d(F), so the description of a factory plus the description of the description of a factory seems intuitively larger than the description of a factory, call it d(F+d(F)).

This seemed to go on ad infinitem, so how could you make a description such that when handed to the factory, the factory would make another factory AND copy the description so you would end up with a new factory plus a new description? Smart people figured out how to do this theoretically with things like the lambda calculus.

John von Neumann was interested in self-replciating robots. Stanislaw Ulam had been working with CAs and suggested von Neumann try buildinga self-replicating CA on these lines, and von Neumann showed this could be done with a 29-state CA. This led to John Horton Conway’s radical simplification that we call “Life” today, and it has been both proven and demonstrated that Life is a universal computing machine. You can build self-replciating patterns that have encoded descriptions, universal Turing machines, hexadecimal calculators, and just about everything else.

Some of the most amusing oddities actually “output” numbers and letters organized as dots as if printed by a dot matrix printer!

Warning: Life is addictive but unremunerative.

"Warning: Life is addictive but unremunerative."

That's how I feel about lowercase life so far.

(comment deleted)
Super-linear complexity is trivially achievable for all computable functions. Perhaps you intended to say sub-linear (i.e. o(n))?
The big problem that people don't seem to understand is, what you were taught in maths class wasn't mathematics. We should rename the popular K12 class "calculations", except for the one called "geometry", which I suppose should be called "formal geometry" or "Euclid class" or something like that.

The issue is the scope of the course. I know plenty of professional mathematicians. None of them can calculate a tip. The way to excite their mathematical minds is to drag out the Cosmic Wimpout dice, play it with them until they understand the rules, and then ask them, "so, it's obvious that strategy doesn't matter much, but what's the optimal strategy?" Bam, they're writing Markov diagrams and fussing over the fact that one of the dice has a wildcard which you can technically choose to be anything in certain rolls, so which one should it be, and so forth.

The most accessible and iconic piece of plain mathematics that I know is the Fundamental Theorem of Calculus. In Math class in high school, you may have been taught this by saying "it's very complicated and rests on some very difficult theorems and trust us, the mathematicians have proved it. But no. That's not mathematics.

I occasionally get the opportunity to talk to high school students and college freshmen about calculus, not in any formal setting, but just one-to-one. I tell them that the fundamental theorem of calculus is about + and −. The − is confusingly called a "derivative" and the + is confusingly called an "integral", but the fundamental theorem just says: + undoes −, and − undoes +, and that's it.

They look at me funny. Suppose we start defining sequences (F₀ F₁ F₂ ...), and suppose we start defining ΔF as a new sequence, Fᵢ − Fᵢ₋₁ . And let ΣF be a new sequence F₀ + F₁ + ... + Fᵢ .

Then it's not too hard to see that Δ(ΣF) = F, in other words, − undoes +. And it's not too hard to see also that Σ(ΔF) = F, because if you look at it:

[Σ(ΔF)]ᵢ = Fᵢ − Fᵢ₋₁ + Fᵢ₋₁ − Fᵢ₋₂ + ... + F₁ − F₀ + F₀.

And those cancel out by pairs into just Fᵢ.

Σ of course forms the basis for a Riemann sum and Δ forms the basis for a difference quotient, and we take limits because the product rule says:

δ(fg) = (f + δf)(g + δg) − fg = g δf + f δg + δf δg

...and we want to say δf δg → 0 to keep the algebra pretty, because if δf and δg are one in a million, g δf and f δg are a million times larger than δf δg, and so on until δf δg is negligible by comparison.

I kind of lost my point, so let me return to it and say that being able to see ∫dx as + and d/dx as −, being able to feel these patterns, that is actual mathematics. Being able to write out the matrix transform of complex multiplication and just seeing, "oh wow, that's a scaled rotation matrix!" and the implication -- "wow, complex-differentiable functions must preserve angles!" -- that sort of thing is mathematics.

I think it would be nice if we taught this sort of thing in a mathematics class, but in the meantime, I'd prefer if we just insisted that "math class" doesn't teach mathematics until you get toward college.

I'd opine that when one first encounters mathematics which is not just rote calculation depends, at least, upon whom one has for a teacher. It can happen long before college that a teacher drives students towards the discoveries you allude to.
Yes. A teacher with talent can steer even an elementary school student to "real math" insights.

Something like Pascal's triangle is very interesting and contains a lot of patterns, relations, and theorems, which open out very wide and deep in the right hands.

Without contortions, you can go all the way to the rudiments of information theory if you want to.

> Then it's not too hard to see that Δ(ΣF) = F, in other words, − undoes +. And it's not too hard to see also that Σ(ΔF) = F, because if you look at it ….

Note that your derivation assumes that F₀ = 0; but that, on the other hand, we can't assume this blanket condition, since it's not preserved by Δ (unless we restrict further to sequences with F₀ = F₁ = 0, etc., until eventually you wind up considering only the 0 sequence, which barely interests even mathematicians). One is thus forced into the definition of the indefinite integral (which I suppose we would call the indefinite summation in this discretised view? :-) ), which, in our context, says that Σ(ΔF) = F + F₀.

(None of this is to take away from your very fine main point, or even your equally fine smaller point; but it's a shame to come so close to a rigorous-and-yet-intelligible picture, and then to fall just shy!)

(Perhaps you were implicitly identifying sequences that differ by a constant.)

Sorry that it took me two days to get back to this; if you're not reading the comment threads, that's okay.

I did indeed forget the one statement which makes this rigorous (and nontrivial): technically this definition for [ΔF]ᵢ = Fᵢ − Fᵢ₋₁ is undefined for [ΔF]₀ , because our sequences don't have a -1'th element. For these we take [ΔF]₀ = F₀, which in some sense means that F₋₁ = 0.

I don't know why you want to require that "F₀ = F₁ = 0, etc." -- perhaps you were, in the back of your mind, iterating through the negative numbers, thinking that they were positive. ^_^;;

Sure, I'm OK with the implicit extension by 0 to the left. No, my problem is with the statement that ΣΔF = F. Indeed, since Δ annihilates constants, it's not possible to recover a function from its Δ-image, by Σ or any other means; just as it's not possible to recover a function from its derivative. For example, ΣΔ1 = 0.
Here is the proof that ΣΔF = F. First we define our "sequences", which are just functions from N to R; N includes 0. Next we define operators, which I will say always take precedence over function arguments. We define two operators in particular:

Δg(i) = g(0) if i = 0, else g(i) − g(i − 1)

Σg(i) = Σ{n: 0 → i} g(n)

Note that Δg(0) = g(0) by these definitions, and note that Σg(0) = g(0) as well. Call these the "zeroth element relations." (Also note that I am saying Σg(0) where perhaps [Σg](0) would be more clear.)

I will observe a very simple lemma which embeds Σg in a recursive form:

Σg(n + 1) = Σg(n) + g(n + 1) for any g, any n in N.

On to the proof. ΣΔF(0) = F(0) by composing those two zeroth element relations. Also, if ΣΔF(n) = F(n) for any n, then I claim that ΣΔF(n + 1) = F(n + 1). Proof:

ΣΔF(n + 1) = ΣΔF(n) + ΔF(n + 1), by the lemma.

ΣΔF(n + 1) = F(n) + ΔF(n + 1), because "if ΣΔF(n) = F(n)".

ΣΔF(n + 1) = F(n) + F(n + 1) − F(n), by definition of Δ.

ΣΔF(n + 1) = F(n + 1)

Since it holds for n=0, and whenever it holds for n, it holds for n + 1, by induction this holds for all n in N and therefore, by induction:

ΣΔF = F

The reverse argument is also pretty simple. It's also obviously true for n = 0 due to the zeroth element relations. For n > 0 we don't even need induction:

ΔΣF(n) = ΣF(n) − ΣF(n − 1), by definition of Δ

ΔΣF(n) = ΣF(n − 1) + F(n) − ΣF(n − 1), by lemma

ΔΣF(n) = F(n)

ΔΣF = F.

If you still don't believe me, here it all is in Python:

    >>> sigma = lambda seq: lambda i: sum(seq(u) for u in range(0, i + 1))
    >>> delta = lambda seq: lambda i: seq(0) if i == 0 else seq(i) - seq(i - 1)
    >>> expand = lambda seq: [seq(i) for i in range(0, 7)]
    >>> constant = lambda N: lambda i: N
    >>> expand(constant(3))
    [3, 3, 3, 3, 3, 3, 3]
    >>> expand(sigma(constant(3)))
    [3, 6, 9, 12, 15, 18, 21]
    >>> expand(delta(constant(3)))
    [3, 0, 0, 0, 0, 0, 0]
    >>> expand(sigma(delta(constant(3))))
    [3, 3, 3, 3, 3, 3, 3]
    >>> expand(delta(sigma(constant(3))))
    [3, 3, 3, 3, 3, 3, 3]
Enjoy. :-)
You're right; sorry. I was stuck (as my comments indicate) on thinking of the classical setting of calculus, where functions can differ by a constant; but, since your functions are implicitly extended by 0, they cannot so differ! That is, unlike the constant function 0 : Z -> R, the 'constant' function 1 : N -> R isn't really constant once we regard it as a function Z -> R.

P.S. I am a mathematician; Python convinces me of nothing but special cases. :-)

Mathematics isn't necessarily solely about what professional mathematicians do in the contemporary world. Saying that 'calculations' and 'geometry' aren't 'mathematics' is like saying that atoms aren't chemistry.... ridiculous.
Memorizing the periodic table does not make you a chemist. It won't even suffice to pass chemistry 101. High school math amounts to barely more than memorizing the periodic table.
(comment deleted)
Okay. so your statement here, in a very convoluted way, says that high school maths alone does not make you a mathemtician, and implies that I am saying the opposite.

Of course high school maths alone will not make you a mathematician, but its still maths, and its silly to say otherwise! This is my only point here.

The problem is that the same word has to pull double duty. Most math courses even in college teach you how to use math. They do not teach you how to do math. Science is similar. During an undergraduate education in physics, you don't really do physics. You learn about physics. You don't actually do physics until you get to grad school and start doing research.
Math is a lot easier to do independently than Physics though. Maybe it is different for pure mathematics, but applied mathematics can be used independently and in class without going to grad school. It is like someone going to school for programming and learning different programming languages, but never really doing to much with it. It would be up to that person to do some independent projects for themselves and the same with applied mathematics.

I don't even think a degree in physics or pure mathematics is even worth much by themselves without grad school.

And your point is right because I would not have been able to do Calc I if I did not know Trig and Algebra. The same way that I would not have been able to do Algebra without without basic math. I hate memorization and prefer immersion, but it was important to remember how to do that stuff because it was needed later on.

I love the more logical side of math that does not even deal with that many numbers (which is why I am majoring in), but the stuff in school is definitely maths. The way it is taught is questionable, but it should not be dismissed.

I took 10 semesters of 'math' in college, was on a 'math' team in high school, and got a 780 on the 'math' SATs. Yet by the OP's definition, I don't know any Math because I'm not doing his definition of Math.

That's like arguing that you can't swim unless you are an Olympic swimmer. I can't do the back stroke like Michael Phelps, but I sure can swim.

High school math - at least for most - includes Calculus 1. Yes, that involves memorizing integrals and using them later. But if you are going to be that flippant about it, basically what you can marginalize everything. Organic chemistry? Meh, that's just a bunch of memorizing compounds, it is not more difficult than 1st grade memorization of multiplication tables. Anatomy? Ha, that's child's play, it's just like memorizing the names of colors.

I think the poster is arguing against the conflation. Organic chemistry is not memorizing compounds; mathematics is not memorizing integrals; anatomy is not like memorizing colors. There is a great difference between understanding and memorizing.

I had a very bad history teacher. On most tests, he required the class to learn about 20 different dates (only) and some facts like where a peace treaty was signed. Nobody cared what those events meant, what were connections between the dates etc. Now, many years later, I don't recall anything. I learned nothing. Was I really learning history or just arbitrary numbers? You could claim that it was real history, and probably be technically correct, but I think that's silly. In my mind, the semantics of what I knew did not resemble history at all. In contrast, my chemistry teacher visibly required understanding, and till now I can discuss basic chemistry, even though I did not exercise my skills for a long time.

It's the same with mathematics, learning algorithms means essentially you are programmed to do mechanical work, just I like I was acting like a lookup table during history classes. "Real" mathematics starts when the problem you are doing is not directly amenable to a mechanical approach.

Exactly. In the case of mathematics, if you aren't doing proofs yet, you don't even know what math is. Most high school math curricula are completely devoid of proofs or abstract reasoning with the exception of at most a few months during geometry. How many college students have wrongly expected "Modern Algebra" to be a rehash of what they learned in high school? The distinction between mathematics and what gets taught in high school is very similar to the distinction between computer science and computer programming.
Your analogy doesn't quite work because the difference between a high school swimmer and an Olympic swimmer is a difference of degree. That its, they both do the same thing, one is just (much) better.

But the difference between what people do when they take most math courses up to about their third year in college, and what mathematicians do, is a difference of kind, not degree. You can see my other comments in this thread to see what I think that difference in kind actually is.

The way I often say it is that most of people's mathematical education - even most of the math that people take in college, up to about junior year - is mechanical and recipe driven. It's usually about "in this situation, do this to solve the problem." You don't need much actual insight into what's going on, you just need to be able to recognize the patterns and know when to apply what. This makes sense, since the vast majority of people who take even differential equations are not people who need that kind of deep knowledge - they're often engineers or scientists who need to know how to solve a particular kind of problem when they encounter it.

In undergrad, I thought I was good at math. Then I got to grad school and took grad level courses in algorithms and operations research. That was when I realized that I had learned - necessarily - mostly recipes, not fundamental mathematics.

I should just point out, contrary to the really true-to-life dialog, that mathematics is NOT a branch of science.

You don't apply the scientific method to establish a law in mathematics, you prove that it is true.

This confusion is one of the four pillars that comprise the Law of Unintended Consequences.

I have always understood that mathematics is a formal science, whereas the scientific method applies to what we call empirical sciences. Then there are things like “Management Science,” which are considered “Bodies of Knowledge” and are unlike both formal and empirical sciences. The theoretical side of Computer Science is a formal science.

I won’t say what psychology or psychiatry are. Depending on who you ask, they’re empirical sciences, bodies of knowledge, or cargo cult medicine. Software development seems to be all over the place. Some of it clearly relies on computer science, some of it seems to be a body of knowledge.

Though "formal science" is not a familiar term to me, I take your word for it and hear what you are saying. However, I always considered mathematics to be it's own thing, different than science. Real science is physics and chem and then biology starts to merge more into a body of knowledge.

Anything that has "science" in the title isn't really. And in line with that, I consider "computer science" to be a misnomer as it is more accurately categorized as a branch of mathematics. And computer science is, of course, completely distinct from software development, which is a craft.

All that said, I would be hesitant to put psychology and psychiatry in the same boat. Neither one is a hard science, but psychiatry is much less likely to be anyone's fall-back major.

Hey, God, how about an insight that will produce fruit in math?

God says...

C:\LoseThos\www.losethos.com\text\HUCKFINN.TXT

so, and I reckon he clean forgot I was in the world, he was so excited and panting.

All of a sudden the lightning let go a perfect sluice of white glare, and somebody sings out:

"By the living jingo, here's the bag of gold on his breast!"

Hines let out a whoop, like everybody else, and dropped my wrist and give a big surge to bust his way in and get a look, and the way I lit out and shinned for the road in the dark there ain't nobody can tell.

I had the road all to myself, and I fairly fle

----

God says... Royalties curiosities affright custom churches goodliness contrary calls Simple pen unsound rigidly don't_you_love_me livelihood entangle delay linger pitied enmities broad confession bait increase glowed NOTICE glad measured Anon treat experience