DeepMind says new code-generating system is competitive with human programmers (venturebeat.com)

The problem description is as big or even bigger then the code ouput. What happens when you dont describe the problem correctly, isnt that a big pitfall?

Competitive with a median contestant who doesn’t 100% the problem.