Does 4K display result in productivity loss?

How did you measure the productivity drop?

It's harder to find suitable wallpaper for HiDPI display.

But if serious, there may be some problems with adapting specific software to run HiDPI.

What is the refresh rate and latency?

I have two monitors on my desk, One of them has a very high resolution, but with noticeably higher latency than the other.

When using the high resolution / high-latency display, I notice that I start to feel more clumsy, frustrated and stressed, so (despite all of the gorgeous pixels), I tend to use it just as a 'side display' for static documents and reference material ...

I think latency could be a critical factor here.

I just upgraded myself to a 4K 28" monitor. Mine has 1ms response time. I noticed a slight loss of productivity in the initial stage, but really just because I was getting used to it. Now I'd say it's increasing productivity since I don't have to Alt+Tab all the time anymore. I can fit 4 source files comfortably now. So, in regards to your drop in productivity, I'd say give it time.

The productivity drop I see is that machines spiking CPU and apps like sketch struggling to keep up with updates and refresh. When that starts then devs and designers start tweaking settings and googling answers and boom. 1 hour gone to IT issues. Swapping adapters. Swapping cables. Cycle continues...

I run two HP Z32x 4K monitors off an nvidia 1060. Text is easy to see. Lots of pixels. Very little alt-tab. Lots of productivity. So my environment isn't really about retina and HiDPI,

On the other hand, take a look at your developer's postures: are they leaning/squinting into the screen? Are they using 1:1 pixels, or scaling?

What does the retina get you when one is a developer?

Coming back to what I said, I have large monitors so that I can make use of and see the pixels. Any thing smaller would probably be a waste of 4K unless I had really good eye-sight.