204 comments

[ 3.1 ms ] story [ 270 ms ] thread
Only for iOS? Surprised they didn't make a version for windows phone... or is this software already built into the camera functionality on WP?
Microsoft's not heavily devoting resources to Windows Phone at the moment. They're phasing out Lumias, and focusing on getting the enterprise experience right on their mobile devices with the HP Elite x3.

Windows Phone isn't really dead, but until Microsoft is ready, they're focusing on winning over iOS and Android users with their apps.

Rumor is that 'Lumia' will be replaced with 'Surface' for branding.
So why not stop it altogether? I'm forced to use one of those Windows Phones. The App Store is terrible. Few apps for the simplest things and god are they terrible...
I honestly can't fathom being forced over to another platform right now. Years of having to support them later, I still don't understand how anyone makes sense of the UI of iOS (it seems intuitive to non-tech people?) and the terrible colossal trainwreck that is where Google has brought Android in the last few years rules out... almost every other phone out there. (I carried an Android for seven years, I've finally had enough, I can't stand that trash anymore.)

We need a third option. If it's not Microsoft, it's gotta come from somewhere else.

My impression is that this is very similar to the official camera app on Windows, if not a direct port. I would not be surprised if at least some code was shared. Maybe?
Windows 10 Mobile has replaced Windows Phone. Much of the practical functionality appears to be incorporated into the Camera app on Windows 10 Mobile.
My untrained eye can't really notice a difference except that the pix version looks brighter (even washed out), the live photo version looks really uncanny valley (where the faces don't move but the world does around them is an unsettling effect).
It's not even that the Pix version looks brighter -- it looks washed-out, like the evergreen trees are more of a blue-green color.
My first thought when playing with the slider was that the Apple version was much better. But, if you focus on the faces (which was probably the intent of the person taking the photo) they do come out better in the Pix version. I think all the examples had the faces underexposed, so the "fix" for each of them was to brighten the photo.
Gotcha. It's a shame it seems to overexpose the background as a result though; that's something I could quickly reproduce in the photos app. Though if it's doing it automatically for each shot based on the faces in the photo I suppose it's novel and more convenient.
That's what I was thinking as well: it'd be really cool if they did some sort of automatic exposure bracketing and combined the best areas of each version since every one of their sample photos generally looks overexposed, especially compared to what Apple's built-in HDR mode would do.
They're also using models with multiple skin tones (dark vs light), which is a classic scenario for underexposed faces.
Agreed on the uncanny valley comment. It's interesting to me that very uniform "smile and look at the camera" is the desired outcome of pictures for so many. This page seems to work on the unspoken assumption that the perfect smile in front of some other thing is the ideal picture. I definitely prefer pictures of people doing something other than staring at a lens with a rehearsed smile. It seems that this and so much other camera algorithmic work is aimed at idealizing that posed picture style.
prefer + perfect = prefect? D:
Most people only care about that smile :/
Downloaded it myself and tried it, and the difference is noticeable. I have to say, the result was better than what the default iOS camera app took. The Live Image is hit or miss (it really depends on what you are capturing), but it works.
I felt the same way; I was amused because I consistently preferred the originals.
> Better mobile photos are just a tap away. Enter your mobile phone number and we'll text you a link to download Microsoft Pix.

Stop teaching people to click on links from SMS texts damn it!

Nice way to gather mobile numbers, not even needed to build a social network first.
We wanted to provide an easy option for users not currently on an iOS device, but only use the number for a 1-time SMS with a link to the Pix listing in the App Store. We do not collect or store the phone numbers, out of respect for user privacy.
I'm in the U.K and I never receive the text, despite saying it has been sent, is it available here?
Does anyone else think of Harry Potter when looking at the last section, Live Image/Live Photo?
I had the exact same thought... but then I was surprised at how "creepy" it seemed to me instead of "neat".

That said, I think I would actually like that sort of thing more than completely static images once I get used to it.

They should take that single feature and package it into a Harry Potter-themed app, along with some sepia filters and parchment textures.

It would be a blast!

In fact, I might start using it like that, as soon as the Android version comes out.

There's an interesting example they use to show the ostensible improvement of Pix over the Apple's Live Photo:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

The Pix-enhanced version:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

Sure, the Live Photo version has distracting movement, but it also has much more natural motion in terms of hair flow. And I feel that the camera instability helps give context to the video, that it's some kind of impromptu selfie on a windy day.

The Pix version offers a clearer image at what I feel is a large cost to the image's integrity. It looks a bit creepy. That said, there are probably lots of good situations for Pix's stabilization tech, but that example didn't seem optimal as a landing page demo.

Looks like a much lower frame-rate too, like Pix is choosing a specific number of "like" frames and dropping the rest in the name of stabilizing the comp.
they didn't pick the most in-focus frame for the still picture in the pix-enhanced version
The Pix version looks like one of those old computer games when they wanted to animate some wind, but cut only afford to animate a handful of sprites on the screen. So everything is stiff as a board except for some flipping scarves/hair/etc...
I think you're referring to color cycling, and it's a much interesting technology than just a handful of sprites:

http://www.effectgames.com/effect/article-Old_School_Color_C...

I think that effect (of having a large static background with a few small sprite overlays providing motion) was largely orthogonal to color cycling.
They're talking about the more general effect of a complete hand-drawn background with mobile sprites/items layered on top (usually with somewhat different color palette &al, so you'd "see the seams" so to speak between what would or had moved and the actual immoveable background)
Easy pick: the Apple version is natural, the Microsoft one is artificial. Maybe some cultural thing and Americans like the unnatural steady shot, I'm from the old continent and am completely distracted by the unnatural look of the Microsoft version.

Microsoft research came up with http://research.microsoft.com/en-us/um/redmond/projects/hype... and it was great. But Microsoft Pix, not for me.

This has nothing to do with Americans and more of a personal preference.
Apple has apparently ceased to be an American company there, as well.
I am imagining a developer at Microsoft somewhere holding their head in their hands when they saw the Live Photo comparison made by the marketing team. To me, they don't really seem to be aiming for the same thing at all - the Pix version is a highly stylised animation whereas the Live Photo is simply a very short, silent video.
I know the Live Photo example on this Pix page is silent, but it's important for people to realize that Apple Live Photos are not -- they capture several seconds of sound.

I "know" this but often forget. When I look through iMessage history at pictures I've sent and received, there are often stray bits of overheard conversation that the sender wouldn't necessarily want or intend to include.

> I "know" this but often forget. When I look through iMessage history at pictures I've sent and received, there are often stray bits of overheard conversation that the sender wouldn't necessarily want or intend to include.

I was surprised with this too. This should be really made more clear.

It wasn't silent for me, I could hear the wind.
The Live Photo is not silent at all, in fact it is very loud with the noise of water and phone fiddling.
(comment deleted)
My first reaction to the Pix version was to shudder and think "that's creepy". Glad I'm not the only one.
I only saw the grass blowing scene in the main video and was like, "what do all these people mean about it being creepy?". That's some shit right there though.

Imagine looking back at your younger years from your pixel hammock overlooking the digital sunset, and you see a picture like that... now you're depressed because you remembered the physical world as being so beatiful compared to the digital world, esp after the big virus slowdown of 2251, but now you think "it must have been a creepy world after all".

I thought it was an interesting example because iOS 10 will stabilize live photos (at least on the new phones, I'd assume the 6s as well).
The Pix example looks like a cinemagraph. They can be really neat and beautiful. There is a popular subreddit, /r/cinemagraphs, and other various google results. Maybe those are examples of what you could accomplish with Pix - but surely you would still need some manual touchup to make it loop perfectly as it should.
There was something happening to "cinemagraphs" in the few years since they became popular. I'm pretty sure it started with this blog: http://iwdrm.tumblr.com/ It took movie scenes/moments and tried to capture their essence in a few frames (of a GIF beneath tumblrs file size restrictions, as an additional challenge). The resultes were quite breathtaking and some of the best, to me, feel very "alive". You could see people breathe and blink and stuff:

http://67.media.tumblr.com/cf5c72b69133541626d6c1e7ff313926/...

Reddit has turned this into a kind of Matrix-time-freeze effect that looks totally unnatural:

http://i.imgur.com/FERUo.gif

It's a totally different thing. If Pix managed to do the first type of "cinemagraph" automatically, that would be pretty cool. But it seems to be more the second type of "cut out some part of the image and make it not move" type of animation. Too bad.

The original creates the perception of realness and dimension through the effect of wiggle stereoscopy.[1]

Wiggle stereoscopy: http://i.imgur.com/FhA9aZI.gif

In contrast, the Pix version removes that sense of depth, and turns it into what is essentially a cinemagraph.[2]

Cinemagraph: http://farm6.static.flickr.com/5303/5659130034_7fc3dcfb8c_o....

[1]https://en.wikipedia.org/wiki/Wiggle_stereoscopy [2]https://en.wikipedia.org/wiki/Cinemagraph

Cinemagraphs are all the rage now. Maybe it was intentional. Probably compresses much better.
I'm sure you're right on both accounts, but man, they are really creepy. I keep thinking I'm watching the trailer from a horror movie and something is going to jump out of the frame - or that I'm staring at a mannequin that should be still, but isn't. Very odd.
About creepy, but in another sense. They are used alot in public ads instead of static pictures, and i think i found one reason they dont do full on video. Saw a underwear ad in the subway with full blown video, as in the whole model was moving, and that looked like a porn ad, nobody dared to stare at it.Especially compared to the ad next to it with some subtle motions of just the hair.
Just as a point of comparison, I thought it was creepy for about 10 seconds, too. Then, for some reason, my brain adapted to where I think I'd now prefer the Microsoft Pix version of the image over the Apple one. Brains are weird, I guess.
I don't find them creepy at all. Guess it's just a preference thing.
Indeed. The LiveImage example reminded me of the paintings on the walls at Hogwarts :)
Seems to me cinemagraphs and wiggle stereoscopy are both red hot right now. I'm sure someone will combine them soonish, creating cinemawiggles.
It might be attractive to people who want to loop an image, I suppose.
Yea, it's interesting tech but it plain doesn't work right. It looks like a "cinemagraph generator" (like it was first popularized with this blog: http://iwdrm.tumblr.com/ ). Only it does the (IMO rather tacky or at least very stylized) version of it that makes people look "frozen in time" instead of trying to loop natural motion. It looks like a filter/effect rather than, well,... an enhanced video. Not better but different.

IMO this is one of those things Apple tends to "get". Did they have comparable tech to Pix somewhere cooked up in their labs? I dare to bet so. Did they use it as a suggestion for a "default" enhancement mode? Nope. That thing is a gimmick and seems more appropriate for a third party app.

The auto-adjust for faces looks nice but don't most cameras have that for years, now?

Id love to know the image proc algorithms they used here.
It probably looks creepy because they don't move a single muscle in their faces.
> It looks a bit creepy.

It's not just a bit creepy, it's firmly planted in the depths of the uncanny valley. The other (still photo) examples make me think they are pulling tricks with exposure and light balance, but the live photo feature is cheesy and gimmicky (and yes, highly unsettling in the example they give).

I'm going to try it out on my iPhone 6, but I expect I'll stick with the iPhone camera app for serious photos unless Pix blows me away.

They just need to get rid of the creepiness by mirroring the loop, so its not a sudden reset
That's not what I found creepy. It's the perfectly still faces with the hair blowing and whipping around.
Since Pix is an alternative to the built in Apple app, it makes sense that it provides an alternative functionality from a user standpoint...and perhaps from a not-getting-booted-out-of-the-app-store standpoint.

As a person pushing get off my lawn age, taking selfies feels unseemly even though I know it is entirely mainstream among younger people. One of the things I find interesting about photography is how rapidly the 'uncanny valley' becomes normal. I look at Instagram filters and wonder 'why I would want to make something look like a Polaroid?' It just means the filter is not for me.

Just a matter of terminology: Instagram filters don't contribute to the uncanny valley--they may make a photo look less accurate, but they don't make the subjects look less like human beings (and therefore don't trigger the revulsion or at least discomfort that is the defining characteristic of the uncanny valley).

Now Snapchat filters, on the other hand... (e.g. the bee filter, shudder)

My first reaction to both of these is that they're unsettling, though the Pix version makes me feel more uncomfortable than the Live Photo.
"The Pix version offers a clearer image"

For me the Live Photo version is more sharper than the Pix one and it has better exposure too.

This feels sort of redundant... that feature list (multiple face detection with auto exposure/white balance, burst mode with an algorithm to guess which one is "best") is all stuff the built-in camera app on iOS does already.

Okay, the color and white balance look a little better for Pix in the example shots, but everyone's RAW processing is better in some situations and worse in others. I'm sure Apple could come back with just as many example shots where their own settings were better than Pix's, and we could go round and round like that forever.

There's nothing wrong with this app, but I'm having trouble understanding why it exists. What does this do for users or Microsoft?

I could be totally wrong, but if I had to guess, they are using it to show off / validate their work on Microsoft Cognitive Services (Project Oxford). It has facial recognition, along with a number of other things, and so this would be an easy way of getting data on possible problems they have and how well their algorithms work. Just a guess though.
Facial recognition is pretty much solid for most cameras now, though. Face registration, which Sony have been rolling into some E-mount cameras, would be more impressive.
It's a bit different if you're prioritizing exposure of different faces based on their attributes though right? If you're taking a photo of a black person and a white person together you're often underexposing someone or overexposing someone.
There's a certain threshold in terms of added-value that has to be met in order for "yet another" app to live on my home screen. This pix app looks like it may have a bit of cool technology behind it, but will I install it? big nope.
Great demo page.

I really love the "People-Centric" feature, especially how the demo shows after shot enhancements for a variety of skin tones.

Thank you Microsoft.

I had the opposite reaction. I found it humorous that the face detection didn't seem to work reliably (e.g. never detecting the guy in the lower right of the indoor restaurant scene), even in their own demo video.
Your thinly veiled racism is not welcome here.
(comment deleted)
Insane. That page loads in 111 network requests totaling about 40MB.
Web development is eventually going to collapse under the weight of itself. This just seems so ... sloppy.
I got 130 totaling 20MB, but still insane. At least some of them are loaded after the initial render.
(comment deleted)
This is what happens when you make everything an API and try to hide all of the details. People program as the docs tell them to, not realizing that what they're doing requires a hideous amount of complexity under the hood. It is sadly rare for an API to actually mention that a particular method is expensive.
Huh, this is a badly coded WordPress site with HTML and CSS?
I don't know how all these big companies talking about the mobile web and PWAs manage to build sites so inefficient for mobile loading.
Came to say the same, crazy. A 4MB+ gif?!
Not sure if this is worse than the redirect hell on the rest of their website...
In other words, it's a typical Wordpress site.
The Pix images just look mostly like the contrast was reduced and all the dark pixels were made brighter. That's not improving the images in my mind, it's just throwing away color depth and subtlety.
I'm amazed how low-quality this MS landing page is.

- The "How it works" tab is just a huge image.

- The search box ( top right ) has some problems with the border. Both in Chrome & Safari.

- It uses `font-family: 'Segoe UI';`, which makes all elements with that font on my Mac "serif"

The "How it works" image is also very pixelated for me (is that intentional?)
Also running on WordPress and exposing version number - which is out of date: https://www.microsoft.com/en-us/research/readme.html
The version number's in every Wordpress page source:

  <meta name="generator" content="WordPress 4.5.4" />
It can be deactivated.

    remove_action( 'wp_head', 'wp_generator' );
We include it in our themes as a best practice.
The worst part is some certified microsoft vendor probably charged $60,000 for it.
No, I know some MS vendors and the work is usually competitive and tightly budgeted. Think of the power Walmart has over vendors to get good pricing and you'll get the idea.
The last item is very prevalent throughout many microsoft.com pages. https://azure.microsoft.com has gotten it right here but I'm wondering what's preventing the rest of microsoft.com to do the same.
I agree, even the comparator descriptions seem weirdly awkward and placeholdery, describing the mechanism used to compare images rather than the comparison (e.g. "animated gif comparison" showing an animated gif used to compare two static images).

The whole thing is presented as if the marketing team haven't worked on it yet.

So, anybody else notice the somewhat limited skin palette they seem to be working on?

I'd be very curious what it does with really dark skin.

(comment deleted)
Sorry for my naivety and maybe a little off topic - but where are the raw image files used in the comparison images ?

I'm sure someone is going to cry foul if the images have been passed through Photoshop.

None of the comparison images have been altered via Photoshop. All comparisons were shot simultaneously on a side-by-side rig, using a trigger that prompted the shutter press for both apps (Apple Camera and Microsoft Pix) at the same time, so conditions would be as equivalent as possible. See a photo of the rig in action here: https://www.microsoft.com/en-us/research/product/microsoftpi.... We did crop the edges of some photos to fit into the available space, but otherwise the images are the unaltered files that were produced by each app.
A curious mind, like myself, found the "Photoshop 3.0" in the jpeg meta data and came to the conclusion that the comparison images had been altered via Photoshop (or similar tool)

To avoid doubt, I would advise that all image comparisons should have references to the original source files.

Thanks for taking the time to reply.

Ooh, they mention it includes Hyperlapse! I haven't looked into Pix too much so I don't know if this is a full implementation (someone commented on an unnatural-looking stabilization demo, but I think it was just a poor choice of an example), but Hyperlapse is one of the coolest research projects I've come across:

http://research.microsoft.com/en-us/um/redmond/projects/hype...

The video demo there is worth checking out.

Will there be an app for windows phone or does microsoft support only iOS?
Will there be an app for windows phone or does microsoft support only iOS?
Will there be an app for windows phone or does microsoft support only iOS?
I wonder if there is a plan for an Android version. I'd love to play with this.
An Android version is development. We can't comment on release date, but we're working on it.
Not an Android user, but this is great to hear.

Will Pix replace hyperlapse on android?

I wonder who is responsible for their site design at Microsoft, their whole website looks very unpolished and quickly made, especially typography.
It's Wordpress, whereas other parts of www.microsoft.com are not. I suspect the Research group was responsible for their own site. Even though it has a marketing feel, I suspect it should be held more to an engineering blog type of standard.
What do they mean by "Apple camera"? Is that camera in macbook, iphone 3g, iphone 7?
We were referring to the default Camera app that ships with iOS devices. All side-by-sides were shot on the same model of iPhone with the most recent version of iOS at the time of the shoot, as well as the latest version of the Apple Camera app and the latest version of Microsoft Pix app, respectively.
The results remind me of older special effects or manual brushing in Photoshop... they are not very natural and not very good. Beeing myself also a photographer: Don't do this to your photos. The best A.I. and filters cannot compensate a well lighted photo. I'm happy that Apple does not include such destructive filtering. IMHO even a flash from your phone is better than this filters.