That’s twice in as many months. They may need some better QA here although to be fair it does seem mostly people with an older SDK version based on the comments. But still. I guess they need to cotton onto whatever server side change happened and roll it back.
At least this time you can disable the SDK (at least one app developer has a kill switch for the SDK) unlike the last issue that happened during code initialisation you couldn’t skip.
There's no such thing as QA at Facebook. Really. Developers are supposedly responsible for all testing of their own code, but all of the pressure is to "move fast" so that goes about as well as you'd expect.
I’ve had the same basic bug crop up 3x in the last two years while using the API - posts to Groups with dollar amounts just… fail. “An unknown error occurred.”
The first two times it took months to fix. It’s happening again currently, too.
Clear there are GAPING holes in their test suites.
There are multiple dimensions you can use to measure success. Market capitalization is certainly one of them, but it is at best only tenuously related to engineering quality.
Close a great venture capital round, suddenly you've got a big pile of money in the bank and your company's valuation is really high. That means you're successful!
Make a lot of money selling lifetime subscriptions, then have nowhere to go for cash next year: look at all that money! I'm very successful now!
If all you're measuring and optimizing for is "valuation" or "revenue" you're not automatically going to get a good product or happy customers...
There is, it just depends on the part of the company you're in.
If you have a team run by idiots, then you won't have QA.
If you have a team run by adults, then there is plenty of QA. I mean its total shite that this wasn't caught. I'd expect much better, especially as this is the second time in n months. But then perhaps working for a company run by a dick who can't listen is getting to people.
I work there (in infra). I have yet to interact with a team that has dedicated QA. Checked the internal jobs board; no relevant term is listed among titles, areas of work, talent profiles, or skills, anywhere in the company including WhatsApp or Oculus. This is consistent with the "developers are fully responsible for their own code" ethos we're all taught in bootcamp - same reason we all get to do oncall. I'm personally familiar with many real adults up to senior director level, so "if you have a team run by adults" is definitely not true. Given your characterization of MZ (whether or not it's accurate), I'd say you clearly don't have any actual knowledge of the situation and shouldn't be presenting your guesses as fact.
AR/VR, Oculus, portal and workplace all have dedicated QA engineering teams. I'm sure there are more.
> Given your characterization of MZ
if it was only zuck. Zuck has two faults: he's not able to articulate any single thought inside of 15 minutes and he consistently throws out integrity products that would have stopped bad PR.
Its not just me, you should look at the micropulse. confidence in leadership has taken a good battering recently.
The whole juneteenth buisness, we had an entire _day_ of people telling us the problem. Senior management then sat next to them and said in reverent tones: "thank you so much [[dabs a tear]] for telling us your lived experience" posts a selfie and then does fuck all else.
Or, having revved up the company to generate hundreds of ideas to tackle abuse on FB platform, do shit all with them, not even acknowledge they exist.
> You must have leet scuba skills with searching skills like that.
Nothing to do with scuba, and if you did any operational work you'd know that it's in the category of tools that are often flaky due to lack of QA. It's also worth noting that the link you provide only shows 41 listings, and most of those merely mention "familiarity with QA" in some boilerplate for what's obviously a developer position. Leet reading skills you have there. Percentage-wise, there are fewer true QA positions there than we have E-10s. For a company that hires 10K a year, that's indistinguishable from zero.
> AR/VR, Oculus, portal and workplace
In other words some small percentage of the company, none of it near me. All of the systems that the profitable parts of Facebook rely on - to provision systems, monitor systems, store data, move data around, analyze data, etc. - rely entirely on developer testing. So here's an amended statement to make you happy.
"99.9% of Facebook has no QA, and the rest has really bad QA."
just over 10% of job openings have something to do with testing or QA. Of course QA is similar to software engineering, the whole point is to automate testing.
> [AR/VR, Oculus, portal and workplace..][..is] some small percentage of the company
yeah, go look at the headcount.
> none of it near me.
Now we are talking.
> rely entirely on developer testing
again, that wasn't the distinction I was making. It doesn't matter who's testing, so long as it gets done properly. Considering PE is a blended role. Its not a hard push to think that testing might be part of the role of a modern developer. Mind you, so is writing documentation, but thats not a priority either.
> to be fair it does seem mostly people with an older SDK version based on the comments.
No, I don’t think there’s any “to be fair” in that, unless those SDKs have been officially EOL’d. Even then it shouldn't take down the whole app, Facebook should just break.
That this only happens with some SDKs certainly makes the problem less severe than last time, but only relatively.
It's fantastic news in that it forces the companies who will undoubtedly lose millions of dollars in revenue from this to evaluate their inclusion of the tracking spyware in their apps.
Obs: I'm being sarcastic but since it's either this or developer laziness, I'm betting on tracking since I always start from the principle that the developers analysed the situation and made an explicit choice.
this, combined with a wrong past technical decision that we can not change it instantly. unfortunately it's not as easy to fix mobile bugs as it is on the web, as you have to both pass the approval process and wait for the full rollout to clients.
in our case, it's just a wrong decision, we don't use it for tracking.
also, if you use facebook login, the terms and conditions force you to use their SDK (which automatically initialises when included and does this crap).
Just to clarify, when I say "the developer" I don't necessarily mean the person writing the code: I mean the organisation developing the software, so they definitely have the choice.
In terms of the person writing the code, they can certainly push back. Being asked to add the Facebook SDK is either a red flag enough for him/her to do so, or they don't mind. In either case, they've made that choice.
You can implement OAuth entirely server-side without any client support beyond displaying a web view. No need to include spyware and the approach is reusable for any OAuth provider.
You can’t, it’s against their policy, if they catch you they’ll revoke your access.
developers.facebook.com/policy/
8.2 (which states “Native IOS and Android apps that implement Facebook Login must use our official SDKs for login.” This has a help icon next to it that opens a dialog on their site that shows a couple pictures and then explains “Android apps should use the default login behavior defined by the SDK, which may use the web-view Login dialog. On iOS, only kiosk apps may use a web-view Login dialog.”
A generic login experience on the web (where the entire login flow is implemented in the browser and the login button just redirects to Safari, including for email/password accounts) should work around it. You can provide technical reasons as excuses for why it's impossible for your app to include edge cases for each login provider to be included natively in the app.
You understand the subtext here right? They literally want you to install their malware on users devices so you can use their SSO. That's the entire ploy. It's how they ensure their malware spreads.
this is not how real life works, you don’t get to choose all the features and you can’t quit jobs every time you’re asked to integrate a sdk. also, you are suggesting that it is a good option to drop support for all the customer accounts that have used fb login, this is silly.
sometimes i wonder if you people lost all the connections with the reality... or if you were ever employed.
I've been everything from making more money than I needed to homeless and borrowing couches and front porches in winter.
I've also refused to include Facebook in my life even though the majority of my extended family practically live there and rarely communicate outside of FB posts. My wife and I just get text messages on the rare occasions instead.
Yes, been employed and unemployed. And I'm under no illusions that FB login support will get dropped anytime soon. I can still lobby for the desirable alternative.
I've been fighting with this for an hour. what the heck Facebook guys, you have a responsibility to not break the apps that use your SDK... You can move fast and break you own things.
I recently learned that you need the FB SDK in your iOS app if you want to promote it on FB or Insta. So I guess most apps doing social marketing have the SDK.
Are you sure? Seem to recall needing FB SDK to manage marketing pixels that provide detail on marketing campaign analytics. If you’re spending money on those platforms to drive traffic you’re going to want to optimize your spend and see performance. To do that I believe you need the FB sdk - are you saying there’s an alternative or are you saying “just don’t optimize performance”.
Can't you optimize ad performance by generating unique URLs for each campaign and then counting the number of clicks/installs per campaign on your own infrastructure?
Doesn't Facebook charge you for clicks regardless? Given the SDK is open-source and is embedded in an app controlled by the developer, there's nothing preventing them from modifying the SDK to report false clicks if Facebook actually relied on the SDK to report ad metrics for billing purposes.
Workaround for users: Enable airplane mode and disable WiFi, or block facebook domains
Workaround for developers: Go to Facebook Developer page > Analytics and disable automatic Events.[0] Apparently this doesn't solve the issue for everyone.
How is it that a third party SDK is consistently causing apps to completely crash rather than, for example, displaying an error message? Is this a failure that cannot be caught? Sorry, just trying to understand. The dependency relationship just seems to be extreme.
I believe last time this happened it was because they were running code (including network requests) in a callback that gets called when they were loaded by the dynamic linker.
Didn't they also use to home in information about surrounding bluetooth devices? When iOS 13 beta dropped, half of the apps suddenly asked for bluetooth permissions because Google/FB SDKs attempted to use it on app launch.
Not sure about Bluetooth devices, but the Facebook SDK definitely phones home with some system data and generates an unique identifier on first launch which is then included every further interaction the SDK has. Both of these enable fingerprinting and persistent tracking.
The fact that we as in industry and we as citizens of free nations do not end these operations, terminate their corporate charters and throw the responsible in jail is indicator #1 that freedom (not the GOP free-dumb) as we know it is in it's death rattle.
When we looked at it, we saw several reasons, all of which are dubious at best:
1. It is required for a "login with Facebook" feature, apparently. I'd presume this goes for other feautures that integrate with facebook, such as "invite from your friends" or "share on facebook(messenger)".
2. It gives access to metrics, when added to your own, greatly enrich your own data collection.
3. It is required when working with Facebook ads - though I would not know how this ties in with mobile apps: on web it is clearer (where you are required to host "the pixel" on your website in order to have ads on facebook link to that website.)
4. It is feature rich, so as a pure SDK, it offers neat utilities; though for each of these, there are better alternatives found in separate libs.
Extending point 2, it’s essential if you’re running ads on Facebook and want to know if clicks on the ad in Facebook bring the user through to the app - ie so you can be sure you got a real conversion
So yes the definition of “freedom” is “give the government more power over both app developers who voluntarily integrate the SDK and users who voluntarily download the app”.
I don't remember apps listing the SDKs they use. As a humble user, how do I voluntarily choose whether to download an app or not based on their usage of Facebook's garbage?
Edit: You can't have it both ways. You can't remove a bunch of power and information from the user in the name of "safety" and "UX" and then claim that users are making a bunch of choices they can't possibly be informed about "voluntarily"
My guess in this particular instance is that it seems like the Facebook SDK is fetching some sort of remote serialized configuration, calling “count” on what it expects to be an array, and instead is calling it on NSNull (an object representing “null”) which is throwing an exception.
There are some intricacies on iOS surrounding null values and serialization, and as a developer it is important to understand that you may encounter NSNull. As a standard practice in my company, we type check every remote value before calling anything on it. Seems like that would have prevented this issue.
This is exactly what happens. Basically a null value is converted into an NSNull object and calling count on it will of course throw an exception... because there is no count method in that class...
It makes me wonder what all those engineers at FB are actually doing... ? Every time I tried to integrate or look into the FB SDK for simple functionality it was a total clusterfuck.
It may not be a conventional null, but I'm not aware of any tech where calling "count" on a Null, .nil? or otherwise null-like construct won't end horribly.
The Facebook SDK is written in obj-c, where calling `count` on `nil` is perfectly valid and returns 0. It only doesn't work on NSNull, which is a weird different thing that does not work like the language's built-in null.
I'm curious about this as well. I've always written frameworks to be used with a 'Policy Layer -> Mechanism Layer -> Utility Layer' general architecture. Each layer captures and does its best to handle errors in a way you would expect that layer to. The only time errors move up towards the policy layer is explicitly because the next layer up will handle it.
Given how protective Apple is about the experience of apps on the App Store and this happening the second time, I'm surprised they have not come up with a way to prevent this from happening. Also, a monolithic SDK should not exist in this day and age.
Indeed, and if the FB SDK (as used by the Spotify app among others) does in fact call home, it might violate Apple's TOS and give rise to rejecting those apps on the App Store. I'm not up to date with Apple's policies, but wasn't Apple specifically rejecting apps that seek to establish a platform within Apple's platform? Now if Apple are going to play hardball on this one is another question alltogether.
So exactly how much money does Apple make from free apps?
If Apple wanted to make money , it should make it harder for advertising supported apps to make money and force apps to either not exist or get people to pay for them.
I guess Apple will have to weigh whether they'll want to tolerate the Fb app on iOS, but more probably they have a deal with Fb (else they would've kicked them out already). Considering Fb is said to be the main portal for "the web" as a Fb user might see it (which I find still very hard to believe for a number of reasons), if the Fb app (or worse, WhatsApp) isn't on iOS, with Safari and FF already blocking most tracking on facebook.com, Fb might block Apple devices altogether, and Apple might or might not stand to loose in device sales, depending on the intersection of Apple's with Fb's target demographic. If it were only for Spotify (which is a direct competitor to Apple Music), they'll have to weigh whether them kicking Spotify out will be seen as anti-competitive vs having a good excuse for eliminating a competitor on their own platform. But maybe my speculations are way off, and Apple will just tell Fb to fix their shit.
A quick solution on Apple's part could be to add an easy to use domain blocker as part of there tracking blockers for Safari. Could be buried in the safari settings pages, but with a toggle to apply at an OS level.
None of the apps broken on my phone require Facebook features for core functionality. Downtime happens, but downtime due to your non-essential & user-hostile ads/tracking/social SDK? These companies deserve every single cent of lost revenue.
If the last time this happens is anything to go by, a lot of apps have the FBSDK to facilitate their "Log In
with Facebook" functionality. The issue is that you have to include a monolithic SDK that initialises itself without your control just for simple functionality like that.
Temporary workaround for people who need Spotify as badly as they need oxygen (Eg: me, and my kids)...
Turn airplane mode on, open Spotify, go to Settings > Playback, set it to Offline, then turn airplane mode off.
At least now you can listen to cached music and browse the web again.
Longer term, install a Pi-Hole on your home network and block all Facebook owned domains, because they’re a scourge on humanity and your life will be much better void of their infection.
Seems like they’re no longer returning a string that older sdks expect and they haven’t handled the case properly back then.
The crash is:
Crash information
-[NSNull count]: unrecognized selector sent to instance 0x1cd6fe1e0
2020-07-10 17:24:26.052924+0800 OSDemo[4198:604792] * Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSNull count]: unrecognized selector sent to instance 0x1cd6fe1e0'
You can login to Spotify via Facebook as you can see in the web version: https://accounts.spotify.com/en/login/
For doing that in a app they require the SDK, which then does all the spying for Facebook.
Because FB doesn't support that. Either you build it in and it runs some code when your app starts or you can't use it.
I would vote for "not use it" but others want Facebook integration supported and happily share information on all their users to Facebook ...
P.S. I believe part of this is also Apple, who don't like runtime loading of code, as that makes their verification harder, but I'm no iOS dev or user.
445 comments
[ 2.6 ms ] story [ 308 ms ] threadAt least this time you can disable the SDK (at least one app developer has a kill switch for the SDK) unlike the last issue that happened during code initialisation you couldn’t skip.
"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." -- Leslie Lamport, 1987 (https://lamport.azurewebsites.net/pubs/distributed-system.tx...)
There's no such thing as QA at Facebook. Really. Developers are supposedly responsible for all testing of their own code, but all of the pressure is to "move fast" so that goes about as well as you'd expect.
The first two times it took months to fix. It’s happening again currently, too.
Clear there are GAPING holes in their test suites.
Make a lot of money selling lifetime subscriptions, then have nowhere to go for cash next year: look at all that money! I'm very successful now!
If all you're measuring and optimizing for is "valuation" or "revenue" you're not automatically going to get a good product or happy customers...
If you have a team run by idiots, then you won't have QA.
If you have a team run by adults, then there is plenty of QA. I mean its total shite that this wasn't caught. I'd expect much better, especially as this is the second time in n months. But then perhaps working for a company run by a dick who can't listen is getting to people.
Engineer in test. QA engineering lead. (https://www.facebook.com/careers/jobs/?q=QA) You must have leet scuba skills with searching skills like that.
AR/VR, Oculus, portal and workplace all have dedicated QA engineering teams. I'm sure there are more.
> Given your characterization of MZ
if it was only zuck. Zuck has two faults: he's not able to articulate any single thought inside of 15 minutes and he consistently throws out integrity products that would have stopped bad PR.
Its not just me, you should look at the micropulse. confidence in leadership has taken a good battering recently.
The whole juneteenth buisness, we had an entire _day_ of people telling us the problem. Senior management then sat next to them and said in reverent tones: "thank you so much [[dabs a tear]] for telling us your lived experience" posts a selfie and then does fuck all else.
Or, having revved up the company to generate hundreds of ideas to tackle abuse on FB platform, do shit all with them, not even acknowledge they exist.
Nothing to do with scuba, and if you did any operational work you'd know that it's in the category of tools that are often flaky due to lack of QA. It's also worth noting that the link you provide only shows 41 listings, and most of those merely mention "familiarity with QA" in some boilerplate for what's obviously a developer position. Leet reading skills you have there. Percentage-wise, there are fewer true QA positions there than we have E-10s. For a company that hires 10K a year, that's indistinguishable from zero.
> AR/VR, Oculus, portal and workplace
In other words some small percentage of the company, none of it near me. All of the systems that the profitable parts of Facebook rely on - to provision systems, monitor systems, store data, move data around, analyze data, etc. - rely entirely on developer testing. So here's an amended statement to make you happy.
"99.9% of Facebook has no QA, and the rest has really bad QA."
just over 10% of job openings have something to do with testing or QA. Of course QA is similar to software engineering, the whole point is to automate testing.
> [AR/VR, Oculus, portal and workplace..][..is] some small percentage of the company
yeah, go look at the headcount.
> none of it near me.
Now we are talking.
> rely entirely on developer testing
again, that wasn't the distinction I was making. It doesn't matter who's testing, so long as it gets done properly. Considering PE is a blended role. Its not a hard push to think that testing might be part of the role of a modern developer. Mind you, so is writing documentation, but thats not a priority either.
No, I don’t think there’s any “to be fair” in that, unless those SDKs have been officially EOL’d. Even then it shouldn't take down the whole app, Facebook should just break.
That this only happens with some SDKs certainly makes the problem less severe than last time, but only relatively.
Obs: I'm being sarcastic but since it's either this or developer laziness, I'm betting on tracking since I always start from the principle that the developers analysed the situation and made an explicit choice.
in our case, it's just a wrong decision, we don't use it for tracking.
also, if you use facebook login, the terms and conditions force you to use their SDK (which automatically initialises when included and does this crap).
https://developers.facebook.com/policy/ - 8.2
"Native iOS and Android apps that implement Facebook Login must use our official SDKs for login. "
In terms of the person writing the code, they can certainly push back. Being asked to add the Facebook SDK is either a red flag enough for him/her to do so, or they don't mind. In either case, they've made that choice.
True, you're just facilitating, not profiting.
Such a clear and obvious good reason to drop support and walk away.
developers.facebook.com/policy/ 8.2 (which states “Native IOS and Android apps that implement Facebook Login must use our official SDKs for login.” This has a help icon next to it that opens a dialog on their site that shows a couple pictures and then explains “Android apps should use the default login behavior defined by the SDK, which may use the web-view Login dialog. On iOS, only kiosk apps may use a web-view Login dialog.”
sometimes i wonder if you people lost all the connections with the reality... or if you were ever employed.
I've also refused to include Facebook in my life even though the majority of my extended family practically live there and rarely communicate outside of FB posts. My wife and I just get text messages on the rare occasions instead.
Yes, been employed and unemployed. And I'm under no illusions that FB login support will get dropped anytime soon. I can still lobby for the desirable alternative.
Not again...
If Facebook, which charges money for advertising, let go of controlling that tracking relationship, they’d lose a lot of money. So they won’t.
Thanks, send her the github issue.
Workaround for developers: Go to Facebook Developer page > Analytics and disable automatic Events.[0] Apparently this doesn't solve the issue for everyone.
Issue in Facebook's tracker: https://developers.facebook.com/status/issues/17391881029111...
[0] https://github.com/facebook/facebook-ios-sdk/issues/1427#iss...
[0] https://nextdns.io/
https://github.com/confirmedcode/lockdown-ios/blob/master/Lo...
https://github.com/confirmedcode/lockdown-ios/blob/master/Lo...
https://github.com/confirmedcode/lockdown-ios/blob/master/Lo...
https://github.com/confirmedcode/lockdown-ios/blob/master/Lo...
>It turns out that by just including the SDK with your app, Facebook runs hidden code on launch. (FBSDKApplicationDelegate.m)
[0] https://twitter.com/sandofsky/status/1258288056399847425
1. It is required for a "login with Facebook" feature, apparently. I'd presume this goes for other feautures that integrate with facebook, such as "invite from your friends" or "share on facebook(messenger)".
2. It gives access to metrics, when added to your own, greatly enrich your own data collection.
3. It is required when working with Facebook ads - though I would not know how this ties in with mobile apps: on web it is clearer (where you are required to host "the pixel" on your website in order to have ads on facebook link to that website.)
4. It is feature rich, so as a pure SDK, it offers neat utilities; though for each of these, there are better alternatives found in separate libs.
I don't remember apps listing the SDKs they use. As a humble user, how do I voluntarily choose whether to download an app or not based on their usage of Facebook's garbage?
Edit: You can't have it both ways. You can't remove a bunch of power and information from the user in the name of "safety" and "UX" and then claim that users are making a bunch of choices they can't possibly be informed about "voluntarily"
There are some intricacies on iOS surrounding null values and serialization, and as a developer it is important to understand that you may encounter NSNull. As a standard practice in my company, we type check every remote value before calling anything on it. Seems like that would have prevented this issue.
Source: have had that issue before
Also maybe it’s time for them to start rewriting things in Swift, where this would be much less likely to happen
It makes me wonder what all those engineers at FB are actually doing... ? Every time I tried to integrate or look into the FB SDK for simple functionality it was a total clusterfuck.
A bit of snark but also maybe stop trying to use a data collection engine to leverage anything for yourself?
If Apple wanted to make money , it should make it harder for advertising supported apps to make money and force apps to either not exist or get people to pay for them.
I would be okay with that.
How is this even possible, even less acceptable from Spotify point of view?
None of the apps broken on my phone require Facebook features for core functionality. Downtime happens, but downtime due to your non-essential & user-hostile ads/tracking/social SDK? These companies deserve every single cent of lost revenue.
IMO, blame Facebook.
It's a total joke that an app would crash on startup because of this idiocy.
Turn airplane mode on, open Spotify, go to Settings > Playback, set it to Offline, then turn airplane mode off.
At least now you can listen to cached music and browse the web again.
Longer term, install a Pi-Hole on your home network and block all Facebook owned domains, because they’re a scourge on humanity and your life will be much better void of their infection.
The crash is:
Crash information
-[NSNull count]: unrecognized selector sent to instance 0x1cd6fe1e0 2020-07-10 17:24:26.052924+0800 OSDemo[4198:604792] * Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSNull count]: unrecognized selector sent to instance 0x1cd6fe1e0'
"Lewisj489 is listening to Band on Spotify"
I would vote for "not use it" but others want Facebook integration supported and happily share information on all their users to Facebook ...
P.S. I believe part of this is also Apple, who don't like runtime loading of code, as that makes their verification harder, but I'm no iOS dev or user.
:)
Mild inconvenience for now, hopefully this will have more people move away from the FB SDK.