HN Explorer

Tell HN: My Hacker News app (HACK) is sort of broken this afternoon

44 points by busymom0 ↗ HN
I am the developer for HACK, a hacker news app on iOS, iPadOS, MacOS and Android.

As of this afternoon, I received lots of emails reporting a bug in the app where:

1. Tapping on a story was taking to the link for the story instead of the comments. There was no way to access the comments.

2. All stories showed "now" as the time posted instead of the correct time.

3. Threads didn't display the title for the parent post.

This occurred due to Hacker News making some changes to its website's HTML today.

Since the Hacker News API is pretty much useless (it's a read only API and doesn't allow voting, favoriting, commenting etc. Also very hard to work with), my app relies on the website which it parses to get the content. It does the parsing using XPath. Due to the changes in the HTML, the XPath in the app were no longer valid.

I have already submitted an update to Apple and the Play Store for fixing this bug. Hoping to have it approved soon (hopefully sometime tonight/tomorrow).

I am really sorry about the inconvenience to all the users who rely on my app.

The apps are:

https://apps.apple.com/ca/app/hack-for-hacker-news-reader/id1464477788

https://play.google.com/store/apps/details?id=com.pranapps.hack

Feel free to ask questions. Thanks.

31 comments

[ 2.8 ms ] story [ 80.9 ms ] thread
gunjx
Thanks for your hard work and fast reaction. Waiting patiently for the approval from the apple app store :-)
busymom0
Thank you. I have requested the "expedited" review from Apple too. But I think the "expedited" review process is slower than their regular review process...
Southland
I haven’t worked on mobile iOS development. My question is about you requiring to wait on the review process of the App Store for your fix. Would a design where you abstract your parsing logic etc behind your own API be allowed? Then you could make changes as required without being stuck in approval.
busymom0
The entire parsing logic couldn't be separated from the app efficiently I think. It's doable but would be hard to do it efficiently. Right now, the parsing happens in Swift (and Kotlin for Android). I could maybe switch the parsing to javascript and then download the parsing code over the internet, then use javascript bridge to do the parsing. But javascript bridge is slower than swift.

The better way I am thinking is to fetch the XPath/CSS query selection strings remotely on each app launch. Since fixes for the website changes usually involve me only having to modify the XPath for each element in my app, I could just modify the XPath on my server and then have my app fetch them on app launch. The XPath is just a string in the app, so it can be done.

Whether this is doable obviously depends upon how big of a change the HN website HTML has. I will look into it.

[deleted]
(comment deleted)
1_1xdev
Your idea to fetch the x path/selector from your api is a good one.

If the logic needs to change more substantially yeah you’re gonna have to ship an app change.

KMnO4
> But javascript bridge is slower than swift

How much slower are we talking? 20ms vs 9ms? I feel like we developers prematurely optimize critical paths without taking the actual timeliness of the objective into account.

busymom0
The parsing plays a major role and is already the bottleneck in the app.

When using the app, the more the number of comments a post has, the longer it takes. On older devices (iPhone 6S or cheaper Android phones for example), it can take 10-15 seconds to parse 500 comment posts. My app already has to take this into account and therefore only displays 20 comments right away and then adds additional comments as it's finishing parsing the rest.

Since the app has a "go to next top level comment thread" button, this button has to stay hidden until all 500 comments have been parsed. So any slowness in the parsing can significantly ruin the experience.

outime
I have no idea about iOS restrictions but wouldn’t it be easier to build an API that does the parsing for the app and if something breaks in the process, you get notified and fix it ASAP instead of having to push an update and wait for it to get approved? Hosting would be free or super cheap if you use Cloudflare Workers or similar places.
filoleg
It wasn't only HACK that got broken today. Around the same time as HACK stopped working properly, Refined HN[0] (browser extension) did the same for me on desktop Firefox. It mostly works, but there is no more "favorite" button (and other extra buttons) next to individual comments and probably other features I am missing (because I don't use them).

Not sure if it is an HN API change or not, but feels like it.

0. https://github.com/plibither8/refined-hacker-news

plibither8
Hey, author of Refined Hacker News. I've since fixed the bug and pushed updated versions to the Chrome Webstore and Mozilla Add-ons. The Chrome Webstore version should roll-out in a couple of days depending on how quickly they review and approve the extension. Thanks for the patience! :)
[deleted]
(comment deleted)
cachvico
So how come HN of all sites doesn't offer a straightforward API? Hello, elephant in the room...
busymom0
They actually even admit that the API garbage. They say:

> "The v0 API is essentially a dump of our in-memory data structures. We know, what works great locally in memory isn't so hot over the network. Many of the awkward things are just the way HN works internally. Want to know the total number of comments on an article? Traverse the tree and count. Want to know the children of an item? Load the item and get their IDs, then load them. The newest page? Starts at item maxid and walks backward, keeping only the top level stories. Same for Ask, Show, etc. I'm not saying this to defend it - It's not the ideal public API, but it's the one we could release in the time we had. While awkward, it's possible to implement most of HN using it."

https://github.com/HackerNews/API

If one wants to get the comments for a post on HN from the API, they have to first fetch the post, then fetch every single comment of the post 1 by 1. So if the post has 500 comments, one has to basically send over 500 network requests......

They did say that they plan on released a better API in the future. However, as far as I know, it will be read only initially. So won't be much useful for my app.

cachvico
I wonder if there's some kinda accelerator they could apply to for funding to fix the problem...
malshe
I am a heavy user of your app on the iPhone and iPad. It’s fantastic!
CharlesW
HACK is wonderful. I've tried several HN apps for iOS, and it's the best I've found.
aim4min
Could you scrape/pre-parse every article and comment on your own server, and then create your own better REST api for your app? One that doesn’t need as much parsing. Maybe you can even charge $ for access (for other apps)
busymom0
Unfortunately not. Since my app allows voting, commenting, favouriting etc, doing that on my server side won’t be possible as I don’t have access to the user authentication cookies.
[deleted]
(comment deleted)
raunak
How do you obtain authentication cookies for HN? Have them login once and store it in keychain?
filoleg
@busymom0 did it get broken today again? Comments on my profile and such all look fine, but the homepage now displays no stories (fully blank). And if I click on any saved comments and try to see the whole thread, it shows all comments, but no story/OP.

Thanks a lot for the app btw, pretty much one of the very few I use daily and actually cannot think of much in terms of complaints. Speed and the feel of non-slugishness is actually the main pro that no other HN app I saw do as well. Not sure if it is really significantly faster or just a perception due to animations done well, but it works lol.

busymom0
Yes :(

Looks like about 3 hours ago, Hacker News changed some HTML again.....no idea why they are doing that.

Fixed it. Already submitted the update to Apple and Google again. Hoping to have it approved soon.

I am really sorry about this inconvenience.

@dang is it possible to keep track of these changes? Or be notified of upcoming changes?

busymom0
All 3 versions of my app updates are now available to fix the bug. iOS/iPadOS, MacOS and Android app updates have all been approved finally.

Please give it a try and let me know if you have questions. The update version should be v86.86. If it's not visible yet on the app store, please give it an hour or so for it to become available.

Here's a direct link:

https://apps.apple.com/ca/app/hack-for-hacker-news-reader/id...

This should solve the bugs related to the HN news website changes.

obviyus
I can confidently say the app drawing up a blank reduced my HN usage by 90% : )

Thanks a ton for the quick update.

[deleted]
(comment deleted)
busymom0
I will be sending out an update which will start fetching the xPath strings remotely so it can be changed on the fly when the HTML changes.
[deleted]
(comment deleted)
bilekas
I'm gonna go out on a weird over-complicated obscure way of identifying the problem when it happens..

Depending on available resources of course, you could run regular checks of the xpath validity, with a mapping of of other identifiers, you could use those others to see the new xpath, auto update the code, call a script to compile and republish.

Thats an ugly way but doable.

Another would be to have a remote mapping, so when you open the app, you get the 'latest' mappings. This way you can still run your 'is_this_xpath_valid' and if not update the xml/json whatever. When the app opens, it downloads the latest version and continues on.

I say that option rather than a remote API because of sheer request load possible is wild, whereas a simple file is cheap.

If you're rich, have your cron/checker run a bit more regularly then and api that, like a cache checks each request, you can have it running in a passive lambda for example (not sure best way to optimize that one).

My long story short is, try to remove the dependency on the services you're consuming. In this case it's a weird one but if you want any help, you're not open source so dm, I love this stuff.