HN Explorer

12 comments

[ 2.8 ms ] story [ 25.9 ms ] thread
dailypeeker
Is this Amazon's response for Kafka?
macksd
There's some overlap in use case, but Amazon already has a more compatible managed Kafka service called Amazon MSK.
bootlooped
"You can process up to 300 transactions per second (TPS) per FIFO topic or FIFO queue"

I want to say Kafka can do significantly higher messages per second on a single partition (and significantly higher than that on a many-partition topic). I don't think it can compete for a lot of use cases as a result. If I were in the market for managed Kafka, I think I would just go straight to Confluent.

baq
> providing strict message ordering and deduplicated message delivery to one or more subscribers.

this is hard yo

User23
I still have nightmares about rendezvous pubsub.
jcims
Kafka and SNS only conceptually intersect, SNS is more for intra-service and endpoint notification at huge scale (millions of subscribers) and is directly integrated into a variety of AWS services (e.g. S3 event notifications). You can set up a topic, subscribe a customer via SMS and send a message in just a few lines of python:

    $ python3
    >>> import boto3
    >>> sns=boto3.client('sns')
    >>> sns.create_topic(Name='DopeService')
    >>> sns.subscribe(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',Protocol='sms',Endpoint='+18008675309')
    >>> ...<repeat for other SMS #'s>...
    >>> sns.publish(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',Message='sup')
If you're a masochist you can have Amazon notify everyone on the topic every time someone uploads to your bucket

    >>> sns.set_topic_attributes(TopicArn='arn:aws:sns:us-east-1:xxxxxxxxxxxx:DopeService',AttributeName='Policy',AttributeValue='<policy>')
    >>> s3.put_bucket_notification_configuration(Bucket='dopebucket',NotificationConfiguration={<notification config>})
You send directly to HTTP/HTTPS service endpoints, other SNS topics, Lambdas, SQS queues and mobile app push notifications. SNS is more about signaling than a raw message pipe. Relative to Kafka it's basically apples and oranges.
pram
If you don't count Kinesis and their literal Kafka offering, MSK.
code4tee
SNS is intended for different use cases. Kinesis and other AWS products provide services in line with Kafka.
saurik
I would have assumed Google's pub/sub, which released a similar feature a few weeks ago.

https://news.ycombinator.com/item?id=24658132

That, or they both had convergent evolution and came to the same set of requirements, maybe from some big user at the same moment contacting both of them a year ago, which actually feels really likely to me (more likely than them responding to anything).

blahthriw
Anyone have a good guide to pubsub for CS students? I need data structures and algos, big o complexity. Most guides focus on implementation trivia.
nsonha
if you could please explain what pubsub has anything to do with those?
ledauphin
I'm a little surprised at the 300 TPS limit. It seems to me that they could have gone with a more Kafka-esque (hah! always wanted an excuse to write that) approach where the write limit is only relative to the message group/ordering id, i.e. a partition.

The thing Kafka doesn't handle so well is increasing the number of partitions dynamically, so in some sense this is just a different name for the abstraction - you could still create N topics upfront and it would be similar to Kafka. But then why have the message group id in the first place if not to allow the topic to scale beyond what a single node can handle?