A Crash Course in Designing Messaging Apis

The Kafka API has become a standard of sorts in the messaging space but sometimes it has been criticized for being too heavyweight. But is that true?

In this talk we're going to look at a variety of different messaging APIs, contrasting their features and guarantees with their ""heaviness"".

Do all messaging APIs support all features and guarantees?

• High scalability
• High availability and durability
• High throughput clients
• Message ordering
  • Total order, partial order, no ordering
• Processing guarantees
  • At-most-once, at-least-once, exactly-once
• Transactions
• Deduplication
• Log or queue semantics
• Replay
• Filtering/Routing

What about queue vs log semantics? What is the difference and when should we use one or the other? Does a distinction between the two really exist?

How do cloud provider messaging APIs compare to open-source messaging APIs? What does it mean for a messaging API to be cloud native?

By the end of this talk you'll have an idea of the various types of messaging APIs and the trade-offs the API designers made regarding the complexity of the APIs vs the features and guarantees the messaging systems provide.