Open-Sourcing Yahoo’s Pulsar, Pub-Sub Messaging at Scale

By Joe Francis and Matteo Merli, Yahoo PlatformsPub-sub messaging is a very common design pattern that is increasingly found in distributed systems powering Internet applications. These applications provide real-time services, and need publish-latencies of 5ms on average and no more than 15ms at the 99th percentile. At Internet scale, these applications require a messaging system with ordering, strong durability, and delivery guarantees. In order to handle the “five 9’s” durability requirements of a production environment, the messages have to be committed on multiple disks or nodes.At the time we started, we could not find any existing open-source messaging solution that could provide the scale, performance, and features Yahoo required to provide messaging as a hosted service, supporting a million topics. So we set out to build Pulsar as a general messaging solution, that also addresses these specific requirements.Pulsar is a highly scalable, low latency pub-sub messaging system running on commodity


Original URL: http://feedproxy.google.com/~r/feedsapi/BwPx/~3/iCneDGViPTE/open-sourcing-pulsar-pub-sub-messaging-at-scale

Original article

Comments are closed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: