AMQP or HTTP

 

A little background.

Very big monolithic Django application. All components use the same database. We need to separate services so we can independently upgrade some parts of the system without affecting the rest.

We use RabbitMQ as a broker to Celery.

Right now we have two options:

  • HTTP Services using a REST interface.
  • JSONRPC over AMQP to a event loop service

    • My team is leaning towards HTTP because that's what they are familiar with but I think the advantages of using RPC over AMQP far outweigh it.

    AMQP provides us with the capabilities to easily add in load balancing, and high availability, with guaranteed message deliveries.

    Whereas with HTTP we have to create client HTTP wrappers to work with the REST interfaces, we have to put in a load balancer and set up that infrastructure in order to have HA etc.

    With AMQP I can just spawn another instance of the service, it will connect to the same queue as the other instances and bam, HA and load balancing.

    Am I missing something with my thoughts on AMQP?

    At first,

  • REST, RPC - architecture patterns, AMQP - wire-level and HTTP - application protocol which run on top of TCP/IP
  • AMQP is a specific protocol when HTTP - general-purpose protocol, thus, HTTP has damn high overhead comparing to AMQP
  • AMQP nature is asynchronous where HTTP nature is synchronous
  • both REST and RPC use data serialization, which format is up to you and it depends of infrastructure. If you are using python everywhere I think you can use python native serialization - pickle which should be faster than JSON or any other formats.
  • both HTTP+REST and AMQP+RPC can run in heterogeneous and/or distributed environment

    • So if you are choosing what to use: HTTP+REST or AMQP+RPC, the answer is really subject of infrastructure complexity and resource usage. Without any specific requirements both solution will work fine, but i would rather make some abstraction to be able switch between them transparently.

    You told that your team familiar with HTTP but not with AMQP. If development time is an important time you got an answer.

    If you want to build HA infrastructure with minimal complexity I guess AMQP protocol is what you want.

    I had an experience with both of them and advantages of RESTful services are:

  • they well-mapped on web interface
  • people are familiar with them
  • easy to debug (due to general purpose of HTTP)
  • easy provide API to third-party services.

    • Advantages of AMQP-based solution:

  • damn fast
  • flexible
  • easy to maintain
  • easy to scale
  • cost-effective (in resources usage meaning)

    • Note, that you can provide RESTful API to third-party services on top of your AMQP-based API while REST is not a protocol but rather paradigm, but you should think about it building your AQMP RPC api. I have done it in this way to provide API to external third-party services and provide access to API on those part of infrastructure which run on old codebase or where it is not possible to add AMQP support.

    If I am right your question is about how to better organize communication between different parts of your software, not how to provide an API to end-users.

    If you have a high-load project RabbitMQ is damn good piece of software and you can easily add any number of workers which run on different machines. Also it has mirroring and clustering out of the box. And one more thing, RabbitMQ is build on top of Erlang OTP, which is high-reliable,stable platform ... (bla-bla-bla), it is good not only for marketing but for engineers too. I had an issue with RabbitMQ only once when nginx logs took all disc space on the same partition where RabbitMQ run.

    The irony of the solution OP had to accept is, AMQP or other MQ solutions are often used to insulate callers from the inherent unreliability of HTTP-only services -- to provide some level of timeout & retry logic and message persistence so the caller doesn't have to implement its own HTTP insulation code. A very thin HTTP gateway or adapter layer over a reliable AMQP core, with option to go straight to AMQP using a more reliable client protocol like JSONRPC would often be the best solution for this scenario.

    链接地址: http://www.djcxy.com/p/34212.html

    上一篇: 像“在RabbitMq上消费？

    下一篇: AMQP或HTTP