Celery Broker

|
| By Webner

A celery broker is an intermediary between a producer and a consumer of messages in a distributed system. It acts as an intermediary to facilitate communication between the two parties. Celery brokers are usually used with message queues, such as RabbitMQ, Redis, or Apache Kafka. The purpose of the broker is to provide reliable delivery of messages from the producer to the consumer.

What are message queues?

A message queue is a type of broker that Celery uses to pass messages between tasks. It stores the messages in an easily accessible format and allows for asynchronous processing, meaning that tasks can be executed independently of each other without one task having to wait for another before it can start. Message queues are important components of distributed systems such as Celery because they allow multiple processes (or machines) to communicate with each other in an efficient manner.

Web and worker nodes

Web nodes are responsible for managing and monitoring the Celery cluster. They typically run a web-based dashboard that allows users to monitor job progress, manage worker nodes in the cluster, and view task results. Worker nodes are responsible for executing tasks that have been assigned to them by the web node. Worker nodes can be configured to execute multiple types of tasks concurrently, or they can focus on one type of task at a time.

Executing Task

  1. Define the Task:
    First, you need to define a task that can be executed by Celery. This is done using the
    @app.task decorator and providing an argument for the name of the task. Here’s an example of how to do this:
    from celery import Celery
    app = Celery('tasks', broker='amqp://guest@localhost//')
    @app.task(name="some_task")
    def some_task():
    # Do something here...
    pass
  2. Executing a Task:
    Once your tasks have been defined, you can execute them using either apply_async() or delay
    from celery import Celery
    app = Celery('tasks', broker='pyamqp://guest@localhost//')
    @app.task
    def subtract(x, y):
    return x - y

Leave a Reply

Your email address will not be published. Required fields are marked *