Learn how to simply combine Ray Serve Deployment with asynchronous Kafka Consumer
Ray is a modern open source framework that allows you to create distributed applications in Python with ease. You can create simple training pipelines, do hyperparameter tuning, data processing and model serving.
Ray allows you to create online inference APIs with Ray Serve. You can easily combine several ML models and custom business logic in one application. Ray Serve automatically creates an HTTP interface for your deployments, taking care of fault tolerance and replication.
But there is one thing that Ray Serve misses for now. Many modern distributed applications communicate through Kafka, but there is no out-of-the-box way to connect Ray Serve service to Kafka topic.
But don’t panic. It will not take too much effort to teach Ray Serve to communicate with Kafka. So, let’s begin.
First of all we will need to prepare our local environment. We will use a docker-compose file with Kafka and Kafdrop UI docker containers to start and explore our local Kafka instance (so we assume that you have Docker and Docker Compose installed). Also we will need to install some Python requirements to get the work done:
All requirements can be downloaded by this link.
Now we will create a ray-consumer.py file with Ray Deployment that will be served with Ray Serve. I will not go into details about Ray Serve concepts, as you can read about that in the documentation. Basically it takes the usual Python class and converts it to a asynchronous service Ray Deployment with @serve.deployment decorator: