Dev

[abelfx]

This pull request introduces Docker-based deployment and environment-driven configuration for a distributed Rust inference service. It adds Docker and CI support, enables running multiple worker nodes and a proxy via Docker Compose, and refactors the codebase to read addresses and ports from environment variables for flexible deployment. Additional code cleanups and a unit test for tensor aggregation logic are also included.

Deployment and Containerization:

• Added Dockerfile, .dockerignore, and docker-compose.yml to containerize the application and orchestrate multi-worker and proxy deployment. The Docker setup builds both inference_proxy and worker binaries and exposes required ports.

Environment-driven Configuration:

• Refactored src/main.rs, src/bin/client.rs, and src/bin/worker.rs to read proxy and worker addresses from environment variables (with sensible defaults), supporting both local development and containerized deployment.

Code Quality and Testing:

• Added a unit test for tensor aggregation logic in the batcher to ensure correctness of batching and splitting responses.
• Performed code style cleanups, improved error handling, and reordered imports for clarity and idiomatic Rust style in several modules.

These changes together streamline local and cloud deployment, improve maintainability, and lay the groundwork for scalable distributed inference.