Deploy codabench using kubernetes #2055
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hahahannes
reviewed
Jan 19, 2026
charts/Chart.yaml
Outdated
| dependencies: | ||
| - name: rabbitmq | ||
| version: "14.7.0" | ||
| repository: "oci://registry.cern.ch/kubeflow/charts" |
There was a problem hiding this comment.
We will replace this one with the original upstream chart https://hub.docker.com/r/bitnamicharts/rabbitmq. Same for redis below.
hahahannes
reviewed
Jan 19, 2026
| RUN poetry install | ||
|
|
||
| # Copy the rest of the application code | ||
| COPY . /app |
There was a problem hiding this comment.
Regarding the discussion about the Dockerfiles. We saw that the django component was missing the code (https://github.com/codalab/codabench/blob/develop/Dockerfile) also in the latest develop branch. Therefor we added node builder image here as well as the copying of the code.
As long as the container is self contained and does not required a volume mount, it will be compatible with this chart.
pauloguilhermepp
force-pushed
the
deploy_codabench_using_kubernetes
branch
from
5ade7ba to
a7a58fa
Compare
ObadaS
reviewed
Jan 26, 2026
| WORKDIR /app/ | ||
| COPY ./compute_worker/ ./ | ||
| COPY ./compute_worker/pyproject.toml ./ | ||
| COPY ./compute_worker/poetry.lock ./ |
Collaborator
There was a problem hiding this comment.
The previous 3 lines are already done on line 29 with the ADD compute_worker .
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Deploy Codabench Using Kubernetes
This PR aims to make Codabench deployable using Kubernetes.
Main Changes
To deploy Codabench using Kubernetes, we:
Issues this PR resolves
How to Test it
minikube start)values.yamlproperlyObservations
Tests mainly made at 1.19.
To expose the codabench UI, you will have to install an ingress controller and create an Ingress.
This PR does not include a template for Minio yet. For now, testing requires an external S3 instance. We can also add the Minio chart if needed.
The helm chart passes secret values directly via environment variables, which is not the best practice, as it will make them visible in deployment tools like ArgoCD. If you want, we can update this PR (or open a future one) with this change.
Inside values.yaml, there is a section (env) with environment variables that are currently passed to multiple pods. This is similar to the .env setup in docker-compose, but as we sometimes had a hard time debugging which variable is used where, we would like to separate those in a later PR.
This PR is making Codabench deployable using Kubernetes, but it is not changing how the compute-worker is running the user submissions. We plan to open another PR in the future, updating the compute-worker to run submissions using Kubernetes. We plan to make it configurable and using docker as default so that it will not break any current deployments. Please note that for the current way the compute worker works, the Kubernetes Pods need to mount a volume from the host node, to mount the Docker socket from the node, and run privileged Docker containers, which is not the best practice. This will mess with the storage on the node and will mess with the scheduling, as containers are directly created, bypassing the Kubernetes scheduler. Therefore, we will open another PR.
The PR also contains the Dockerfiles that we are using. Ideally, we should see if they are still compatible with deploying Codabench using the original docker-compose setup and keep a single Dockerfile for each component. The images were updated because, ideally, we don't want to mount the code from a PersistentVolume or the host node but rather have the image contain the code.
Checklist