Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -163,3 +163,5 @@ cython_debug/
data/

test.bmp
# Photoshop files
image-source/
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
rev: v5.0.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 22.6.0
rev: 25.1.0
hooks:
- id: black
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ COPY .env /cncnet-map-api/
# Copy files needed for build
COPY requirements.txt /cncnet-map-api
COPY requirements-dev.txt /cncnet-map-api
COPY start.sh /cncnet-map-api
COPY web_entry_point.sh /cncnet-map-api

RUN apt-get update && apt-get install -y liblzo2-dev # Compression library used by westwood.
RUN apt-get install libmagic1 # File type checking.
RUN pip install --upgrade pip
# The cflags are needed to build the lzo library on Apple silicon.
RUN CFLAGS=-I$(brew --prefix)/include LDFLAGS=-L$(brew --prefix)/lib pip install -r ./requirements-dev.txt

RUN chmod +x /cncnet-map-api/start.sh
ENTRYPOINT "/cncnet-map-api/start.sh"
RUN chmod +x /cncnet-map-api/web_entry_point.sh
ENTRYPOINT "/cncnet-map-api/web_entry_point.sh"
13 changes: 13 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#desolate:
# POSTGRES_DATA_DIR = $(shell cat .env | grep POSTGRES_DATA_DIR)

serve:
docker compose build django
docker compose up nginx-server

stop:
docker compose stop

test:
docker compose build django
docker compose run test
45 changes: 32 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,27 +17,32 @@ The mascot for the backend API is Kirovy, by [Direct & Dominate](https://www.you

## Frontend devs

Just set up your environment file and run the full docker compose.
Just set up your environment file and run `docker compose up web -d`.

This will launch the database, run the migrations, and start the django web server.

[Example env file](example.env)

## Backend devs

You can use the docker files if you'd like, but Django + docker is known to have issue attaching
to debuggers and hitting breakpoints, so here are the native OS instructions.
You can use docker compose if you'd like, but here are the native OS instructions.

### Linux and Mac

1. Download and install [pyenv](https://github.com/pyenv/pyenv)
> You don't have to use `pyenv` but it makes life much easier when dealing with virtual environments.
2. Install [PostgreSQL](https://www.postgresql.org/) for your system. This is required for Django
- On Mac you can do `brew install postgresql` if you have brew installed.
3. Install LibMagic for [Python Magic](https://github.com/ahupp/python-magic)
- On Mac you can do `brew install libmagic` if you have breq installed.
- On Mac you can do `brew install libmagic` if you have brew installed.
> LibMagic is used for checking file types.
4. Checkout the repository
5. Switch to the repository directory
6. Setup Python
- Install Python 3.12 `pyenv install 3.12` or whatever the latest python is.
- Setup the virtual environments `pyenv virtualenv 3.12 cncnet-map-api`
- Set the virtual enviornment for the directory `pyenv local cncnet-map-api`
7. Setup requirements `pip install -r requirements-dev.txt`
- Set the virtual environment for the directory `pyenv local cncnet-map-api`
7. Install the dev requirements `pip install -r requirements-dev.txt`
- On Apple Silicon you'll need to install lzo with `brew install lzo` then run
`CFLAGS=-I$(brew --prefix)/include LDFLAGS=-L$(brew --prefix)/lib pip install -r requirements-dev.txt`
to get `python-lzo` to install. You shouldn't need to include those flags again unless `python-lzo` updates.
Expand All @@ -49,23 +54,37 @@ to debuggers and hitting breakpoints, so here are the native OS instructions.
- If the app doesn't run due to a missing required variable, add said variable to `example.env` because the person
who made the variable forgot to do so.
10. Run the `db` service in `docker-compose`
11. Load your `.env` file into your shell, (you can use `source load_env.sh && read_env`)
11. Load your `.env` file into your shell, (you can use `source load_env.sh && read_env`)<a name="load-shell-env"></a>
then migrate the database `./manage.py migrate`
12. `./manage.py runserver`
12. Run the django server with `./manage.py runserver`

Tests can be run by following [these instructions](#running-tests-backend-devs)


### Windows

Chairman Bing of the Massivesoft corporation strikes again; getting the `LZO` libraries running
natively on Windows is a... less-than-pleasant effort. So use docker instead.

1. Install docker for windows. I have had success with [Rancher Desktop](https://rancherdesktop.io/)
or [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/)
2. After docker is running, switch to your local git repo and run `docker compose up windows-dev -d`.
Make sure the build succeeds.
3. Set `windows-dev` as your python interpreter for whichever editor you use.

You can technically use PyCharm to launch everything via `docker-compose`, but there is some
weird issue with breakpoints not triggering.
> [!TIP]
> In Pycharm you go to `Settings > Project > Python Interpreter > Add Interpreter > Docker Compose`


## Running tests
## Running tests (backend devs)

I **strongly** recommend using PyCharm and the `.env` plugin for running the PyTests.
I strongly recommend using PyCharm (or any other Python IDE with breakpoints) and the `.env` plugin for running the PyTests.
All you need to do is run the database from `docker-compose`, then launch the tests via PyCharm.

**If you want to run the tests via CLI:**

- Make sure your database is running from the docker compose file. `docker-compose start db`
- Make sure your environment variables are setup and loaded to your shell. See [backend dev setup](#backend-devs)
- Make sure your environment variables are setup and loaded to your shell. See [backend dev setup](#load-shell-env)
- Run `DJANGO_SETTINGS_MODULE="kirovy.settings.testing" pytest tests`

Django should automatically run migrations as part of the test startup.
Expand Down
4 changes: 2 additions & 2 deletions ci.env
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ POSTGRES_USER=kane
POSTGRES_PASSWORD=cidevelopmentpasswordtechnologyofpeace
POSTGRES_PORT=5432
POSTGRES_DATA_DIR=/data/db/
MEDIA_ROOT=/data/cnc_net_files/
STATIC_ROOT=/data/cnc_net_static/
HOST_MEDIA_ROOT=/data/cnc_net_files/
HOST_STATIC_ROOT=/data/cnc_net_static/
POSTGRES_TEST_HOST=db
SECRET_KEY=";thetechnologyofpeaceforcidevwork6352722!@#$$#@"
RUN_ENVIRONMENT=ci
40 changes: 35 additions & 5 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
version: "3.9"

services:
db:
image: postgres
Expand All @@ -14,18 +12,35 @@ services:
ports:
- "127.0.0.1:${POSTGRES_PORT}:${POSTGRES_PORT}"
command: -p ${POSTGRES_PORT}
web:

django:
# Won't serve files on its own. Launch nginx-server to run the whoe app.
build: .
volumes:
- .:/cncnet-map-api
- ${MEDIA_ROOT}:/data/cncnet_files
- ${STATIC_ROOT}:/data/cncnet_static
- ${HOST_MEDIA_ROOT}:/data/cncnet_silo # django will save user-uploaded files here. MEDIA_ROOT
- ${HOST_STATIC_ROOT}:/data/cncnet_static # django will gather static files here. STATIC_ROOT
ports:
- "8000:8000"
env_file:
- .env
depends_on:
- db

nginx-server:
# This is the prod server service.
# `docker compose up nginx -d` will run the whole app.
# nginx proxies requests to django via gunicorn.
image: nginx:latest
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ${HOST_STATIC_ROOT}:/usr/share/nginx/html/static # website static assets.
- ${HOST_MEDIA_ROOT}:/usr/share/nginx/html/silo # website user-uploaded files.
ports:
- "80:80"
depends_on:
- django

test:
build:
context: ./
Expand All @@ -40,3 +55,18 @@ services:
- db
command:
- pytest

windows-dev:
# Use this for windows. The LZO libraries are annoying to deal with without using docker.
# Chairman Bing of Massivesoft strikes again.
build:
context: ./
dockerfile: test.DockerFile
volumes:
- .:/cncnet-map-api
env_file:
- .env
environment:
POSTGRES_TEST_HOST: db # Necessary to connect to docker db. Overrides the .env setting.
depends_on:
- db
18 changes: 12 additions & 6 deletions example.env
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,18 @@ POSTGRES_TEST_HOST=localhost
# Port for postgres
POSTGRES_PORT=5432

# Required. The location where the app data (maps and such) will be saved. The docker volume mounts here
MEDIA_ROOT=./data/cncnet_files/

# Required. The location where static files will be served from.
# Make sure to configure the webserver to serve files from here and run ``./manage.py collectstatic``
STATIC_ROOT=./data/cncnet_static/
# Required. The location where the app data (maps and such) will be saved on the docker host.
# The docker volume mounts here.
# You should have this backed up because it's where user-uploaded files will live.
# nginx and django will serve these files under the url ``/silo/``
HOST_MEDIA_ROOT=./data/cncnet_silo/

# Required. The directory where collected static files will be saved on the docker host.
# The docker volume mounts here.
# ``collectstatic`` will gather all static assets here to be served by nginx.
# Remember to run ``./manage.py collectstatic`` during build so that this directory will be populated.
# nginx and django will serve these files under the url ``/static/``
HOST_STATIC_ROOT=./data/cncnet_static/

# Django debug, never enable on prod
DEBUG=0
Expand Down
10 changes: 10 additions & 0 deletions kirovy/exceptions/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""
All exceptions for our app belong in this package.
"""

from django.core.exceptions import * # Import django exceptions for use elsewhere.
from typing import Optional
from django.utils.translation import gettext_lazy as _
Expand Down Expand Up @@ -74,3 +75,12 @@ def __init__(
code: Optional[int] = None,
):
super().__init__(game_name_or_slug, detail, code)


class BanException(Exception):
"""Raised when there is an issue during the ban process.

``str(e)`` will be returned to the UI.
"""

pass
9 changes: 9 additions & 0 deletions kirovy/models/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
import typing

from django.db.models import UUIDField, Model

from .cnc_game import CncGame, CncFileExtension
from .cnc_map import CncMap, CncMapFile, MapCategory
from .cnc_user import CncUser
from .file_base import CncNetFileBaseModel
from .map_preview import MapPreview


class SupportsBan(typing.Protocol):
def set_ban(self, is_banned: bool, banned_by: CncUser) -> None:
...
8 changes: 7 additions & 1 deletion kirovy/models/cnc_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from kirovy.models import file_base
from kirovy.models import cnc_game as game_models, cnc_user
from kirovy.models.cnc_base_model import CncNetBaseModel
from kirovy import typing as t
from kirovy import typing as t, exceptions


class MapCategory(CncNetBaseModel):
Expand Down Expand Up @@ -160,6 +160,12 @@ def get_map_directory_path(self) -> pathlib.Path:
self.id.hex,
)

def set_ban(self, is_banned: bool, banned_by: cnc_user.CncUser) -> None:
if self.is_legacy:
raise exceptions.BanException("legacy-maps-cannot-be-banned")
self.is_banned = is_banned
self.save(update_fields=["is_banned"])


class CncMapFile(file_base.CncNetFileBaseModel):
"""Represents the actual map file that a Command & Conquer game reads."""
Expand Down
5 changes: 5 additions & 0 deletions kirovy/models/cnc_user.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,11 @@ def create_or_update_from_cncnet(user_dto: CncnetUserInfo) -> "CncUser":

return kirovy_user

def set_ban(self, is_banned: bool, banned_by: "CncUser") -> None:
# TODO: bannable objects should probably be an abstract class
self.is_banned = is_banned
self.save(update_fields=["is_banned"])


class CncNetUserOwnedModel(CncNetBaseModel):
"""A mixin model for any models that will be owned by a user."""
Expand Down
Loading