Configure default related models for new projects #905

mihow · 2025-07-22T23:58:57Z

Summary

Currently there are several manual steps required each time a new project is created. This PR introduces several changes to streamline the creation of new projects, both for local development and in live environments for users.

List of Changes

Automatically create a Default Station (deployment) and assign the Default Research Site & Default Device
Automatically create or add a Processing Service, if specified in the environment settings (which is for local dev)
Allow specifying a list of pipelines that should be enabled/disabled by default (e.g. limit the choices of models)
Automatically create a Default Collection with all images (will not auto-populate)

TODO: set default lat & lon based on current country in the Request
TODO: create a default project when a user is registered (a hidden scratch project) #899
TODO: allow setting a default pipeline for a project to use for processing (multiple are enabled & available, even after this PR, so which should be used for automatic processing) #901
TODO: auto-populate some collections #907

Related Issues

This was added as part of #901, and it closes part of #899

Related #893

Screenshots

Deployment Notes

Copy the new DEFAULT_* settings from the .production/.django-example config file to the live environments if desired.

Checklist

I have tested these changes appropriately.
I have added and/or modified relevant tests.
I updated relevant documentation or comments.
I have verified that this PR follows the project's coding standards.
Any dependent changes have already been merged to main.

netlify · 2025-07-22T23:59:01Z

✅ Deploy Preview for antenna-preview canceled.

Name	Link
🔨 Latest commit	`7a80de4`
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6887fc0a8c5baf0007db6ffc

Copilot

Pull Request Overview

This PR streamlines the creation of new projects by automatically configuring default related models and services. It introduces environment-driven configuration for processing services and pipelines to reduce manual setup steps for both local development and production environments.

Automatically creates default related models (Station/Deployment, Research Site, Device, Collection) when projects are created
Adds configurable default processing service creation based on environment variables
Implements selective pipeline enabling/disabling through configuration settings

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
config/settings/base.py	Adds environment variables for default processing service configuration
ami/tests/fixtures/main.py	Updates test fixtures to handle new project creation patterns and return project identifiers
ami/ml/models/project_pipeline_config.py	Removes unused imports
ami/ml/models/processing_service.py	Adds custom manager, pipeline filtering logic, and default service creation functionality
ami/main/tests.py	Adds comprehensive tests for new project setup functionality with various configuration scenarios
ami/main/models.py	Implements custom ProjectManager with automatic default model creation and refactors related helper functions
.envs/.production/.django-example	Adds production environment configuration for default processing service
.envs/.local/.django	Adds local development environment configuration for default processing service

ami/tests/fixtures/main.py

ami/ml/models/processing_service.py

ami/main/models.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

.envs/.production/.django-example

mohamedelabbas1996 · 2025-07-28T15:30:58Z

Nice work! Quick question:is default project creation triggered from the admin or the project view? Also, would using a JSON/YAML config for defaults be more flexible than env vars?

mohamedelabbas1996 · 2025-07-28T15:32:11Z

One more thought: what about adding a few default images to the default deployment so users can immediately experiment with processing pipelines?

mihow · 2025-07-28T22:49:02Z

Thank you @mohamedelabbas1996!

I agree with all your points.

The default projects should use a config file if we add any more options. This would be a great way to make project templates! So users can essentially clone a project with the same device configs, models, data sources, etc already configured for them.
The default project will be created when a user is first created. For now, it's only created in dev when you start with an empty database (after the migrations are run a signal is triggered).
I like the idea of a few images already in one of the stations, to make getting started really easy.

I added all of these ideas to the implementation details of this ticket.
#899

I am going to merge this PR while it's all simple & safe changes.

* feat: configure default related models with new projects * feat: allow specifying which pipelines are enabled by default * feat: add denmark/uk model to default pipelines * feat: add tests for default enabled pipelines * chore: rename default station * fix: undefined variables in certain cases * chore: cleanup typos and comments * Update ami/ml/models/processing_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: live processing endpoint URL

sentry-io · 2025-07-29T20:04:24Z

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

‼️ HTTPError: 404 Client Error: Not Found for url: ht... in production

_{Did you find this useful? React with a 👍 or 👎}

annavik · 2025-08-05T08:01:21Z

Exciting! Looking forward to check this out.

* feat: clean up when events are regrouped for a deployment * feat: add created & updated at columns to sessions/events list * feat: configure default related models with new projects * feat: allow specifying which pipelines are enabled by default * feat: add denmark/uk model to default pipelines * feat: add tests for default enabled pipelines * chore: rename default station * fix: undefined variables in certain cases * chore: cleanup typos and comments * fix: update default sampling method for collections * Configure default related models for new projects (#905) * feat: configure default related models with new projects * feat: allow specifying which pipelines are enabled by default * feat: add denmark/uk model to default pipelines * feat: add tests for default enabled pipelines * chore: rename default station * Update ami/ml/models/processing_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: live processing endpoint URL * feat: process images immediately after uploading (prototype) * feat: read timestamp from EXIF data in special cases * chore: clean up comments and unused * feat: query method to select pipelines enabled for project * feat: process_single_image function in a new home * fix: default pipeline query * feat: fallback to the current datetime for test uploads * chore: disable auto-processing manual uploads by default * fix: select only pipelines with an avail processor for the project * feat: move the create method to the view * fix: allow the current project to be passed in post / form data * feat: require project in source image upload * feat: use project feature flag for auto processing * fix: pass project ID when creating source image in test * fix: separate titles for source images & source image collections * feat: use default pipeline in project settings first

feat: configure default related models with new projects

bfc2944

mihow changed the title ~~Configure default related models with new projects~~ Configure default related models for new projects Jul 22, 2025

mihow added 2 commits July 24, 2025 17:08

feat: allow specifying which pipelines are enabled by default

deb0528

feat: add denmark/uk model to default pipelines

ef8f27e

mihow marked this pull request as ready for review July 25, 2025 00:29

Copilot AI review requested due to automatic review settings July 25, 2025 00:29

This comment was marked as outdated.

Sign in to view

mihow requested a review from Copilot July 25, 2025 01:27

This comment was marked as outdated.

Sign in to view

mihow added 4 commits July 24, 2025 18:42

feat: add tests for default enabled pipelines

48ee1b2

chore: rename default station

5f72480

fix: undefined variables in certain cases

08bc992

chore: cleanup typos and comments

599f48f

mihow requested a review from Copilot July 25, 2025 01:43

Copilot AI reviewed Jul 25, 2025

View reviewed changes

ami/tests/fixtures/main.py Show resolved Hide resolved

ami/ml/models/processing_service.py Outdated Show resolved Hide resolved

ami/main/models.py Show resolved Hide resolved

Update ami/ml/models/processing_service.py

35e0f5d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

mihow commented Jul 25, 2025

View reviewed changes

.envs/.production/.django-example Outdated Show resolved Hide resolved

fix: live processing endpoint URL

7a80de4

mihow merged commit 41eb1e6 into main Jul 28, 2025
6 checks passed

mihow deleted the feat/quickstart-prototype branch July 28, 2025 22:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Configure default related models for new projects #905

Configure default related models for new projects #905

Uh oh!

mihow commented Jul 22, 2025 •

edited

Loading

Uh oh!

netlify bot commented Jul 22, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mohamedelabbas1996 commented Jul 28, 2025

Uh oh!

mohamedelabbas1996 commented Jul 28, 2025

Uh oh!

mihow commented Jul 28, 2025

Uh oh!

Uh oh!

sentry-io bot commented Jul 29, 2025

Uh oh!

annavik commented Aug 5, 2025

Uh oh!

Uh oh!

Configure default related models for new projects #905

Configure default related models for new projects #905

Uh oh!

Conversation

mihow commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

List of Changes

Related Issues

Screenshots

Deployment Notes

Checklist

Uh oh!

netlify bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for antenna-preview canceled.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mohamedelabbas1996 commented Jul 28, 2025

Uh oh!

mohamedelabbas1996 commented Jul 28, 2025

Uh oh!

mihow commented Jul 28, 2025

Uh oh!

Uh oh!

sentry-io bot commented Jul 29, 2025

Suspect Issues

Uh oh!

annavik commented Aug 5, 2025

Uh oh!

Uh oh!

mihow commented Jul 22, 2025 •

edited

Loading

netlify bot commented Jul 22, 2025 •

edited

Loading