Feature/s3 upload #309

kshitijrajsharma · 2024-12-12T08:44:09Z

What does this PR do ?

Adds s3 pmtiles option
Transfers all training files to s3 ( Currently folder produced at efs is not deleted , this should be done after testing s3 fully )
Introduces background task using django-q

…nable authentication and improve caching

…odel_training

…perties

kshitijrajsharma added 18 commits December 9, 2024 16:13

Added initial version of s3 upload transfer class

61525a3

Added working code for the s3 downloads and API response

4dc99df

Refactor TrainingWorkspaceView and TrainingWorkspaceDownloadView to e…

3e99467

…nable authentication and improve caching

Add checkpoint.tflite copy to ramp_model_training and clean up yolo_m…

a7befc7

…odel_training

Added pmtiles implementation with direct fetching from API

2d77e86

Change Max Char Value for Model Description

178a55c

Add h5 and tflite generation for ramp

9c5f818

Add GeoJSON file processing and validation in LabelViewSet

fe704d2

Add background processing for labels geojson

f0f468c

Implement asynchronous processing for raw data retrieval in AOI view

7af98de

Add django-q dependency for background task processing

42498ba

Add label upload endpoint and enhance label creation logic

4f3d5cd

Fix URL path for label upload endpoint

819e5aa

Added djago q

8696abc

Refactor label upload logic to include AOI ID directly in feature pro…

3b07c65

…perties

Added label upload view

471e181

Add instructions for starting background tasks in README

114ea30

Alter redis requirement

4db41ed

kshitijrajsharma marked this pull request as ready for review December 22, 2024 09:01

kshitijrajsharma merged commit b9d59c3 into develop Dec 22, 2024
2 checks passed

kshitijrajsharma deleted the feature/s3-upload branch December 22, 2024 09:02

kshitijrajsharma restored the feature/s3-upload branch December 22, 2024 09:04