Helm: problem with psql in 2.36.0

[kiblik]

Problematic combination: Django 4.2 and PSQL 11

Logs:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 330, in cursor
    return self._cursor()
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 306, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 289, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 272, in connect
    self.init_connection_state()
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 309, in init_connection_state
    super().init_connection_state()
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 239, in init_connection_state
    self.check_database_version_supported()
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 214, in check_database_version_supported
    raise NotSupportedError(
django.db.utils.NotSupportedError: PostgreSQL 12 or later is required (found 11.22).

Issue is under investigation

[kiblik]

Simple upgrade is not possible because of

postgresql 16:17:34.77 INFO  ==> ** Starting PostgreSQL **
2024-07-02 16:17:34.796 GMT [1] FATAL:  database files are incompatible with server
2024-07-02 16:17:34.796 GMT [1] DETAIL:  The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 16.3.

[mtesauro]

@kiblik
PostgreSQL 11 is pretty old - in fact it's no longer supported as of November 9, 2023 by the PostgreSQL project -https://www.postgresql.org/support/versioning/

Do we have PostgreSQL 11 somewhere in our Helm charts? If so, we really shouldn't.

I do recall a long time ago when I updated the public demo (demo.defectdojo.org) from PostgreSQL from 11 to 12, there was a change that required a migration. I tend to update PosgreSQL on the public demo regularly - I think it's currently running PostgreSQL 16.

I had to look it up but apparently there's a tool to update from the 11 data storage format to later ones called pg_upgrade - a decent article is at https://dev.to/jkostolansky/how-to-upgrade-postgresql-from-11-to-12-2la6

[kiblik]

@kiblik PostgreSQL 11 is pretty old - in fact it's no longer supported as of November 9, 2023 by the PostgreSQL project -https://www.postgresql.org/support/versioning/

Do we have PostgreSQL 11 somewhere in our Helm charts? If so, we really shouldn't.

I do recall a long time ago when I updated the public demo (demo.defectdojo.org) from PostgreSQL from 11 to 12, there was a change that required a migration. I tend to update PosgreSQL on the public demo regularly - I think it's currently running PostgreSQL 16.

I had to look it up but apparently there's a tool to update from the 11 data storage format to later ones called pg_upgrade - a decent article is at https://dev.to/jkostolansky/how-to-upgrade-postgresql-from-11-to-12-2la6

I fully agree, PostgreSQL 11 is old.
Unfortunately, it is hardcoded in the helm chart right now:

django-DefectDojo/helm/defectdojo/values.yaml

Line 372 in 659f50b

tag: 11.22.0-debian-11-r4

The start of a freshly new instance with a new database is not problematic. However, there is an issue with upgrading to 12+ (for users which have been already using DD).
I was also searching for some elegant solution for migration but haven't been successful.

[kiblik]

I would probably:

• drop hardcoder value from helm chart
• write manual for migration between databases - it will be useful not only for migration from Postgresql 11 to 12+ but from MySQL to Postgresql as well

[kiblik]

I would probably:

• drop hardcoder value from helm chart
• write manual for migration between databases - it will be useful not only for migration from Postgresql 11 to 12+ but from MySQL to Postgresql as well

Same as #10359 (comment)

[kiblik]

Issue has been solved in #10507 and released as 2.36.1 (HELM chart 1.6.138)
Described how to upgrade in #10524 and published in https://documentation.defectdojo.com/getting_started/upgrading/2.36/