Correcting Group.num_comments counter error during upgrade to 24.8.0

[gpkvt]

Self-Hosted Version

24.4.2

CPU Architecture

x86_64

Docker Version

27.3.1

Docker Compose Version

2.29.7

Steps to Reproduce

1. checkout 24.8.0
2. run ./install.sh
3. migration fails with error (see actual result)

Expected Result

Successful upgrade to 24.8.0 so I can move on to 24.11.x. (Tried 24.11.1 already but got the same error.)

Actual Result

Running migrations for default
Operations to perform:
  Apply all migrations: auth, contenttypes, feedback, flags, hybridcloud, nodestore, remote_subscriptions, replays, sentry, sessions, sites, social_auth, uptime, workflow_engine
Running migrations:
  No migrations to apply.
Creating missing DSNs
Correcting Group.num_comments counter
Traceback (most recent call last):
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.InternalError_: posting list tuple with 2 items cannot be split at offset 25


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
    raise_the_exception(self.db, e)
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.InternalError_: InternalError_('posting list tuple with 2 items cannot be split at offset 25\n')
SQL: 
        UPDATE sentry_groupedmessage SET num_comments = (
            SELECT COUNT(*) from sentry_activity
            WHERE type = %s and group_id = sentry_groupedmessage.id
        )
    

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/.venv/bin/sentry", line 4, in <module>
    raise SystemExit(main())
                     ^^^^^^
  File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
    func(**kwargs)
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 147, in upgrade
    _upgrade(
  File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 88, in _upgrade
    call_command("sentry.runner.commands.repair.repair")
  File "/usr/src/sentry/src/sentry/runner/__init__.py", line 49, in call_command
    command.invoke(ctx)
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
    return ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/commands/repair.py", line 67, in repair
    fix_group_counters()
  File "/usr/src/sentry/src/sentry/silo/base.py", line 158, in override
    return original_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/runner/commands/repair.py", line 45, in fix_group_counters
    cursor.execute(
  File "/.venv/lib/python3.12/site-packages/sentry_sdk/utils.py", line 1788, in runner
    return sentry_patched_function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/sentry_sdk/integrations/django/__init__.py", line 653, in execute
    result = real_execute(self, sql, params)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/.venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
    raise_the_exception(self.db, e)
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
  File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
    return func(self, sql, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 84, in execute
    return self.cursor.execute(sql, clean_bad_params(params))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.InternalError: InternalError_('posting list tuple with 2 items cannot be split at offset 25\n')
SQL: 
        UPDATE sentry_groupedmessage SET num_comments = (
            SELECT COUNT(*) from sentry_activity
            WHERE type = %s and group_id = sentry_groupedmessage.id
        )
    
Error in install/set-up-and-migrate-database.sh:36.
'$dcr web upgrade --create-kafka-topics' exited with status 1
-> ./install.sh:main:37
--> install/set-up-and-migrate-database.sh:source:36

Event ID

No response