Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MDEV-35519 Populate thread_id of Backward Compatibility Query event #3707

Open
wants to merge 1 commit into
base: 11.7
Choose a base branch
from

Conversation

ParadoxV5
Copy link
Contributor

@ParadoxV5 ParadoxV5 commented Dec 16, 2024

  • The Jira issue number for this PR is: MDEV-35519

Description

This commit bubbles info->fdev to Query_log_event::begin_event so it can reconstruct the GTID event from binlog for real conversion.

What problem is the patch trying to solve?

For replicas that don't support GTIDs, a MariaDB primary will “convert” (read: replace) Gtid_log_events into Query_log_events of BEGIN queries.
The two events had no common fields previously, so this “conversion” simply zeroed all fields, thread_id among them.
That background changed when MDEV-7850 added thread_id records to GTID events as well, and now this “conversion” is dropping this field.

Release Notes

Compatibility replication to pre-GTID replicas now include Thread IDs.

How can this PR be tested?

TODO

And note too, that we don't have any MTR way to test backwards compatibility to Mysql.
@bnestere on Slack DM

Testing this conversion (in general) in MTR would require somehow tricking a GTID-oriented primary to think it’s replicating to a old-Maria/MySQL replica.

Basing the PR against the correct MariaDB version

  • This is a new feature or a refactoring, and the PR is based against the main branch.
  • This is a bug fix, and the PR is based against the earliest maintained branch in which the bug can be reproduced.

PR quality check

  • I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
  • For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.

@ParadoxV5 ParadoxV5 requested a review from bnestere December 16, 2024 04:24
enum_binlog_checksum_alg checksum_alg)
{
uchar *p= (uchar *)packet->ptr() + ev_offset;
uchar *q= p + LOG_EVENT_HEADER_LEN;
size_t data_len= packet->length() - ev_offset;
Gtid_log_event original_event= Gtid_log_event(p, data_len, fdev);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query_log_event::begin_event occurs during pre-send, which is layers away from where the GTID event originates.
Persumably, this is because replication works by sending primaries’ already-serialized binary logs to replicas as relay logs.

The GTID event has a fixed-size format comprising two variable-sized sections, the thread_id being the very last optional entry.
This means, if I don’t leverage the existing constructor, I’ll have to duplicate its parsing code.

Let’s pretend that, instead of overwriting the buffer with a blank BEGIN query, we’re finally starting to convert the GTID event for real.

@@ -1932,7 +1934,7 @@ Query_log_event::begin_event(String *packet, ulong ev_offset,
int2store(p + FLAGS_OFFSET, flags);

p[EVENT_TYPE_OFFSET]= QUERY_EVENT;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is LOG_EVENT_OFFSET a duplicate of EVENT_TYPE_OFFSET?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is actually meant to refer to the binlog magic header, though it seems that has somewhere along the way been confused with EVENT_TYPE_OFFSET. I wonder if LOG_EVENT_OFFSET should be removed, and its usages either changed to (depending on the usage) EVENT_TYPE_OFFSET (for extracting event type) or BIN_LOG_HEADER_SIZE (for usage in Rotate_log_event construction).

@andrelkin , @knielsen any thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
$ git grep -n LOG_EVENT_OFFSET
sql/log.cc:5904:    Rotate_log_event r(new_name + dirname_length(new_name), 0, LOG_EVENT_OFFSET,
sql/log_event.h:95:#define LOG_EVENT_OFFSET 4
sql/sql_repl.cc:2272:  DBUG_PRINT("info", ("log event code %d", (*packet)[LOG_EVENT_OFFSET+1] ));
sql/sql_repl.cc:2539:  event_type= (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET+ev_offset]);
sql/sql_repl.cc:2674:  event_type= (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET + ev_offset]);
sql/sql_repl.cc:2943:        (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET+ev_offset]);

Copy link
Contributor

@andrelkin andrelkin Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOG_EVENT_OFFSET should be removed ..?

Right. Somewhat I have not crossed this really bizarre const. Great catch!

sql/log_event.cc Outdated
@@ -1932,7 +1934,7 @@ Query_log_event::begin_event(String *packet, ulong ev_offset,
int2store(p + FLAGS_OFFSET, flags);

p[EVENT_TYPE_OFFSET]= QUERY_EVENT;
int4store(q + Q_THREAD_ID_OFFSET, 0);
int4store(q + Q_THREAD_ID_OFFSET, original_event.thread_id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add an extra condition here, to ensure original_events.flags_extra & FL_EXTRA_THREAD_ID must be set, in order to use original_event.thread_id, and otherwise 0. Where the current state of the code should guarantee that, let's make this check just in case, as we don't have any MTR testing of this, and we don't want this to silently break from future changes.

This is so we don't overwrite the thread_id for events that originate before MDEV-7850, e.g. from old binary logs, or in chain replication setups like MariaDB 11.4 -> MariaDB 11.5+ -> MySQL 5.7.

This commit bubbles `info->fdev` to `Query_log_event::begin_event`
so it can reconstruct the GTID event from binlog for *real* conversion.

`Gtid_log_event`s and `Query_log_event`s had no common fields previously,
but that changed when MDEV-7850 added Thread ID records to GTID events.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

4 participants