-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MDEV-35519 Populate thread_id of Backward Compatibility Query event #3707
base: 11.7
Are you sure you want to change the base?
Conversation
enum_binlog_checksum_alg checksum_alg) | ||
{ | ||
uchar *p= (uchar *)packet->ptr() + ev_offset; | ||
uchar *q= p + LOG_EVENT_HEADER_LEN; | ||
size_t data_len= packet->length() - ev_offset; | ||
Gtid_log_event original_event= Gtid_log_event(p, data_len, fdev); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Query_log_event::begin_event
occurs during pre-send, which is layers away from where the GTID event originates.
Persumably, this is because replication works by sending primaries’ already-serialized binary logs to replicas as relay logs.
The GTID event has a fixed-size format comprising two variable-sized sections, the thread_id being the very last optional entry.
This means, if I don’t leverage the existing constructor, I’ll have to duplicate its parsing code.
Let’s pretend that, instead of overwriting the buffer with a blank BEGIN
query, we’re finally starting to convert the GTID event for real.
@@ -1932,7 +1934,7 @@ Query_log_event::begin_event(String *packet, ulong ev_offset, | |||
int2store(p + FLAGS_OFFSET, flags); | |||
|
|||
p[EVENT_TYPE_OFFSET]= QUERY_EVENT; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is LOG_EVENT_OFFSET
a duplicate of EVENT_TYPE_OFFSET
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is actually meant to refer to the binlog magic header, though it seems that has somewhere along the way been confused with EVENT_TYPE_OFFSET
. I wonder if LOG_EVENT_OFFSET
should be removed, and its usages either changed to (depending on the usage) EVENT_TYPE_OFFSET
(for extracting event type) or BIN_LOG_HEADER_SIZE
(for usage in Rotate_log_event
construction).
@andrelkin , @knielsen any thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ git status
On branch main
Your branch is up to date with 'origin/main'.
nothing to commit, working tree clean
$ git grep -n LOG_EVENT_OFFSET
sql/log.cc:5904: Rotate_log_event r(new_name + dirname_length(new_name), 0, LOG_EVENT_OFFSET,
sql/log_event.h:95:#define LOG_EVENT_OFFSET 4
sql/sql_repl.cc:2272: DBUG_PRINT("info", ("log event code %d", (*packet)[LOG_EVENT_OFFSET+1] ));
sql/sql_repl.cc:2539: event_type= (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET+ev_offset]);
sql/sql_repl.cc:2674: event_type= (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET + ev_offset]);
sql/sql_repl.cc:2943: (Log_event_type)((uchar)(*packet)[LOG_EVENT_OFFSET+ev_offset]);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LOG_EVENT_OFFSET should be removed ..?
Right. Somewhat I have not crossed this really bizarre const. Great catch!
sql/log_event.cc
Outdated
@@ -1932,7 +1934,7 @@ Query_log_event::begin_event(String *packet, ulong ev_offset, | |||
int2store(p + FLAGS_OFFSET, flags); | |||
|
|||
p[EVENT_TYPE_OFFSET]= QUERY_EVENT; | |||
int4store(q + Q_THREAD_ID_OFFSET, 0); | |||
int4store(q + Q_THREAD_ID_OFFSET, original_event.thread_id); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add an extra condition here, to ensure original_events.flags_extra & FL_EXTRA_THREAD_ID
must be set, in order to use original_event.thread_id
, and otherwise 0
. Where the current state of the code should guarantee that, let's make this check just in case, as we don't have any MTR testing of this, and we don't want this to silently break from future changes.
This is so we don't overwrite the thread_id
for events that originate before MDEV-7850, e.g. from old binary logs, or in chain replication setups like MariaDB 11.4
-> MariaDB 11.5+
-> MySQL 5.7
.
4137a72
to
11bcbaa
Compare
This commit bubbles `info->fdev` to `Query_log_event::begin_event` so it can reconstruct the GTID event from binlog for *real* conversion. `Gtid_log_event`s and `Query_log_event`s had no common fields previously, but that changed when MDEV-7850 added Thread ID records to GTID events.
11bcbaa
to
8ffabc8
Compare
Description
This commit bubbles
info->fdev
toQuery_log_event::begin_event
so it can reconstruct the GTID event from binlog for real conversion.What problem is the patch trying to solve?
For replicas that don't support GTIDs, a MariaDB primary will “convert” (read: replace)
Gtid_log_event
s intoQuery_log_event
s of BEGIN queries.The two events had no common fields previously, so this “conversion” simply zeroed all fields,
thread_id
among them.That background changed when MDEV-7850 added
thread_id
records to GTID events as well, and now this “conversion” is dropping this field.Release Notes
Compatibility replication to pre-GTID replicas now include Thread IDs.
How can this PR be tested?
TODO
Testing this conversion (in general) in MTR would require somehow tricking a GTID-oriented primary to think it’s replicating to a old-Maria/MySQL replica.
Basing the PR against the correct MariaDB version
main
branch.PR quality check