bufio: textproto's bufio design causes half closed idle connections on TCP connection closed by a remote server

[kayrus]

Long living (e.g. keepalive, connection reuse) textproto client connections don't get an event when a remote server closes a connection causing CLOSE_WAIT half closed TCP connections on the client side.

See an example: https://gist.github.com/kayrus/31023f2a1cc30b1744b3c45cd3557d21

$ go run .
220  ESMTP Service Ready
EHLO client
250-Hello client
250-PIPELINING
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-CHUNKING
250-AUTH PLAIN
250 SIZE
MAIL FROM:<[email protected]> BODY=8BITMIME
250 2.0.0 Roger, accepting mail from <[email protected]>
RCPT TO:<[email protected]>
250 2.0.0 I'll make sure <[email protected]> gets this
DATA
354 2.0.0 Go ahead. End your data with <CR><LF>.<CR><LF>
test
.
250 2.0.0 OK: queued
221 2.4.2 Idle timeout, bye bye
2022/07/18 10:51:43 tcp       34      0 127.0.0.1:59336         127.0.0.1:1025          CLOSE_WAIT  22336/smtp-test

I see that for an HTTP client go lang has a workaround. IMHO this solution is too complicated and I wonder whether it's possible to implement something general and reuse it for HTTP/SMTP/etc.

see also emersion/go-smtp#185

[seankhliao]

The CLOSE-WAIT is from your client never closing the connection.
Since you never interact with the connection after the close event, I don't see how this could be handled. The same goes for plain net.Conn / net.TCPConn connections.
If you want to maintain a connection pool, you'd be responsible for ensuring that an application level healthcheck/keepalive can still go through. HTTP/SMTP are very different protocols in that regard.

I don't see anything to do here.

[kayrus]

@seankhliao thanks for your reply. Nevertheless I disagree with you. Without bufio it is quite easy to detect that the connection is closed. But with bufio you always need to Peek(1) in advance. I don't know how exactly it could be done, I'm not an expert in OS TCP stack or bufio package, but I'd suggest to introduce some internal channel, which can detect in advance that the TCP connection is closed and an interface that notifies about the closed connection.

[ianlancetaylor]

The socket API does not provide any mechanism for detecting that a TCP socket is closed without reading from it.

[kayrus]

HTTP/SMTP are very different protocols in that regard

@seankhliao generally they are quite similar, they both use textproto wrapped around bufio. And you can see that http transport implementation peeks bufio in a loop (Peek(1)) to detect the closed connection.

@ianlancetaylor, my aim is to avoid reinventing the wheel and have a general implementation to detect a closed connection.

[ianlancetaylor]

@kayrus If you can find a general mechanism, then, great. I don't know of one.

[kayrus]

with my latest change to go-smtp where I'm checking for a connection + buffering what is read + write the buffer down the test looks better: the connection is closed.

the recent test (see https://gist.github.com/kayrus/31023f2a1cc30b1744b3c45cd3557d21) closes the connection properly.

I'm sure that the similar logic can be done in bufio. @ianlancetaylor what do you think?