-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
gh-99813: Start using SSL_sendfile
when available
#99907
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Start using SSL_sendfile when available
SSL_sendfile
when available
Conflicts: Modules/clinic/_ssl.c.h
@gpshead could you please review this when you have a chance? |
I've recently pushed some changes after testing the new functionality manually by sending small and large files from Linux with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I won't have time to look at this in detail for a while. I'm just dropping a note about one issue seen while quickly skimming it.
I'd appreciate it if this can make it in 3.13 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments. The if-else if
blocks need to follow PEP-7 but you can also use a switch if you want (it can be clearer to read).
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
few final comments
with client_context.wrap_socket(socket.socket(), | ||
server_hostname=hostname) as s: | ||
s.connect((HOST, server.port)) | ||
# kTLS seems to work only with a connection created before |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you check whether it "seems" to work or not? (for a personal project I would accept this but I'm interested in knowing whether this is an issue with Python SSL or OpenSSL)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, I'll post more details about this soon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can reproduce the same issue with C code using OpenSSL (without Python). Please check my findings here. I mentioned a possible patch, let me know if we can apply it or there can be undesirable consequences.
FYI, ssl.OP_ENABLE_KTLS
to enable kTLS has been available since Python 3.12 (and it could be added to options as a simple integer even before). And the issue is not specific to SSL_sendfile
, it affects kTLS availability for simple reads and writes.
I added my reproducer to an existing OpenSSL issue openssl/openssl#19676 (comment).
Misc/NEWS.d/next/Library/2023-03-13-22-51-40.gh-issue-99813.40TV02.rst
Outdated
Show resolved
Hide resolved
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
@picnixz OpenSSL provides |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel we need explicit functions for common error messages just to always use the same ones (but it's a separate issue)
} | ||
else if (sockstate == SOCKET_HAS_BEEN_CLOSED) { | ||
PyErr_SetString(get_state_sock(self)->PySSLErrorObject, | ||
"Underlying socket has been closed."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just checking this:
- if the error message already exists, use the same text (namely capitalized text ending with a period)
- if not, try to see if error messages are using periods or not and/or are capitalized and pick the one that is the most common
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The exact error message exists in four other places in the file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding the message "The sendfile operation timed out" above without the period, it's siblings don't end with a period too:
- "The handshake operation timed out"
- "The write operation timed out"
- "The read operation timed out"
SSL_sendfile()
from OpenSSL 3.0 inSSLSocket.sendfile
when available. #99813New
_ssl__SSLSocket_sendfile_impl
function is very similar to_ssl__SSLSocket_write_impl
with a few adjustments for theSSL_sendfile
syscall.I am glad there was non-SSL
socket.socket.sendfile
and underlyingsocket.socket._sendfile_use_sendfile
implemented. I reused that code for newssl.SSLSocket._sendfile_use_ssl_sendfile
by moving shared logic tosocket.socket._sendfile_zerocopy
.And
test.test_ssl.ThreadedTests.test_sendfile
needed to have a socket bound before an SSL context wrapped it because this seemed to affect kTLS availability.