You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the schema validation checks .data exists but not it's contents for INamedFile and INamedImage.
But since this is implemented as a property decorator this validation will load the whole file into memory. If this is a large file or you are doing a big import, this can slow down the process considerably as often the data is transferred using fs cp.
djay
changed the title
imports/uploads slow due to reading files into memory due to schema validation
imports/uploads slow due to reading files into memory due to schema validation and exif (for images)
Dec 2, 2024
Currently the schema validation checks
.data
exists but not it's contents for INamedFile and INamedImage.But since this is implemented as a property decorator this validation will load the whole file into memory. If this is a large file or you are doing a big import, this can slow down the process considerably as often the data is transferred using fs cp.
I have a PR that avoids this check. #166
This might also be able to be fixed in zope.schema itself - zopefoundation/zope.schema#127
In addition for images the file is being read for unnessarily for exif and content-type extraction. The attached PR also fixes this.
The text was updated successfully, but these errors were encountered: