Support for directly reading standard JPEG input images

[HansBull]

Thank you very much for this new feature!
Just a note: this works (in our usecases) with acceptable speed for baseline jpgs, but not for files with progressive encoding, where processing is excruciatingly slow.

[ruven]

Unfortunately, progressive scans are simply longer to decode, so there's not so much that can be done about it.

But as the image is decoded progressively through quality layers, it would be possible to limit the number of progressive layers that are decoded, which would speed things up a bit. This, however, would come at the cost of a degraded image quality - the amount of degradation would depend on how many layers you skip.

[HansBull]

There is something that makes iipsrv abort with directly reading jpeg, after having served a couple of tiles of a jpeg with 26754x25373 pixels:

Sep 28 08:56:25 server08 systemd[1]: iipsrv.service: Main process exited, code=killed, status=6/ABRT
Sep 28 08:56:25 server08 systemd[1]: iipsrv.service: Failed with result 'signal'.
Sep 28 08:56:25 server08 systemd[1]: iipsrv.service: Consumed 57.377s CPU time.

[HansBull]

Sep 28 09:09:32 server08 systemd[1]: iipsrv.service: Failed with result 'signal'.
Sep 28 09:09:33 server08 systemd[1]: iipsrv.service: Scheduled restart job, restart counter is at 4.
Sep 28 09:09:33 server08 systemd[1]: Stopped IIP.
Sep 28 09:09:33 server08 systemd[1]: Started IIP.
Sep 28 09:09:33 server08 systemd[1]: iipsrv.service: Main process exited, code=killed, status=11/SEGV
Sep 28 09:09:33 server08 systemd[1]: iipsrv.service: Failed with result 'signal'.
Sep 28 09:09:33 server08 kernel: [403049.148174] iipsrv.fcgi[1070680]: segfault at 7f04020d1000 ip 00007f0401fa0aca sp 00007fff661f42a8 error 7 in libc.so.6[7f0401e28000+195000] likely on CPU 7 (core 3, socket 0)
Sep 28 09:09:33 server08 kernel: [403049.148196] Code: 00 01 00 00 00 74 99 83 f9 c0 0f 87 7b fe ff ff c5 fe 6f 4e 20 48 29 fe 48 83 c7 3f 49 8d 0c 10 48 83 e7 c0 48 01 fe 48 29 f9 a4 c4 c1 7e 7f 00 c4 c1 7e 7f 48 20 c5 f8 77 c3 0f 1f 44 00 00
Sep 28 09:09:33 server08 systemd[1]: iipsrv.service: Scheduled restart job, restart counter is at 5.
Sep 28 09:09:33 server08 systemd[1]: Stopped IIP.
Sep 28 09:09:33 server08 systemd[1]: Started IIP.
Sep 28 09:09:34 server08 kernel: [403049.607155] iipsrv.fcgi[1070688]: segfault at 7f921a0fc000 ip 00007f9219fa0aca sp 00007ffcf2597c08 error 7 in libc.so.6[7f9219e28000+195000] likely on CPU 4 (core 0, socket 0)
Sep 28 09:09:34 server08 kernel: [403049.607172] Code: 00 01 00 00 00 74 99 83 f9 c0 0f 87 7b fe ff ff c5 fe 6f 4e 20 48 29 fe 48 83 c7 3f 49 8d 0c 10 48 83 e7 c0 48 01 fe 48 29 f9 a4 c4 c1 7e 7f 00 c4 c1 7e 7f 48 20 c5 f8 77 c3 0f 1f 44 00 00

[ruven]

Firstly, I would definitely NOT recommend using JPEG images of this size! 4kx4k is probably the highest that iipsrv can handle efficiently when using JPEG.

What does it say in the iipsrv log file? And does this occur for all your images over a certain size or is there one particular image causing problems? If so, is there a way to send this to me to test it out? Is this image a progressive JPEG?

[HansBull]

Of course I understand it's not recommended ...
Just tried with another one of 25576x13230, the smallest I could find. Same result. All baseline.

Now syslog gives
Sep 28 14:48:15 server08 iipsrv.fcgi[1091417]: double free or corruption (!prev)
Sep 28 14:48:15 server08 systemd[1]: iipsrv.service: Main process exited, code=killed, status=6/ABRT

Compiled with JPEG_DEBUG, here is the relevant logfile extract
JPEG :: Full resolution : 26754x25373
JPEG :: Virtual resolution : 13377x12686
JPEG :: Virtual resolution : 6688x6343
JPEG :: Virtual resolution : 3344x3171
JPEG :: Virtual resolution : 1672x1585
JPEG :: Virtual resolution : 836x792
JPEG :: Virtual resolution : 418x396
JPEG :: Virtual resolution : 209x198
JPEG :: 8 bit data
JPEG :: 3 channels
JPEG :: 1x1 pixels/cm
JPEG :: loadImageInfo() :: 263 microseconds
JPEG :: openImage() :: 428 microseconds
JPEG :: getRegion() :: 710514 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 659734 microseconds
JPEG :: closeImage() :: 33 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 510898 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 29 microseconds
JPEG :: getRegion() :: 600051 microseconds
JPEG :: closeImage() :: 26 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 620471 microseconds
JPEG :: closeImage() :: 23 microseconds
JPEG :: openImage() :: 33 microseconds
JPEG :: getRegion() :: 676840 microseconds
JPEG :: closeImage() :: 21 microseconds
JPEG :: openImage() :: 31 microseconds
JPEG :: getRegion() :: 697771 microseconds
JPEG :: closeImage() :: 29 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 686913 microseconds
JPEG :: closeImage() :: 25 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 763687 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 779675 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 841132 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 691506 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 781593 microseconds
JPEG :: closeImage() :: 6 microseconds
JPEG :: openImage() :: 22 microseconds
JPEG :: getRegion() :: 634685 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 704787 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 771993 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 822237 microseconds
JPEG :: closeImage() :: 6 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 804594 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 646785 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 29 microseconds
JPEG :: getRegion() :: 750989 microseconds
JPEG :: closeImage() :: 10 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 687053 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 808021 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 23 microseconds
JPEG :: getRegion() :: 655041 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 882193 microseconds
JPEG :: closeImage() :: 6 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 879536 microseconds
JPEG :: closeImage() :: 6 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 888944 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 895115 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 32 microseconds
JPEG :: getRegion() :: 815362 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 32 microseconds
JPEG :: getRegion() :: 871273 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 897670 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 59 microseconds
JPEG :: getRegion() :: 727445 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 961149 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 961787 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 839310 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 959618 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 904305 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 973411 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 23 microseconds
JPEG :: getRegion() :: 913918 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 975429 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 44 microseconds
JPEG :: getRegion() :: 1036516 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 1028488 microseconds
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 842268 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 1026258 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 781339 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 1028114 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 780701 microseconds
JPEG :: closeImage() :: 6 microseconds
JPEG :: openImage() :: 26 microseconds
JPEG :: getRegion() :: 901329 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 36 microseconds
JPEG :: getRegion() :: 961874 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 837876 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 29 microseconds
JPEG :: getRegion() :: 1020614 microseconds
JPEG :: closeImage() :: 12 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 775643 microseconds
JPEG :: closeImage() :: 8 microseconds
JPEG :: openImage() :: 24 microseconds
JPEG :: getRegion() :: 915494 microseconds
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 972690 microseconds
CVT :: Error flushing output data
JPEG :: closeImage() :: 7 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 859778 microseconds
CVT :: Error flushing output data
CVT :: Error writing strip: 4438
CVT :: Error writing output
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 30 microseconds
JPEG :: getRegion() :: 1039385 microseconds
CVT :: Error flushing output data
JPEG :: closeImage() :: 10 microseconds
JPEG :: openImage() :: 40 microseconds
JPEG :: getRegion() :: 787402 microseconds
CVT :: Error flushing output data
JPEG :: closeImage() :: 9 microseconds
JPEG :: openImage() :: 27 microseconds
JPEG :: getRegion() :: 847593 microseconds
CVT :: Error flushing output data
JPEG :: closeImage() :: 30 microseconds
JPEG :: openImage() :: 28 microseconds
JPEG :: getRegion() :: 783864 microseconds
CVT :: Error flushing output data
CVT :: Error writing strip: 3038
CVT :: Error writing output
JPEG :: closeImage() :: 27 microseconds
JPEG :: openImage() :: 25 microseconds
JPEG :: getRegion() :: 802534 microseconds

[ruven]

You mentioned "tiles", but iipsrv thinks this is a request for a region not a tile. So, what exactly is the request you are making here? In fact can you also paste the lines just before, so back to the beginning of the request and including the line that says "Full Request is ..."

[HansBull]

Regions, yes. Sorry for the misunderstanding.

New log. Seems like iipsrv crashes after the last CVT log entry, it is then restarted by systemd.
Stopping memcached daemon makes no difference.

<----------------------------------->
Sun Sep 29 16:22:37 2024

IIPImage Server. Version 1.3
*** Ruven Pillay [email protected] ***

Verbosity level set to 2
Running in standalone mode on socket: /run/iipsrv/iipsrv.sock with backlog: 2048

Setting maximum image tile data cache size to 10MB
Setting maximum image metadata cache size to 1000 images
Setting filesystem prefix to ''
Setting filesystem suffix to ''
Setting default JPEG quality to 75
Setting default PNG compression level to 1
Setting default WebP compression level to 50
Setting maximum CVT size to 5000
Setting HTTP Cache-Control header to 'max-age=86400'
Setting 3D file sequence name pattern to 'pyr'
Setting default IIIF Image API version to 3
Setting Allow Upscaling to true
Setting ICC profile embedding to true
Setting codec passthrough to true
Setting up TIFF support via LIBTIFF, Version 4.3.0
Setting up JPEG support via libjpeg-turbo 2.1.2
Setting up JPEG2000 support via OpenJPEG 2.4.0
Setting image processing engine to CPU processor
OpenMP enabled for parallelized image processing with 8 threads
Memcached support enabled. Connected to servers: 'localhost' with timeout 86400

Initialisation Complete.
<----------------------------------->

Full Request is IIIF=/tmp/osd/0204969.jpg/10752,17920,512,512/256,256/0/default.jpg
[1/1]: Command / Argument is IIIF : /tmp/osd/0204969.jpg/10752,17920,512,512/256,256/0/default.jpg
FIF :: Image metadata cache initialization
FIF :: JPEG image detected
JPEG :: Full resolution : 26754x25373
JPEG :: Virtual resolution : 13377x12686
JPEG :: Virtual resolution : 6688x6343
JPEG :: Virtual resolution : 3344x3171
JPEG :: Virtual resolution : 1672x1585
JPEG :: Virtual resolution : 836x792
JPEG :: Virtual resolution : 418x396
JPEG :: Virtual resolution : 209x198
JPEG :: 8 bit data
JPEG :: 3 channels
JPEG :: 1x1 pixels/cm
JPEG :: loadImageInfo() :: 320 microseconds
JPEG :: openImage() :: 573 microseconds
FIF :: Image metadata cache size: 1
FIF :: Image dimensions are 26754 x 25373
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Fri, 27 Sep 2024 10:22:43 GMT
FIF :: Total command time 1008 microseconds
CVT handler reached
JPEG :: getRegion() :: 1055902 microseconds
CVT :: Region decoding time: 1055941 microseconds
CVT :: Total command time 1056495 microseconds
IIIF :: Total command time 1057736 microseconds
JPEG :: closeImage() :: 11 microseconds
Total Request Time: 1059938 microseconds
Image closed and deleted
Server count: 1

Full Request is IIIF=/tmp/osd/0204969.jpg/10240,17408,512,512/256,256/0/default.jpg
[1/1]: Command / Argument is IIIF : /tmp/osd/0204969.jpg/10240,17408,512,512/256,256/0/default.jpg
FIF :: Image metadata cache hit
FIF :: JPEG image detected
JPEG :: openImage() :: 27 microseconds
FIF :: Image metadata cache size: 1
FIF :: Image dimensions are 26754 x 25373
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Fri, 27 Sep 2024 10:22:43 GMT
FIF :: Total command time 96 microseconds
CVT handler reached
JPEG :: getRegion() :: 971000 microseconds
CVT :: Region decoding time: 971032 microseconds
CVT :: Total command time 971451 microseconds
IIIF :: Total command time 971598 microseconds
JPEG :: closeImage() :: 24 microseconds
Total Request Time: 971893 microseconds
Image closed and deleted
Server count: 2

Full Request is IIIF=/tmp/osd/0204969.jpg/10240,17920,512,512/256,256/0/default.jpg
[1/1]: Command / Argument is IIIF : /tmp/osd/0204969.jpg/10240,17920,512,512/256,256/0/default.jpg
FIF :: Image metadata cache hit
FIF :: JPEG image detected
JPEG :: openImage() :: 22 microseconds
FIF :: Image metadata cache size: 1
FIF :: Image dimensions are 26754 x 25373
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Fri, 27 Sep 2024 10:22:43 GMT
FIF :: Total command time 86 microseconds
CVT handler reached
JPEG :: getRegion() :: 987965 microseconds
CVT :: Region decoding time: 988002 microseconds
CVT :: Total command time 988399 microseconds
IIIF :: Total command time 988533 microseconds
JPEG :: closeImage() :: 27 microseconds
Total Request Time: 988872 microseconds
Image closed and deleted
Server count: 3

Full Request is IIIF=/tmp/osd/0204969.jpg/9728,17664,256,256/256,256/0/default.jpg
[1/1]: Command / Argument is IIIF : /tmp/osd/0204969.jpg/9728,17664,256,256/256,256/0/default.jpg
FIF :: Image metadata cache hit
FIF :: JPEG image detected
JPEG :: openImage() :: 25 microseconds
FIF :: Image metadata cache size: 1
FIF :: Image dimensions are 26754 x 25373
FIF :: Image contains 3 channels with 8 bits per channel
FIF :: Image timestamp: Fri, 27 Sep 2024 10:22:43 GMT
FIF :: Total command time 93 microseconds
CVT handler reached
JPEG :: getRegion() :: 985624 microseconds
CVT :: Region decoding time: 985662 microseconds
CVT :: Total command time 986118 microseconds
<----------------------------------->
Sun Sep 29 16:22:47 2024

IIPImage Server. Version 1.3
*** Ruven Pillay [email protected] ***

Verbosity level set to 2
Running in standalone mode on socket: /run/iipsrv/iipsrv.sock with backlog: 2048

Setting maximum image tile data cache size to 10MB
Setting maximum image metadata cache size to 1000 images
Setting filesystem prefix to ''
Setting filesystem suffix to ''
Setting default JPEG quality to 75
Setting default PNG compression level to 1
Setting default WebP compression level to 50
Setting maximum CVT size to 5000
Setting HTTP Cache-Control header to 'max-age=86400'
Setting 3D file sequence name pattern to 'pyr'
Setting default IIIF Image API version to 3
Setting Allow Upscaling to true
Setting ICC profile embedding to true
Setting codec passthrough to true
Setting up TIFF support via LIBTIFF, Version 4.3.0
Setting up JPEG support via libjpeg-turbo 2.1.2
Setting up JPEG2000 support via OpenJPEG 2.4.0
Setting image processing engine to CPU processor
OpenMP enabled for parallelized image processing with 8 threads
Memcached support enabled. Connected to servers: 'localhost' with timeout 86400

Initialisation Complete.
<----------------------------------->

[ruven]

I tried creating a JPEG with identical dimensions to yours and ran the exact same sequence of requests as in your log, but couldn't reproduce your segfault. Can you increase the VERBOSITY environment variable to 6, run again and paste the new log entries?

Perhaps it's something to do with your particular JPEG file. With what was it created? Is it possible to send it to me to try?