#87224GifBuilder creates filename hashes over and over
Description
Hi,
we recently updated all our installations from 7.6 to 8.7. and we experienced a problem:
/typo3temp/assets/images overflowed with images.
Our filesystem can handle only ~22000 files per folder, and our Installations are huge. But that was not the main problem. Some files were in there many hundered times, in one case over 7000 times.
Each time with a different hash.
I’m not a PHP developer and not too familiar with the innards of the TYPO3 core, but my analysis showed, that the hash is generated in GifBuilder.php by putting $setup through md5. Not a problem per se.
But in our case $setup was up to 32MB big (when written als Text to disk) – and even worse differed from call to call.
The culprit seemed to be a sub-array of $setup [‘BBOX’] containing huge amounts of data, recursions and a changing order of its elements.
The code is nearly unchanged since TYPO3 6.2 (oldest copy I had) and I even found a open issue going back some 6 years:
https://forge.typo3.org/issues/44518#change-391424
I did not receive any answer when I updated that old issue, so I do it here. I hope double-posting this is not too bad.
I now have a working work-around in place, but I don’t think it’s the best solution:
in function fileName in typo3/sysext/frontend/Classes/Imaging/GifBuilder.php I changed the line 708
return 'typo3temp/' . $pre . $filePrefix . '_' . GeneralUtility::shortMD5(serialize($this->setup)) . '.' . $this->extension();
to
$setupLight=$this->setup;
foreach ($setupLight as $theKey =>$theValue) {
unset($setupLight[$theKey.'.']['BBOX']['originalFile']);
unset($setupLight[$theKey.'.']['BBOX']['processedFile']);
unset($setupLight[$theKey.'.']['BBOX']['fileCacheHash']);
}
return 'typo3temp/' . $pre . $filePrefix . '_' . GeneralUtility::shortMD5(serialize($setupLight)) . '.' . $this->extension();
This is roughly the same approach of the mentioned issue 44518.
Since the problem seemingly to arrived for us with 8.7 (or at least worsened to be noticeable) and since the code in GifBuilder.php hasn’t really changed in a long time, I think the problem lies more in having too much data in that BBOX array.
To my understanding it is for the “BoundingBox” of an image layer, but it contains so much more ‘stuff’.
I can not rule out that this array gets this big because of something we do somewhere else, but I have no idea, what this could be.
I have to admit, I struggled with getting GifBuilder to work with FAL when we switched to TYPO3 7.6 so my quirky typoscript does some ‘things’ to the image ID to produce a version of it with text layered on top:
lib.imageplus=FILES
lib.imageplus{
# The Fluid Template gives an array of "fields", using currentValueKey we set the current object,
# which contains the id of the file we want
files.current=1
# The String we get starts with "file:" in front of the id, so we cut it away using substing
files.stdWrap.substring=5
# But sometimes this code is called in a way, that gives us the id without "file:"
# So we check if the given String already is a number. There seems to be no isNumber() function though
# so we use this abomination that gives us the same result
# This was written to be compatible with data stored in databases written with TYPO3 6.2, 7.6 and 8.7
# and was not re-written for a long time so we had to cope with old obsolete data formats
# The version for 8.7 now uses a ViewHelper to get the ID from strings containing t3://
# So this code may be not needed anymore.
files.stdWrap.substring.if.isLessThan=100000000
files.stdWrap.substring.if.value={field:image}
files.stdWrap.substring.if.value.insertData=1
renderObj = COA
renderObj{
# After entering the scope of GIFBUILDER we seem to be unable to access the file
# object so we store everything we need in the register
10 = LOAD_REGISTER
10 {
fileid.data = file:current:uid
creator.data = file:current:creator
status.data = file:current:status
visible.data = file:current:visible
storage.data = file:current:storage
identifier.data = file:current:identifier
identifier_hash.data = file:current:identifier_hash
width.data= file:current:width
height.data= file:current:height
filecaption.data = file:current:caption
desc.data = file:current:description
title.data = file:current:title
alternative.data = file:current:alternative
link{
typolink.parameter.data = file:current:link
typolink.returnLast = url
}
}
20=IMG_RESOURCE
20{
file = GIFBUILDER
file {
# The size of the to-be-created images is equal to the image we load in layer 10:
XY=[10.w],[10.h]
XY.override.if.isFalse.data = register:visible
XY.override=[15.w],[15.h]
format = jpg
quality=90
10 = IMAGE
10 {
if.isTrue.data = register:visible
file {
#We get the ID from the register and use it as the source of the image:
import.data = register:fileid
# we get the wanted width from the ViewHelper in the field array
maxW={field:width}
maxW.insertData=1
}
}
...