#90180DataHandler is running into a 500 Error while importing large data
Description
I have built an extension to import a lot of data from another database into my TYPO3 instance.To prevent running out of memory or reaching the max execution time the import is devided into several chunks and multiple ajax requests until all chunks are imported. After about 65 chunks (96 chunks total) the import running into an internal server error with the response “Houston, we have a problem”.
I’m calling the process datamap method with alot of data and don’t know the problem. I’m also not able to catch this exception:
// Custom log method to store my logs into separat log files for each chunk in the typo3temp directory.
$this->log( 'Starting DataHandler' );
$this->log( json_encode( $this->data, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES ) );
$this->dataHandler->start( $this->data, [] );
try {
$this->dataHandler->process_datamap();
} catch ( \Exception $e ) {
$this->log( $e->getMessage() );
}
$this->log( 'DataHandler finished' );
The exception will be thrown every time and the last log “DataHandler finished” won’t be logged.
In the files there is an example of data to import (one of the breaking chunks) formatted in json. It is a lot of data, but in some chunks before there is more data but no errors. I’m trying to fix it since 2 weeks without success.
I’m working with TYPO3 v9.5.13 and PHP 7.2
Reviews 0
No change resolves or names this issue in its commit message, and no comment links one.
Fix this issue
Start the commit message of your patch with this. Add what the patch does, and the versions it goes to in "Releases:". How to create a patch
- Commit message
-
[BUGFIX] DataHandler is running into a 500 Error while importing large data Resolves: #90180 Releases: main
Files 1
History 1
-
Benni Mack commented and changed status
NewNeeds Feedback <span data-processor="relativetime" data-value="2026-11-09T16:45:17+00:00" > 11/09/26, 16:45 </span>
Thanks for the report, and sorry that nobody answered it back then.
Unfortunately the report does not contain the actual error. “Houston, we have a problem” is not a message of the TYPO3 Core, and catch (\Exception) does not catch PHP errors (\Error, e.g. TypeError) or fatal errors like an exhausted memory_limit or max_execution_time, which would match “fails after ~65 chunks”. Note that each chunk also updates the finder record with the complete, growing list of products, so the MM relation is rewritten with more entries on every chunk.
If this still happens with a maintained TYPO3 version (13.4 or 14.x), please add the error from var/log/typo3_*.log or the PHP error log, including the stack trace, and catch \Throwable instead of \Exception. Without that, there is nothing we can reproduce, so I’m setting this to “Needs Feedback” and will close it if there is no reply.