Review main Merged

70814[BUGFIX] Support parsing of XML files larger than 10 MB

Alexander Nitsche CR +2 V +2 Expert +2007 −23 updated 03/24/23, 14:28

Commit message

EXT:impexp may run into memory limits when converting XML to PHP
arrays, even before all PHP memory is used up, because libxml has a
specific limit of 10 MB. This limit can be turned off by using the
`XML_PARSE_HUGE` (libxml) or `LIBXML_PARSEHUGE` (PHP) option, but
only for the DOMDocument implementation of libxml, and not for the
current XmlParser implementation.

By replacing the XmlParser implementation with a DOMDocument
implementation, larger XML files can be parsed with lower peak memory
consumption as a side effect. For example, parsing a 4 MB dummy XML
file consumes

56.03 MB (memory) / 168.72 MB (memory peak)

with the XmlParser, while using DOMDocument reduces the consumption to

56.15 MB (memory) / 60.08 MB (memory peak).

Besides the replacing of the implementation, XML parsing has been moved
to separate classes (Typo3XmlSerializer / Typo3XmlParser), fully covered
by tests, and restructured to reduce the number of required parameters to
a minimum. The functional scope was not reduced in any way.

Review

  • Code review CR +2

    Imported before Forger kept the single votes, the next import adds them.

  • Verified V +2

    Imported before Forger kept the single votes, the next import adds them.

Issues 1

Files 11