Review 11.5 Merged

78249[BUGFIX] Support parsing of XML files larger than 10 MB

Andreas Kienast CR +2 V +2 Expert +2013 −23 updated 03/24/23, 19:20

Commit message

EXT:impexp may run into memory limits when converting XML to PHP
arrays, even before all PHP memory is used up, because libxml has a
specific limit of 10 MB. This limit can be turned off by using the
`XML_PARSE_HUGE` (libxml) or `LIBXML_PARSEHUGE` (PHP) option, but
only for the DOMDocument implementation of libxml, and not for the
current XmlParser implementation.

By replacing the XmlParser implementation with a DOMDocument
implementation, larger XML files can be parsed with lower peak memory
consumption as a side effect. For example, parsing a 4 MB dummy XML
file consumes

56.03 MB (memory) / 168.72 MB (memory peak)

with the XmlParser, while using DOMDocument reduces the consumption to

56.15 MB (memory) / 60.08 MB (memory peak).

Besides the replacing of the implementation, XML parsing has been moved
to separate classes (Typo3XmlSerializer / Typo3XmlParser), fully covered
by tests, and restructured to reduce the number of required parameters to
a minimum. The functional scope was not reduced in any way.

Review

  • Code review CR +2 Done
    • Andreas Kienast +2
  • Verified V +2 Done
    • Andreas Kienast +2
    • core-ci +1 CI

Issues 1

Files 11

Review log 58

  1. Andreas Kienast uploaded patch set 1 03/24/23, 15:15

  2. Andreas Kienast updated the commit message in patch set 2 03/24/23, 15:15

  3. core-ci voted on patch set 1 V -1 03/24/23, 15:31 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/39568

  4. core-ci voted on patch set 2 V -1 03/24/23, 15:52 Core CI is not happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/39569

  5. Andreas Kienast uploaded patch set 3 V -1 03/24/23, 19:04

  6. core-ci voted on patch set 3 V +1 03/24/23, 19:12 Core CI is happy: https://git.typo3.org/typo3/CI/cms/-/pipelines/39587

  7. Andreas Kienast voted on patch set 3 V +2 CR +2 03/24/23, 19:20

  8. Andreas Kienast merged patch set 4 03/24/23, 19:20