Issue Bug Closed

#106639Wrong Conten-type for subsitemaps

Martin Pribyl 04/30/25, 08:56 updated 05/03/25, 09:09

Description

Hello, I get an issue for crawlering my sitemaps by google. It can find the main sitemap, but all inner sitemaps are marked as not readable. I can access them over browser as usually and they looks like xml files, but when I try to scan them over curl, I can see, that the submaps are in wrong content-type. I have it updated over routeEnhancers, but the same result is when I disable it.

routeEnhancers:
  PageTypeSuffix:
    type: PageType
    map:
      /: 0
      sitemap.xml: 1533906435

Here is an example of the outputs:

Correct root sitemap

curl -I https://seo-saurus.com/sitemap.xml

output:
content-type: application/xml;charset=utf-8

Wrong single page sitemap, taken from sitemap.xml

curl -I https://seo-saurus.com/sitemap.xml?sitemap=pages&cHash=9359115ffc1730420fe6142e1b3554af

output:
content-type: text/html; charset=utf-8

Version of the Seo extension is v13.4.9, but it seems to be like that already from the beginning of the v13 project.

Reviews 0

No change resolves or names this issue in its commit message, and no comment links one.

History 6

  1. Georg Ringer commented and changed status New Needs Feedback 04/30/25, 12:38

    works for me

    curl -I https://seo-saurus.com/sitemap.xml\?sitemap\=pages\&cHash\=9359115ffc1730420fe6142e1b3554af
    
    Alias for: curl -I
    HTTP/2 200
    server: nginx
    date: Wed, 30 Apr 2025 12:38:08 GMT
    content-type: application/xml;charset=utf-8
    x-powered-by: PHP/8.3.20
    content-language: en-US
    cache-control: private, no-store
    x-robots-tag: noindex
    x-ua-compatible: IE=edge
    x-content-type-options: nosniff
    x-powered-by: PleskLin
  2. Martin Pribyl commented 04/30/25, 12:44

    Isn’t possible you had there the backslashes like here in the preview? Then it ommits that part and for me would be correct too, but if I don’t have there the slashes, then it’s not correct

    curl -I https://seo-saurus.com/sitemap.xml?sitemap=pages&cHash=9359115ffc1730420fe6142e1b3554af
    [1] 10655
    Martins-MacBook-Pro:~ martinpribyl$ HTTP/2 404 
    server: nginx
    date: Wed, 30 Apr 2025 12:40:13 GMT
    content-type: text/html; charset=utf-8
    x-powered-by: PHP/8.3.20
    x-ua-compatible: IE=edge
    x-content-type-options: nosniff
  3. Garvin Hicking commented 04/30/25, 17:16

    Youw either need to escape the URL special chars like Georg did, or put the whole URL in quotes:

    curl -I "https://seo-saurus.com/sitemap.xml?sitemap=pages&cHash=9359115ffc1730420fe6142e1b3554af"

    Otherwise you are requesting a different regular HTML page because “&” in bash ends the command into the background and the rest of the commands are swallowed in your hidden shell.

  4. Martin Pribyl commented and changed done 0 % 100 % 04/30/25, 17:21

    Ahh, thank you for this feedback. So this is ok, just my google console cannot read it and I have no idea why. But that’s not typo3.

    Thank you for all your feedbacks and your time.
    I’m not sure how to close this issue, can you do it?

  5. Garvin Hicking commented and changed status Needs Feedback Closed 04/30/25, 17:31

    No worries, thanks for getting back quickly. Sadly I dont know what could be causing the google error.
    Maybe some htaccess rule, htacces, or URL encoding? Can you identify the google crawler requests in your access log maybe to see why they might fail?

    Anyhow – closing for now, but feel free to create a new issue if you can pinpoint a bug!

  6. Martin Pribyl commented 05/03/25, 09:09

    Thank you for the support, I found, that for some reasons google didn’t liked for the subsitemaps “X-Robots-Tag:noindex” comming from seo_sitemap.config.additionalHeaders.20.header.

    I have no idea, why it has issue only here and only for the subsitemaps, but at least this is something what I can track only on my side.

    Again thank you for your time :)