Resolved Google Sitemap for Multi-lingual site

Hello Eddie,

Can you tell me what you see when you run the command

php -v

from the command line? The issue with the key is that CSVI is not passed any key value. So we need to figure out why that is.

Unlike in v5 which displays the cron command line, the CRON button has no effecct when pressed in v7

You don't see the fieldnames show up that can be used in the cron command?

Hello Roland

See attached screenshot for php info

<You don't see the fieldnames show up that can be used in the cron command?>

Sorry, see them where?

Best

Eddie

Hey Eddie,

The PHP info shows there are a number of security layers added, the Zend Guard and Suhosin patches. Now I wonder if any kind of argument makes it into the script or if these patches prevent any argument from going in. If you run the cron job like this, does that give any output?

php administrator/components/com_csvi/helper/cron.php --help

Something else coming to mind, perhaps those patches don't like the name key and just filter that out. Only your hosting provider could tell us that.

In the Setting up a cron job document I gave you earlier there you see red boxes in the first screenshot. The explanation below the image tells you what happens.

The PHP info shows there are a number of security layers added, the Zend Guard and Suhosin patches. Now I wonder if any kind of argument makes it into the script or if these patches prevent any argument from going in. If you run the cron job like this, does that give any output?

php administrator/components/com_csvi/helper/cron.php --help

Yes, help options are returned.

Something else coming to mind, perhaps those patches don't like the name key and just filter that out. Only your hosting provider could tell us that.

I will check with the host.

In the Setting up a cron job document I gave you earlier there you see red boxes in the first screenshot. The explanation below the image tells you what happens.

Yes, that's not a problem.

Best

Eddie

Hi

I have asked the question of the host but, after checking other jobs, I see that v5 outputs our sitemap correctly on our live site.

That's WITH a key and also user & password.

I tried the new one with the addition of --user --passwd but it still threw out the key.

Can I use the v5 cron.php in the v7 setup?

Best

Eddie

Hello Eddie,

Can I use the v5 cron.php in the v7 setup?

That is not possible unfortunately. What is even more interesting, since you mentioned version 5, I compared this with version 7 and the way CSVI collects the arguments is the same.

Would it be possible to send me SSH access so I can see what is going on?

Hello Roland

I'll do that after my host has responded.

They are pretty good and will test all available options.

Best

Eddie

Hello Roland

Ok, my host has confirmed no 'key' blockage.

I can send you ssh access.

Please advise where to send.

Best

Eddie

Hello Eddie,

Please send it to contact@csvimproved.com Thanks.

Hi

Email sent.

Best

Eddie

Hey Eddie,

Thanks, I will check it and let you know my findings.

Hello Eddie,

Thanks for the access, I have been able to trace the issue. The issue being that the first parameters was dropped, so the secret key was being ignored. Would you by chance had the template ID as first argument, this thread would have been about the template not being used

So the fix is in your CSVI 7 setup, I am of course going to include it in the next release.

Hello Roland

The issue being that the first parameters was dropped, so the secret key was being ignored.

I understand that.

Would you by chance had the template ID as first argument, this thread would have been about the template not being used

?

So the fix is in your CSVI 7 setup,

Is it!? Where?

Are you saying not to use the --key parameter first?

If I use another parameter first, will that work or also be ignored?

Best

Eddie

Hello Eddie,

Is it!? Where?

In the cron.php file.

Are you saying not to use the --key parameter first?

I am not saying that.

If I use another parameter first, will that work or also be ignored?[/quoite]It doesn't matter which parameter you use first, that is the issue that I resolved.

You can run the CLI command as before, no changes needed.

Ah! I see you have amended the cron.php file.

I set the job to run at 20:05 (GMT).

The cronjob email status has now changed to:

"Process started"

At the time of writing this message, the job has been running for 19 minutes.

It should take less than 1 minutes.

I wonder if the job cannot handle the switch currency rule?

Best

Eddie

Hi

Ok, after some 'dabbling' I have some stats:

On the Live site, using v5,

12,863 records processed in 41 SECONDS - sitemap
2, 108 records processed in 10 SECONDS - sitemap2

On the Test site (back), using v7

220 records processed in 7 MINUTES - sitemap-en (no rules); 2 teams (manufacturers) + 4 categories
78 records processed in 2.5 MINUTES - sitemap-en (no rules); 2 teams (manufacturers) + 4 categories

Any attempts at running jobs with rules (language switching) do not complete

It seems then, that there is a definite resource issue with v7 cron

I wonder if there is an issue with the fact that the jobs originate from a sub-directory (back)?

Best

Eddie

Hello Eddie,

Perhaps just like you, I am also quite surprised how slow the export is. Usually with such huge differences there must be a smoking gun somewhere. Do you have the debug log of the export in CSVI 7?

Hey Eddie,

Tharuna reminded me that there is an issue with slow exports due to SEF URLs. Can you have a look at my post here. It has a solution for that. I think that should help.

Hello Roland

I will take a look at that thread later today.

Will it also solve the issue of non-sef urls being produced even though SEF is requested?

Also, during the night (zzzzzzzzz), I replicated the sitemap with replacement rules in v5, and the job runs well and at normal speed.

So the issue does appear to be with v7.

A sample 'nl' sitemap is attached, which was generated by v5.

It adds the 'nl' sub-directory which I have removed in htaccess but I am yet to be able to remove the ?lang=nl parameter.

Best

Eddie

Hello Eddie,

The change that was introduced in CSVI 7 is that I built another way of retrieving SEF URLs because generating SEF URLs in the backend/cron is a nightmare with all the different SEF extensions out there. What happens now is the CSVI makes a call to the front-end to retrieve the actual SEF URL because the front-end is the only place where Joomla can really make a SEF URL. This way CSVI doesn't have to care about what SEF extension and settings you are using.

The downside of this change is that each call to the front-end takes some time thus slowing down the export. The thread link I posted adds a SEF cache, so only the first export is slow and subsequent ones are faster as the URL comes from the database.

Now I have also build a maintenance task to run the update of the SEF URLs cache independent of the export, so it can be scheduled.