Categories
View Replies (8)

Resolved SEF cache VM problem

  1. ptrouw
    ptrouw
  2. RO CSVI
  3. Thursday, 24 August 2017
  4.  Subscribe via email
Hi Roland,

I have another problem, I am getting this time out error running an export, with SEF on:
An error occurred processing the records. The error occurred is shown below if available. Additionally check your server log for any errors or warnings.
504 Gateway Time-out

So I tried it with Maintenance options first. Selecting a template with just 1 SKU, it runs in time-out 504 again. Looking in the sefurls table it has now 51020 new records. Which seems strange because I only selected 1 template with 1 SKU.

If I a look in this table I don't understand it, I have 5 values in plainurlIndex which are the same, this because I have 5 active languages. Those 5 fields in plainurlIndex have 5 different values for column sefurl. How can CSVI know which plainurl/sefurl is for which languages? It seems it is missing language in plainurlindex! Is this correct?

Example:
So I have 5 the same plainurlIndex fields like:
index.php?option=com_virtuemart&view=productdetails&virtuemart_product_id=116&virtuemart_category_id=75&Itemid=1
SEFURL: http://www.xyz.com/de/shop/category1/produkt-eins-detail.html

index.php?option=com_virtuemart&view=productdetails&virtuemart_product_id=116&virtuemart_category_id=75&Itemid=1
SEFURL: http://www.xyz.com/nl/shop/category1/product-een-detail.html

3 more the same!



CSVI PRO 7.3.2
VM 3.2.4
PHP version 7.1.8
Joomla! version 3.7.5
Database schema version 7.2.0
0
Responses (8)
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

Looking in the sefurls table it has now 51020 new records. Which seems strange because I only selected 1 template with 1 SKU.
So you ran an export with 1 product and the table has now 51020 new records? That is not what is supposed to happen nor do I see why that would happen. Does the number 51020 correlate to about 10.000 products you have? This is not from the maintenance task?

Those 5 fields in plainurlIndex have 5 different values for column sefurl.
That I can see happening. As you stated this is not correct. I will create a patch file for this and send it to you when ready.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 1
ptrouw
ptrouw
Accepted Answer Pending Moderation
Yes, I tried maintenance create cache SEF with 1 product. Table grew from 0 to 50000+ record. Yes 10000+ product times 5 languages = roughly 50000 in sefurl table. So the 50000+ seems OK.
So 2 problems:
1. 5x the same value field plainurlIndex, not unique per languages
2. template with only 1 product generated sef for all products
  1. more than a month ago
  2. RO CSVI
  3. # 2
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

First thing I have done is look into the language issue. You can load the attached patch file and that should index the URLs including the language dependency. So in your database you should see no duplicate plain URLs anymore. Each URL should have the language included.

Later I am going to see if I can reproduce the URL generation on export.
Attachments (1)
patch_sef_lang_20160825.zip
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 3
ptrouw
ptrouw
Accepted Answer Pending Moderation
thx again for the quick response and patch. I will try now
  1. more than a month ago
  2. RO CSVI
  3. # 4
ptrouw
ptrouw
Accepted Answer Pending Moderation
Unfortunately it doesn't work. I have again 51010 records until 504 time-out. Although I selected one template with only 300 products.
If I do this SQL:
SELECT * FROM `zlto4_csvi_sefurls` WHERE `plainurl` like "%virtuemart_product_id=5118%"
I am getting 5x time this record in plainurl, which is incorrect as it should have language to make it unique.
index.php?option=com_virtuemart&view=productdetails&virtuemart_product_id=5118&virtuemart_category_id=3265&Itemid=1

What I did next is NOT use the maintenance option to create the cache SEF, but just run the Export, with all options on, use SEF, USE sef emply cache, sef cache. Now it deleted all 51010 records, and created 266 new correct records
SQL return 266 unique German records, which is correct:
index.php?option=com_virtuemart&view=productdetails&virtuemart_product_id=177&virtuemart_category_id=81&Itemid=1&lang=de
Only a 504 time-out again, although all 266 er in! Seems the script is always ending with 504 although a records are in!

1. So why is there a difference between an export and maintenance mode running sef?
2. And why is it doing all products and not using the selected template only?

Hope this helps
  1. more than a month ago
  2. RO CSVI
  3. # 5
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

I can tell you why there is a difference between the Maintenance option and the export option. The export option requests 1 URL at a time. If you have 5000 products, you are sending 5000 requests to your own server. This can severely impact the performance of your site. The Maintenance option requests 500 URLs at a time, so now you only have 10 requests if you have 5000 products. This is a much smaller impact on the server.

You are correct, the patch doesn't fix the maintenance option, that I had forgotten but the attached patch fixes this.

2. And why is it doing all products and not using the selected template only?
I cannot reproduce that here. If I run an export with 1 product in the template the SEF URLs table has 1 entry only.

Seems the script is always ending with 504 although a records are in!
SEF URL generation takes quite a bit of resources, that is why we have the maintenance option to ease the load. The idea behind it is, you once load the SEF URLs via the maintenance option and after that you run your export so it can take the data from the database which is a lot faster than having to request it from your server. The option to Empty the SEF cache should ideally be off when running an export. With a lot of products you don't want to give your own server a DDOS ;)
Attachments (1)
patch_sef_url_20170826.zip
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 6
ptrouw
ptrouw
Accepted Answer Pending Moderation
He Roland,
Thx it works, maintenance SEF mode works.

Still one thing:
"2. And why is it doing all products and not using the selected template only?
I cannot reproduce that here. If I run an export with 1 product in the template the SEF URLs table has 1 entry only."

Selecting this template with just one category with 267 products, generated 51010 records. so all products sef for all categories and all languages.

But For now I can live with it.
  1. more than a month ago
  2. RO CSVI
  3. # 7
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

Good to hear that it works fine in maintenance mode now as well. As for the category filter export, I did this as well with a category with 33 products but I only get 33 URLs. In the end it doesn't matter so much I think as you would fill the table via the Maintenance section. Any missing entries will be added as soon as you run the export. You just need to make sure that Empty SEF URLs is set to No in your export template.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 8
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!