Resolved [Solved] Since uprade to V7 exports are slower

Hello,

You don't mention your CSVI version but there was an issue in 7.0 but this has been fixed in the current 7.0.2 release. Assuming you are using it as a cron job.

Hello,

So I have also checked the logfile and I see that approximately 5 products are exported every second. That is about 300 products per minute. That is not a lot if you have +/- 5000 products to export.

The problem is unfortunately complicated. It all starts with Joomla being unable to build SEF URLs from the administrator side because they are not used there. So we have to look to the front of the site. Now there isn't a universal way there either to generate SEF URLs because a lot of people thought they could build a better SEF generator and so we have a myriad of SEF URL generators. From experience I can tell that the majority have no easy way of getting a SEF URL programmatically. Even if they have, you sometimes would end up with the wrong SEF URL. So the conclusion was that we had to imitate being a visitor. That is how CSVI now gets the SEF URLs, so you are sure you always have the same URL as a visitor has on the website regardless of which SEF extension you use.

The downside of this is that for each URL a request needs to be send to the website to get the SEF URL, much like a visitor clicking a link. This is where the delay usually comes from in the export. This also depends on the servers ability to respond to these requests. The more requests it can handle, the faster the export will be.

While writing this story, I did get an idea. CSVI could cache any previous requested URLs and read them from the database. This would make the first export slow but any subsequent ones would be much faster as the data can be read from the database. This is something I will look into.

So far my look into this issue for now.

Hello,

So it is time to put the money where my mouth is :lol:

The SEF cache system has been built and it runs very well here, the exports are noticeably faster even with very small datasets. This makes me believe it is going to make a real difference in your case as well.

The instructions to get this up and running for now. First you need to add a new table to your database. Usually this is done by going to PhpMyAdmin as provided by your hosting provider or another database management tool. Assuming you have PhpMyAdmin, click on the SQL tab and paste the following statement:

CREATE TABLE IF NOT EXISTS `ey6v4_csvi_sefurls` (

	`sefurl_id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT,

	`plainurl` VARCHAR(255) NOT NULL,

	`sefurl` VARCHAR(255) NOT NULL,

	PRIMARY KEY (`sefurl_id`),

	INDEX `plainurl` (`plainurl`)

) CHARSET=utf8 COMMENT='Stores cached SEF URLs for export';

and then click Go. This is the table that holds the SEF cache.

Second step is to load the attached patch file. This will add 2 new options to your product export template. The options are Cache SEF URLs and Empty SEF Cache. These options are only visible when the Export SEF URLs options is set to Yes.

In your case, edit your template and set the Cache SEF URLs to Yes and save the template. After this run the export and you will notice no difference in speed because the first run will fill the SEF cache as well. Now once the first export is finished, run the same export again and it should be much faster.

The Empty SEF Cache is there just in case you want to start clean, you can set this option to Yes and prior to exporting all cached SEF URLs will be removed and the export will fill it again if Cache SEF URLs is set to Yes.

This information is also available in the template by clicking the Help button.

Let me know how it goes.

Problem solved!!! Speed has been reduced form 450s per export to 28s per export.

I am very impressed at the level of support. In my 17 years in this industry I expect the reverse every time so many poor extensions for commercial for and open source products . Your level of support and speed of fix gives me hope that there are more like you out there

Please never change

For those reading this I have been in private messaging since I opened this topic. I did not want to post data on public forums so they accommodated my request and dealt with this issue privately.

Hello,
Thank you so much. We are glad to hear that your issue has been solved. We would appreciate if you can give us a review on Joomla Extension Directory

This looks like an excellent solution. Will the cache system be added to a future release?

Hello,
Yes, cache system is included in our upcoming CSVI 7.1.0 version.