Resolved Product Export throwing database table error

Hello Tim,

is there any similar procedure that can be done for exporting?

Not with the current CSVI because it exports straight into the browser, so you get a download window. Any interference in that will break the download of the file.

I am looking into it for CSVI 6 but it's just on the drawing board, nothing concrete at this point. For now, all that can be done is to tune the server so it won't stop the export process.

Thanks Roland, i thought that may be the case.

Would you be kind enough to have a look at this export template with me. I have had great difficulty resolving this issue with our host Heart and am not sure what next to try!

This is their latest reply after i gave them access to our Joomla admin (and i have been speaking to them at length about possible RLimitMem settings which they have advised is not an issue)

"After going through and testing this on our end, we have found that the issue here is that the script itself is taking up too much memory, causing the memory issues here. While the file is uploaded, the file seems to be doing other things, causing the memory to be as big as it is.

This means that the memory for the script is over 512mb, which Premium Hosting or Shared Hosting cannot handle. Honestly, a file shouldn't need that much memory anyway.

Have you contacted the creator of the script to see why this is taking up so much memory? While I understand that you need to export this file, it shouldn't be failing with 512mb available. In this case, the script is doing something else while exporting and is going over the limit."

What would you need from me? Could i send you log in details seperately to view the issue?

Kind regards

Tim

Hello Tim,

This means that the memory for the script is over 512mb, which Premium Hosting or Shared Hosting cannot handle. Honestly, a file shouldn't need that much memory anyway.

How much memory a file needs depends on how much data it needs to process. Actually the memory on import is not an issue as long as you use CSV or XML files, since they are read line by line.

Have you contacted the creator of the script to see why this is taking up so much memory? While I understand that you need to export this file, it shouldn't be failing with 512mb available. In this case, the script is doing something else while exporting and is going over the limit."

The issue is that I don't know if it is CSVI taking up this memory or parts of VirtueMart that is used to export certain data. The question is also, how many records and fields are you trying to export.

What would you need from me? Could i send you log in details seperately to view the issue?

There isn't really anything I can do, the root cause is that you run out of memory. So you can either split your export into smaller parts or export less fields. The former is more likely than the latter one.

Thanks Roland.

I think i have exhausted all options server side, so perhaps i should come at this from a different way and look at the export.

Basically i am exporting an xml file for Google shopping.

This is an example product extract that the csvi export creates:

<item>
<g:id><![CDATA[00054X]]></g:id>
<title><![CDATA[GBC ID Neck Chain 30 inch (760mm) Ref EB100000 [Pack 100] 00054X]]></title>
<description><![CDATA[<ul><li>ID neck chains</li><li>Suitable for laminated name tags</li><li>30 inches (760mm)</li><li>Packed 100. </li><li>Supplier reference: EB100000</li><li>Our code: 00054X. </li></ul><br /><br /><br /><a href="http://officeneedsdirect.opnet.co.uk/AddToBasket.aspx?ProductCode=00054X"; rel="nofollow"><img src="images/stories/addtobasket-small.gif" width="91" height="40" alt="addtobasket-small" /></a><br /><br /><br /><a href="http://officeneedsdirect.opnet.co.uk/Detail.aspx?SearchText=00054X"; rel="nofollow">View Quantity Breaks & Full Info</a>]]></description>
<g:google_product_category><![CDATA[Office Supplies]]></g:google_product_category>
<g:condition><![CDATA[new]]></g:condition>
<g:availability><![CDATA[in stock]]></g:availability>
<grice><![CDATA[71.20]]></grice>
<g:brand><![CDATA[GBC]]></g:brand>
<g:mpn><![CDATA[00054X]]></g:mpn>
<g:image_link><![CDATA[http://www.officeneedsdirect.co.uk/images/stories/virtuemart/product/00054X.jpg]]></g:image_link>
<groduct_type><![CDATA[Facilities Management>Safety & Security>I D Badges>Others]]></groduct_type>
<link><![CDATA[http://www.officeneedsdirect.co.uk/GBC-ID-Neck-Chain-30-inch-(760mm)-Ref-EB100000-[Pack-100]-00054X-details.html]]></link>
</item>

I have about 15000 products.

Looking at the xml file, it fails generally around line 80,000 which is appox 5000 products.

I havent used this before but are you suggesting it would be best to create three seperate xml feeds for Google using "Number of records to export" filter option on the template?

Kind regards

Tim

Hello Tim,

I havent used this before but are you suggesting it would be best to create three seperate xml feeds for Google using "Number of records to export" filter option on the template?

Correct, that is the option I was referring to.

Just out of curiosity, would you be able to setup a test site with Joomla 4, VirtueMart 3 and CSVI 6 beta 4? If so, you could see if the export would work on that for you. Just an idea.

thanks Roland, splitting into three has worked fine.

I have another site i need to upgrade coming up, so may well try the above to see if that works.

Thanks for your advice

Kind regards

Tim

Hello Tim,

Glad to hear that worked