Support is offline. We are back Monday 23 November, 09:00.

Memory problems after migration #78257

Resolved Locked p.barg opened this topic Wednesday, 08 November 2023 · 10 replies
Author

Hello,

I'm currently migrating a customers site from J3/VM3 to J4/VM4. One of the last things remaining is to get CSVI working and I really got stuck here....
On the current site CSVI has been running for ages without any problems. It is used to export product data as XML file for Google. The produced output file
contains around 6000 records. The current versions are J3,VM3 CSVI 7.20.0.
I've now migrated the site (on a local server) to J4/VM4.2.4. I then updated CSVI to the current version 8.9.2. Although I'm on a local server with 512M memory,
I seem to run into memory limit problems. On the LIVE site the memory limit is 256M (and can't be increased any further) and was always sufficiant. On a preview server where I also tested, the process sometimes comes up with status 200 OK, but then throws warnings about Virtuemart varaibles.
Can it really be the case that 512M are not enough memory for the current version of CSVI? Or might there be a different problem involved?
I'm really stuck now (can't put the site online) and would appreciate any help/hints.

Kind regards,

Petra

Support

Hello Petra,

I seem to run into memory limit problems. On the LIVE site the memory limit is 256M (and can't be increased any further) and was always sufficiant.

What do you see in warnings to say these are memory limit problem? Can you post a screenshot?

but then throws warnings about Virtuemart varaibles.

Since these are warnings and are coming from VirtueMart you can try setting up Error reporting to None or System default in Joomla global configuration page and check the import. See if import finished without these warnings.

Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory

Author

Hi Tharuna,

this is really mysterious. Yesterday I kept getting 500 errors with memory exceeded infos in it.
Today I can't reproduce them... No I get 200 OK status, but with warnings like in attached image.
The export files are created though and do look ok.
But I'm worried now if the issue might come back if I put the installation of the LIVE server...
Also the error window might be confusing for the customer.
What do you think?

Kind regards,

Petra

  • status200.jpg
Support

Hello Petra,

On the current site CSVI has been running for ages without any problems. It is used to export product data as XML file for Google.

The Google export did get some changes in the form of the g:shipping_price field. This now calculates the real shipping price for a product. Are you using that by any chance? Although users who tested this feature have not seen any problems with that.

Can it really be the case that 512M are not enough memory for the current version of CSVI? Or might there be a different problem involved?

Memory is one of the hardest things to test. However I do not see any reason to change the amount of memory and 512M sounds more than sufficient to me. There can always be a different problem unless we can definitely pinpoint the problem. A 500 error is a general error without checking server logs you won't know what the real error is. Even then it might be a memory issue but that can also be because the server was under load from something else. Although I see you mention you did get memory issues in your messages, it may be a combination of circumstances.

oday I can't reproduce them... No I get 200 OK status, but with warnings like in attached image.

Unfortunately it happens, even though computers are all about 1 and 0, circumstances are never :D That makes things difficult to handle sometimes. As for the warnings, they come from the VirtueMart files, so not something we can fix. Unfortunately, VirtueMart is full of them and all we can do is turn off error reporting, as it should be on a production server. Production servers should always have error reporting off because you do not want visitors to run into server related information disclosure might an error occur. Since they are warnings, they are not a problem for the export and the user will not see them once error reporting is off.

But I'm worried now if the issue might come back if I put the installation of the LIVE server...

My suggestion would be to monitor the situation. If this happens more often we can look into what might be causing this. Alternatively, you can run the export also as a cron job, that way it runs unattended and resources should be less of an issue.

Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory

Author

Hi Roland,

thanks for your feedback and explanations!
Yes error 500 and memory problems are the worst...

Meanwhile I found out some more things. On the development server I had debugging turned on and that lead to the memory issues.
As soon as debugging was switched off, the export went through (and of course you're right: after setting errors to default - which is set like this
on our LIVE sites - no popup with warnings is shown anymore). So probably the debugger is consuming much memory too?...
Interestingly, on a preview server with only 156M memory the export went through after overriding the system limits in the backend, although that is
on a shared hosting server and as I understood from the documentation that shouldn't work actually.
Anyway - I'm glad that the export is running now!

Cron job would actually be great, but in the past we weren't able to implement that with our provider. But we might give it another trial.

Thanks for your support!

Petra

Support

Hello Petra,

The debug option on Joomla 4 is very heavy on memory, so that will explain why you were running out of memory. I was not aware you had debugging on when you ran the import.

Interestingly, on a preview server with only 156M memory the export went through after overriding the system limits in the backend, although that is
on a shared hosting server and as I understood from the documentation that shouldn't work actually.

It generally does not work but on hosts it could work as long as you stay within the limits I think.

Cron job would actually be great, but in the past we weren't able to implement that with our provider. But we might give it another trial.

Do know that with the current version we have a native Joomla 4/5 cron import so that may improve your chances on getting it running. We have explained how to use it in our cron documentation. Feel free to create a separate ticket if you need help with that.

Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory

Author

Hi RolandD,

thanks for your hints and explanations. Good to know that J4 debugger is so gready ;)
I was also thinking about the native Joomla task scheduler and I might get back to you on that topic.
Are there any CSVI task plugins or would I have to create them?

Thanks again and best regards,

Petra

Support

Hello Petra,
If you are talking about cron to setup RO CSVI import and export please check Setting up a cron job in RO CSVI document. The document also explains on how to set cron jobs on Joomla 4 installation.

Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory

Author

Hi Tharuna,

I was actually talking about the native Joomla! task scheduler. I though RolandD was referring to that.
Maybe a misunderstanding....

Kind regards,

Petra

Support

Hello Petra,

That is indeed a misunderstanding :D We have not tested RO CSVI with the Joomla Task Scheduler. It is indeed the document Tharuna is referring to, if you scroll down there is a section on setting up cron jobs on Joomla 4.

Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory

Author

OK, thanks :D

This topic is locked. New replies are not possible.

Log in to reply to this topic.