Categories
View Replies (22)

Resolved cron.php huge memory usage

  1. nextcomp
    nextcomp
  2. RO CSVI
  3. Tuesday, 05 July 2016
  4.  Subscribe via email
My import template works fine if I start backend manually. But not working if I start cron.php. The problem is I have got only 256MB virtual memory my web server.

/usr/local/bin/php /home/xxx/public_html/xxx/administrator/components/com_csvi/helper/cron.php --key="xxx" --template_id="8" --form.import_nolines=100 --form.import_wait=5 --file="/home/xxx/public_html/xxx/tmp/export.csv"

export.csv is about 1,5MB

I have to set --form.import_nolines=100 and --form.import_wait=5 in the backend but the cron.php is not using this. The server killed my process because 256MB is the limit!

How can I solve this problem without upgrade server memory limit?
0
Responses (22)
nextcomp
nextcomp
Accepted Answer Pending Moderation
The export is own development. It's working fine. I use only CSVI import template. Cron.php import my export.csv. How can I tell cron.php that working parts of big csv like JS? Sorry for my English :)

cron.php read 100 rows and than read other 100 rows etc.
  1. more than a month ago
  2. RO CSVI
  3. # 21
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

I have to set --form.import_nolines=100 and --form.import_wait=5 in the backend but the cron.php is not using this.
Cron cannot use that because it is not using any webbrowser. Cron doesn't have JavaScript.

How can I solve this problem without upgrade server memory limit?
Run multiple exports with a record limit. So let's say you run one export to do the first 10000 records and an second do the next 10000 etc.

Finally when you have your files you can run a final cron job with this command to create a single file out of it 
cat file1.csv file2.csv > totalfile.csv


A CSV file of 1.5MB is quite large so you are going to need more resources as well. The question is how much more memory does it need on your server? It's kind of weird because every line that is being exported is written directly to file.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 22
  • Page :
  • 1
  • 2


There are no replies made for this post yet.
Be one of the first to reply to this post!