Categories
View Replies (16)

Resolved Importing image failes

  1. Poiter
    Poiter
  2. RO CSVI
  3. Wednesday, 18 October 2017
  4.  Subscribe via email
Hi,

A new issue with importing product images into Virtuemart. I followed this link Import VirtueMart images
But I cannot download alle the pictures, they have to come from the server of the manufacturer in the csv file.
So I copied my previous template wich I used to import the products (there were also the images, but this didn't go well). I skipped every field I don't need and only use the file_url, product_sku and made one field wich media_published. (there was one field in my csv file I don't use but has yes in the fields).
When importing the import is abborded very quickly due a 500 error.
I see the pictures from the manufacturer are very big (like 4mb) so I changed the thumbnail setting in my template under the image tab to 9999 height/width and under the limit tab I changed the lines per import to 1 and time to 5. Now the import is aborted after about 94 and later after 14
So I think due the large image files, my server can't handle it? Are there more options to make this work. Or maybe I need to split the csv in lower files. Now it has almost 690 rows, maybe split them in 14 csv files wich each 50 rows? Attached the error log from the server and the import template XML form csvimproved

Gr. Pieter
0
Responses (16)
tharuna
tharuna
Accepted Answer Pending Moderation
Hello Pieter,
I see the pictures from the manufacturer are very big (like 4mb) so I changed the thumbnail setting in my template under the image tab to 9999 height/width and under the limit tab I changed the lines per import to 1 and time to 5. Now the import is aborted after about 94 and later after 14

The maximum width resize and height resize fields are to check the image sizes are above these given limits, if yes then the image resize wont happen. This is to avoid timeout during import. I think your images are of lesser size than what is set here so resize is done on all images and so the import is stopped. Probably this setting wont help.

So I think due the large image files, my server can't handle it? Are there more options to make this work. Or maybe I need to split the csv in lower files. Now it has almost 690 rows, maybe split them in 14 csv files wich each 50 rows? Attached the error log from the server and the import template XML form csvimproved

Yes, your server is hitting its limit while resizing and import the images and so it stops after a certain records. One way is you can try increasing max_execution_time on your server and see if it helps. You need to set this in your php.ini file. You can ask your hosting provider for help if you don't know on how to set it.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 1
Poiter
Poiter
Accepted Answer Pending Moderation
Hi Tharuna,

My host increased it from 120 sec to 4 minutes. But still no result and a 500 error :-(
They offered to have a deeper look, so I will wait there response. Unless you know another solutions :-)
  1. more than a month ago
  2. RO CSVI
  3. # 2
tharuna
tharuna
Accepted Answer Pending Moderation
Hello,
One other solution would be uploading your images to VirtueMart default location before running the import. In the import file use image name in file_url field so it gets linked to the product. Since the image is already in default location, image processing time will be less and your import should finish. Let me know if it helps.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 3
Poiter
Poiter
Accepted Answer Pending Moderation
Hi Tharuna,

Unfortunally I don't have the pictures. Already asked my contact person of the manufactur to send them, but he can't he tells me, because there is no central image bank?

Can I only import the url? So the image shown on my webshop is the image on the manufactur server? https://plm.popal.nl/media/uploads/product_files/2614_c003e84c79dcc4ad569f37d9acad922b92ae52c5.jpg
  1. more than a month ago
  2. RO CSVI
  3. # 4
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Pieter,

Here are some thoughts. Increasing the execution time from 120 seconds to 4 minutes makes no difference if you have set to import 1 record and then wait 5 seconds. Every time it has waited 5 seconds it will restart the session and then the 120 seconds start recounting. One record will usually always fit within 120 seconds.

What is more important and very much needed is to know what that 500 error is. See the 500 Internal Server Error document for more details. Most likely your host will need to check the server error log. Without the real error, it is a shot in the dark as to why it happens.

You could set that CSVI doesn't create the thumbnails and have VirtueMart create the thumbnails to see if that helps. If the original images are 4MB, there is a good chance the 500 error is actually an out of memory error. In that case to be able to resize the images on your server, you would need to increase the memory limit.

VirtueMart doesn't support external URLs as images. So that is not a solution. As said before, you need to find out what is the 500 error. With that info you know what is really happening.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 5
Poiter
Poiter
Accepted Answer Pending Moderation
Hi Roland,

Yes that was what I thought, but because Tharuna talked about max_execution_time I asked this to my host. :-)
The memory limit is now 128, will ask my host to make it 256.

Thanks for your reply at this time!
  1. more than a month ago
  2. RO CSVI
  3. # 6
Poiter
Poiter
Accepted Answer Pending Moderation
Ok, my host changed the memory limit to 256. So I made an import again, but now there's a 503 error. But I see 100 records are added.
I made an import with a template of Virtuemart and product. In the templatefields I skipped every row I don't need. Only product_sku and file_url
Attached the server error log, 503 error and the xml form the maintenance page
Attachments (3)
server-error-log.png server-error-log.png
503error.png 503error.png
csvi_templates_fietspleinholland.nl_20171019.xml
  1. more than a month ago
  2. RO CSVI
  3. # 7
tharuna
tharuna
Accepted Answer Pending Moderation
Hello Pieter,
503 error occurs when the server is overloaded or not available at that moment. Your error log screenshot says the same. Did you try running import after sometime? Do you still see the same error?
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 8
Poiter
Poiter
Accepted Answer Pending Moderation
Hi Tharuna,

About 6 hours ago I made a new import and after importing 165 records he aborded with the 503 error. 2 hours ago again an import and after 164 records he aborded. I see those images are imported correct in Virtuemart. So I thought trew the first 164 rows in the csv file away and import the images that aren't imported. Now he's aborded after 3 records... :(
For both import I put the limits to 1 line
The last csv file is missing an image url on the first 3 lines. So I deleted that lines and tried to import again. And now it's aborted right away..
Very frustrating...
  1. more than a month ago
  2. RO CSVI
  3. # 9
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

And now it's aborted right away..
I would say that the image for that record is the issue. The question would be, what is so different with this image that causes it? Is the error you get also a 503 error? If so, the question would be why the server is overloading. Are you resizing the images or just importing them as-is?

Very frustrating...
Sure this is frustrating but it is also not really a CSVI issue but we try to help where we can.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 10
Poiter
Poiter
Accepted Answer Pending Moderation
Hello,

Are you resizing the images or just importing them as-is?


I did resize, because I don't want that big files on my server, but I turned this setting of and so far so good! Fingers crossed. Changed the lines to 10 and he is now up to 80!

Sure this is frustrating but it is also not really a CSVI issue but we try to help where we can.

It's no criticism to you and Tharuna. You 2 do a great job, wish I was more technical to understand things :-)

150 now!
  1. more than a month ago
  2. RO CSVI
  3. # 11
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Pieter,

I did resize, because I don't want that big files on my server
Understandable.

but I turned this setting of and so far so good!
This confirmed my suspicion. You're server is simply running out of resources to resize the images. You are going to need more memory to do the resizing in this case.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 12
Poiter
Poiter
Accepted Answer Pending Moderation
This confirmed my suspicion.

Well you got that right. Why didn't you tell me that earlier! :P All imported now!
Hosting is on shared hosting now, maybe and for webshop a must have a dedicated vps hosting I think. Will talk with the webshop owner about it

Thanks again for all the help and thinking!

Have a nice evening
  1. more than a month ago
  2. RO CSVI
  3. # 13
rolandd
rolandd
Accepted Answer Pending Moderation
Why didn't you tell me that earlier! :P
Getting slow :pinch:

Hosting is on shared hosting now
That is not ideal for resizing images. The process of resizing images just takes a lot of memory which is usually not available on shared hosting. The only other option would be to resize the images on a local computer if you plan to stay on a shared host.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 14
Poiter
Poiter
Accepted Answer Pending Moderation
That is not ideal for resizing images. The process of resizing images just takes a lot of memory which is usually not available on shared hosting. The only other option would be to resize the images on a local computer if you plan to stay on a shared host.


Yeah I know, but the manufacturer can't give me the pictures manually only form the url...
Looking into other hosting options right now

One thing, fyi, when I hit reply on this forum. I see a different profile picture of you. :-) Maybe bug?
  1. more than a month ago
  2. RO CSVI
  3. # 15
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

I see a different profile picture of you. :-) Maybe bug?
Seems to be some caching going on. I have seen it on some occasions, let's see if it will change over time.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 16
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!