1. roderic
  2. RO CSVI
  3. Thursday, 25 February 2021
  4.  Subscribe via email
Hello,

I am trying to import XML files from VendIT into Virtuemart. I'm struggling with it as these XML files Vendit offers, don't seem to suit CSVI. Could you please tell me if it is even possible with the attached XML files? The hierarchy makes it that it only imports the first item.. For example, in the groups.xml one, it will only import the "Bikes" category and not the ones in the subgroups. Is there a workaround?

Groups = categories.

Joomla 3.9.24
CSVI: 7.20.0
Attachments (2)
Accepted Answer Pending Moderation
Hello Roderic,
That indeed sounds like the best and most logical solution.

Attached patch file will update has_medias field when Update media links is set to Yes. Load the patch file, try the import and let me know.

The problem in my case is as well that you need to set the file_title / file_description for every media file. But when you use combine to get multiple images, how to get multiple values for file_title / file_description as well?

May i ask why not set Image name based on to Product name field instead of Product ID? This way with Fill media fields set to Yes, image name will be used for file_title / file_description which will be product name and you will achieve it in a simpler way than with regex rule.

This will replace every file_url I've combined into "image". Now if I could replace it with the value of product_name instead, like you can with conditional field plugin, I would achieve the desired result as well. This could also be helpful in many other situations.

The problem here is that with Find and Replace you can replace with a given text but not with a value of field. With conditional plugin you can check for a value and replace with value from product_name field but in your case it is multiple values combined with |. Check if Image name based on product name solution will help in solving this issue.

This gives the same problem as above. But even when I set this as default value, it unfortunately won't help. I've tried many things.

You mean setting custom_ordering field does not work for you? Can you explain this in detail?
Attachments (2)
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 1
Accepted Answer Pending Moderation
Hi Tharuna,

Attached patch file will update has_medias field when Update media links is set to Yes. Load the patch file, try the import and let me know.


Thanks, will try later.


May i ask why not set Image name based on to Product name field instead of Product ID? This way with Fill media fields set to Yes, image name will be used for file_title / file_description which will be product name and you will achieve it in a simpler way than with regex rule.


Because when I try to import with the setting set to Product name, and you want it to have an effect, you need to set Create image name to yes, it will then not be able to find the images. I know realize the Fill media fields doesn't obey the Image name based on when Create image name is set to no.

So the actual problem might be that with Create image name set to yes, it tries to rewrite the filename although it maybe should only set the file_title?


Warning: getimagesize(/pathtomywebsite/images/virtuemart/product/Specialized_Recon_2.0_Mountainbike_Schoenen_1.jpg): failed to open stream: No such file or directory in /pathtomywebsite/plugins/csviaddon/virtuemart/com_virtuemart/model/import/product.php on line 1777


You can login to the test site and change the settings on the parent template and run a few tests if you like.


The problem here is that with Find and Replace you can replace with a given text but not with a value of field. With conditional plugin you can check for a value and replace with value from product_name field but in your case it is multiple values combined with |. Check if Image name based on product name solution will help in solving this issue.


Yes, that's what I tried to describe, you've used a few less words ;) So I'm asking to have one of the plugins changed/updated to make them (and thus also CSVI in general) even more powerful. Not just for this situation.

You mean setting custom_ordering field does not work for you? Can you explain this in detail?


You mean file_ordering. So, here is an example product (trimmed):


<Product>
<PageTitle>Brooks Glycerin 19 dames</PageTitle>
<ProductVariations>
<ProductVariation>
<ProductId>166415-782</ProductId>
<Size>7</Size>
<Color>Black/Ombre/Primrose</Color>
<Images>
<Image ImageOrder="0">CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg</Image>
<Image ImageOrder="1">69504C8E-B898-4020-B01B-4D924258A805.jpg</Image>
<Image ImageOrder="2">406681BB-A258-49B3-964F-31992ABE5381.jpg</Image>
<Image ImageOrder="3">EB624DD2-179A-4900-B23F-85C5622B3DC2.jpg</Image>
<Image ImageOrder="4">A75E6074-A5A6-49CB-9A9F-9EBDA2F5EABD.jpg</Image>
<Image ImageOrder="5">C6966F9E-4581-4EC9-92C8-014C328D3ABF.jpg</Image>
</Images>
</ProductVariation>
<ProductVariation>
<ProductId>166415-783</ProductId>
<Size>8</Size>
<Color>Black/Ombre/Primrose</Color>
<Images>
<Image ImageOrder="0">CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg</Image>
<Image ImageOrder="1">69504C8E-B898-4020-B01B-4D924258A805.jpg</Image>
<Image ImageOrder="2">406681BB-A258-49B3-964F-31992ABE5381.jpg</Image>
<Image ImageOrder="3">EB624DD2-179A-4900-B23F-85C5622B3DC2.jpg</Image>
<Image ImageOrder="4">A75E6074-A5A6-49CB-9A9F-9EBDA2F5EABD.jpg</Image>
<Image ImageOrder="5">C6966F9E-4581-4EC9-92C8-014C328D3ABF.jpg</Image>
</Images>
</ProductVariation>
<ProductVariation>
<ProductId>166415-784</ProductId>
<Size>9</Size>
<Color>Black/Ombre/Primrose</Color>
<Images>
<Image ImageOrder="0">CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg</Image>
<Image ImageOrder="1">69504C8E-B898-4020-B01B-4D924258A805.jpg</Image>
<Image ImageOrder="2">406681BB-A258-49B3-964F-31992ABE5381.jpg</Image>
<Image ImageOrder="3">EB624DD2-179A-4900-B23F-85C5622B3DC2.jpg</Image>
<Image ImageOrder="4">A75E6074-A5A6-49CB-9A9F-9EBDA2F5EABD.jpg</Image>
<Image ImageOrder="5">C6966F9E-4581-4EC9-92C8-014C328D3ABF.jpg</Image>
</Images>
</ProductVariation>
</product>



As you can see, every product variation has the same set of images. In this case 6 images but it can be any number from 1 to max 10. So I've set the parent template like:

roland-images2.jpg

Resulting in this import preview:


CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg|69504C8E-B898-4020-B01B-4D924258A805.jpg|406681BB-A258-49B3-964F-31992ABE5381.jpg|EB624DD2-179A-4900-B23F-85C5622B3DC2.jpg|A75E6074-A5A6-49CB-9A9F-9EBDA2F5EABD.jpg|C6966F9E-4581-4EC9-92C8-014C328D3ABF.jpg|CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg|69504C8E-B898-4020-B01B-4D924258A805.jpg|406681BB-A258-49B3-964F-31992ABE5381.jpg|EB624DD2-179A-4900-B23F-85C5622B3DC2.jpg


As you can see, the 6 images + 4 duplicates. The import results in:


roland-image-order.jpg


Now when I change the the file_ordering to be file_title, I can show the problem:

roland-image-order2.jpg

The first set of image values are ignored (or overwritten) and it starts with the 2nd set instead. If the import could be changed that it actually ignores the duplicate values instead, that would fix this issue.


Cheers,

Roderic
Attachments (3)
  1. more than a month ago
  2. RO CSVI
  3. # 2
Accepted Answer Pending Moderation
Hello,
Because when I try to import with the setting set to Product name, and you want it to have an effect, you need to set Create image name to yes, it will then not be able to find the images. I know realize the Fill media fields doesn't obey the Image name based on when Create image name is set to no.

Thats correct, you need to set Create image name to yes and Image name based on Product Name. This will then change the image name and based on this new image name(product name) file_title/file_description will be created.

So the actual problem might be that with Create image name set to yes, it tries to rewrite the filename although it maybe should only set the file_title?

No, that is not the way. Create image name should be set to Yes, this will change image filename based on Image name based on field. This image name will be used for file_title field when Fill media fields is set to yes. They are all inter connected.

Yes, that's what I tried to describe, you've used a few less words ;) So I'm asking to have one of the plugins changed/updated to make them (and thus also CSVI in general) even more powerful. Not just for this situation.

You don't need to change the current rule plugins. You can write one for yourself. Check Writing a rule plugin document to know on how to write a new rule for RO CSVI.

As you can see, every product variation has the same set of images. In this case 6 images but it can be any number from 1 to max 10. So I've set the parent template like:

I think i understood the issue here. In your parent product import template file_ordering field is empty. When there is empty file_ordering, RO CSVI uses a counter in code to increment the ordering for the number of images imported. Since your import has 10 where 6 are correct and 4 are duplicated. The duplicated 4 gets incremented ordering and that replaces the existing ordering. From the sample XML you posted above, i see you have ImageOrder="0" field with ordering value. You can use this field with combine template fields and combine values like you did for images. When you do that your ordering should become 0|1|2|3|4|5|0|1|2|3. This way ordering of the image stays the same even with duplicated values. Try this and let me know if it fixes the issue.
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 3
Accepted Answer Pending Moderation
Hi again,

Thats correct, you need to set Create image name to yes and Image name based on Product Name. This will then change the image name and based on this new image name(product name) file_title/file_description will be created.


Yes I figured it should work like that, but then I get the error message I posted in my previous post:


Warning: getimagesize(/pathtomywebsite/images/virtuemart/product/Specialized_Recon_2.0_Mountainbike_Schoenen_1.jpg): failed to open stream: No such file or directory in /pathtomywebsite/plugins/csviaddon/virtuemart/com_virtuemart/model/import/product.php on line 1777


In your parent product import template file_ordering field is empty


Currently yes, but I had all kind of values there.

i see you have ImageOrder="0" field with ordering value. You can use this field with combine template fields and combine values like you did for images

I actually tried this yesterday but it didn't work.

Ok after some more testing, It does work, but it doesn't accept 0's. So I made a rule to replace the 0 values with a 1. Then another rule to turn the existing 1 to a number higher than 10 so it won't override the real 1. Now the 0 in the XML file is the real number 1 (VM media file ordering starts with 1).

Cheers!
  1. more than a month ago
  2. RO CSVI
  3. # 4
Accepted Answer Pending Moderation
Hello Roderic,
Warning: getimagesize(/pathtomywebsite/images/virtuemart/product/Specialized_Recon_2.0_Mountainbike_Schoenen_1.jpg): failed to open stream: No such file or directory in /pathtomywebsite/plugins/csviaddon/virtuemart/com_virtuemart/model/import/product.php on line 1777

The error says that image in the location /pathtomywebsite/images/virtuemart/product/Specialized_Recon_2.0_Mountainbike_Schoenen_1.jpg does not exists. Can you check if you have that image in the path?

Ok after some more testing, It does work, but it doesn't accept 0's. So I made a rule to replace the 0 values with a 1. Then another rule to turn the existing 1 to a number higher than 10 so it won't override the real 1. Now the 0 in the XML file is the real number 1 (VM media file ordering starts with 1).

Glad to know you found a workaround for that. So the issue with file_ordering field is also solved?
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 5
Accepted Answer Pending Moderation
Good morning,

The error says that image in the location /pathtomywebsite/images/virtuemart/product/Specialized_Recon_2.0_Mountainbike_Schoenen_1.jpg does not exists. Can you check if you have that image in the path?


Yes I can read ;) and no the image is not in that path, that is what I was trying to explain. The filenames are like:


<Image ImageOrder="0">CDBD539B-A69C-4300-A4A7-F549A66B7EB3.jpg</Image>


So when I have Create image name set to No, the images are imported. As soon as I turn it On, it will look for non-existing pictures. As if it skips the actual renaming of files, then trying to import them. It also happens when I choose product_id:


Warning: getimagesize(images/virtuemart/product/7952_1.jpg): failed to open stream: No such file or directory in /plugins/csviaddon/virtuemart/com_virtuemart/model/import/product.php on line 1777



So expectation:
RO CSVI renames the files to the chosen setting, then imports them to the database with that name.

What seems to happen?
RO CSVI skips renaming the actual file and then tries to import the files that then logically do not exist.


Glad to know you found a workaround for that. So the issue with file_ordering field is also solved?


Yes, today I've implemented it also on the live site and improved it a bit. I made a conditional rule that makes every number +1. This way the problem with 0's is solved and the order stays the same.

roland-upanumber.jpg



Cheers!
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 6
Accepted Answer Pending Moderation
Hello,
The problem is that the original image is not renamed to product name as image name. That is because of setting Convert image type. For renaming an image we check this field is set to none. In your case it was set to JPG and so rename of image did not happen. That lead to Image does not exist error.
Screenshot 2021-06-03 at 5.27.33 PM.png

So i changed that setting and tried the import. Now i am seeing 500 Internal server error. This error does not give any details on why it is happening. Can you check your server error logs and get the details of this error? You can post it here so we can check.
Attachments (1)
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 7
Accepted Answer Pending Moderation
Hi Tharuna,

In your case it was set to JPG and so rename of image did not happen. That lead to Image does not exist error.


So if it would've been a PNG/GIF, it would have converted it and changed the name?

So i changed that setting and tried the import. Now i am seeing 500 Internal server error. This error does not give any details on why it is happening. Can you check your server error logs and get the details of this error? You can post it here so we can check.



mod_fcgid: stderr: PHP Warning: getimagesize(--/images/virtuemart/product/suplest_edge+_xc/gravel_performance_xc/gravel_mountainbike_schoenen_4.jpg): failed to open stream: No such file or directory in ---/plugins/csviaddon/virtuemart/com_virtuemart/model/import/product.php on line 1780, referer: https://csvi.woest-sport.nl/administrator/index.php?option=com_csvi&view=import


Ok so it seems it does work but it runs into an error because of the + sign. I guess I will need a patch that filters out the plus?

Roderic
  1. more than a month ago
  2. RO CSVI
  3. # 8
Accepted Answer Pending Moderation
Hello Roderic,
So if it would've been a PNG/GIF, it would have converted it and changed the name?

Yes, we check if the image extension with the extension set in Convert image type field to rename. You can try setting it any other option other than jpg in the list. Recommend option is Do not convert.

Ok so it seems it does work but it runs into an error because of the + sign. I guess I will need a patch that filters out the plus?

I see the + sign in the path of the image. May i ask where is the suplest_edge+_xc folder coming from?
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 9
Accepted Answer Pending Moderation
Hi Tharuna,


I see the + sign in the path of the image. May i ask where is the suplest_edge+_xc folder coming from?


It's simply the (rather long and stupid) Product name.

And maybe the forward slash in the product name is the actual problem then.


mod_fcgid: stderr: PHP Warning: getimagesize(/images/virtuemart/product/Shimano_Ultegra/XT_HG93_9-Speed_Ketting.jpg): failed to open stream: No such file


Link to product

Thanks
  1. more than a month ago
  2. RO CSVI
  3. # 10
Accepted Answer Pending Moderation
Hello Roderic,
And maybe the forward slash in the product name is the actual problem then.

Yes, the forward slash is the problem not the +. With the slash the name is considered as folder name and so it says image does not exists. Try to replace / with any character withRO CSVI Multi replace rule and see if it solves the problem.
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 11
Accepted Answer Pending Moderation
Hello Roderic,
And maybe the forward slash in the product name is the actual problem then.

Yes, the forward slash is the problem not the +. With the slash the name is considered as folder name and so it says image does not exists. Try to replace / with any character withRO CSVI Multi replace rule and see if it solves the problem.


I did after my post, and the import runs without error.

https://csvi.woest-sport.nl/en/bikes/remmen/suplest-edge-sport-crosscountry-mountainbike-schoenen-detail

But as you can see, images are being duplicated. Before it would see that the image is the same, but now with renaming, it thinks every image is unique...

So, any idea how to fix this?

Cheers
  1. more than a month ago
  2. RO CSVI
  3. # 12
Accepted Answer Pending Moderation
Hi again,

Another problem that I almost forgot about:

When my client changes the color, the new color gets imported but the old value stays. (Delete custom fields relations set to yes)
roland-oldvalue.jpg

When I open the product in the backend, it doesn't list the old color value:

roland-colors.jpg


So then when I hit save, the old value is gone. How to trigger this automatically?

Roderic
Attachments (2)
  1. more than a month ago
  2. RO CSVI
  3. # 13
Accepted Answer Pending Moderation
Hello Roderic,
I did after my post, and the import runs without error.

https://csvi.woest-sport.nl/en/bikes/remmen/suplest-edge-sport-crosscountry-mountainbike-schoenen-detail

But as you can see, images are being duplicated. Before it would see that the image is the same, but now with renaming, it thinks every image is unique...

Image names are not same so it thinks every image is unique and creates a new record. Can you send me the debug log of this import to check? To get the debug log check How to collect debug information? document.

When my client changes the color, the new color gets imported but the old value stays. (Delete custom fields relations set to yes)

With delete custom field relation set to yes, we were not deleting values from custom fields for all tables and so old values were left. Load the attached patch file and run the import again. This should remove the old values and insert the new values for products.
Attachments (1)
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 14
Accepted Answer Pending Moderation
Hello Tharuna,


Can you send me the debug log of this import to check?


Attached

With delete custom field relation set to yes, we were not deleting values from custom fields for all tables and so old values were left. Load the attached patch file and run the import again. This should remove the old values and insert the new values for products.


I installed the patch on the test site and now get this error:


2021-06-04 13:08:17 4 [QUERY] SELECT `customsforall_value_id` FROM `xxx_virtuemart_product_custom_plg_customsforall` WHERE `virtuemart_product_id` = 7954 AND `customfield_id` = 494634
2021-06-04 13:08:17 0 [DEBUG] You have an error in your SQL syntax; check the manual that corresponds to your DB server version for the right syntax to use near ')' at line 3
2021-06-04 13:08:17 0 [QUERY] DELETE FROM `xxx_virtuemart_custom_plg_customsforall_values` WHERE `customsforall_value_id` IN ( )



Cheers
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 15
Accepted Answer Pending Moderation
Hello Roderic,
I see the issue with images. You have duplicate images with file_url field so with rename the file goes with product name and a increment value. Like for image CE289D45-24ED-4419-B041-8797B65E7D0E.jpg image, after renamed it takes names as suplest_edge+_sport_crosscountry_mountainbike_schoenen_1.jpg and suplest_edge+_sport_crosscountry_mountainbike_schoenen_3.jpg which is the same image but different names. When checking in database the renamed value is checked, it does not exist and so new record is created. I guess this is because of the combine field of ProductExport/Products/Product/ProductVariations/ProductVariation/Images/Image node ranging from 1 to 10 and having duplicate images.


SELECT `virtuemart_media_id` FROM `#__virtuemart_medias` WHERE `file_url` = 'images/virtuemart/product/suplest_edge+_sport_crosscountry_mountainbike_schoenen_1.jpg'

SELECT `virtuemart_media_id` FROM `#_virtuemart_medias` WHERE `file_url` = 'images/virtuemart/product/suplest_edge+_sport_crosscountry_mountainbike_schoenen_3.jpg


There is no direct way to do this but you can write some custom code and create a RO CSVI rule plugin to read values of file_url field, remove duplicates and update the value to file_url field.

I installed the patch on the test site and now get this error:

I have installed updated patch on your test site. Please check import and let me know.
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 16
Accepted Answer Pending Moderation
I have installed updated patch on your test site. Please check import and let me know.


That worked, thanks!

There is no direct way to do this but you can write some custom code and create a RO CSVI rule plugin to read values of file_url field, remove duplicates and update the value to file_url field.


That sounds complicated but I can give it a go.

Have a good weekend!
  1. more than a month ago
  2. RO CSVI
  3. # 17


There are no replies made for this post yet.
Be one of the first to reply to this post!