1. roderic
  2. RO CSVI
  3. Thursday, 25 February 2021
  4.  Subscribe via email
Hello,

I am trying to import XML files from VendIT into Virtuemart. I'm struggling with it as these XML files Vendit offers, don't seem to suit CSVI. Could you please tell me if it is even possible with the attached XML files? The hierarchy makes it that it only imports the first item.. For example, in the groups.xml one, it will only import the "Bikes" category and not the ones in the subgroups. Is there a workaround?

Groups = categories.

Joomla 3.9.24
CSVI: 7.20.0
Attachments (2)
Accepted Answer Pending Moderation
Hello Roderic,
Now when this problem arised on Saturday, one of the things I tried was to use the Skip rule to skip records based on IsDeleted = true as that is the common node. This unfortunately didn't help.

Can you tell me what was the problem with the rule? It did not skip the rows on import? Please share a screenshot on the rule.

Next problem, I'm trying to improve the import of prices. Before I would import the discounted prices during Parent and Child import. Now I'm importing the normal price only and then importing the Discounted price ActionPrices/ActionPrice afterwards in a separate import so I can import the publish_up and publish_down properly. But eventhough I first import the normal prices and afterwards the actionprices, the order is reversed in the administration and this makes that the discounted price doesn't show up. I can also not swap them around manually.

There is no ordering field in VirtueMart for prices so whatever order you are seeing with prices is with VirtueMart code. From what i noticed, product prices are sorted based on the highest price. Since other prices are same but discount price is higher for one row, that is shown as first price.
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. one week ago
  2. RO CSVI
  3. # 1
Accepted Answer Pending Moderation
Good morning,

Here a screenshot of the rule:
skip-deleted.jpg

I tested it again just now and it worked this time. I changed the path to Products/Product/IsDeleted instead of Products/Product/ProductVariations/ProductVariation/IsDeleted

But this is not a great solution since the deleted ProductVariations (Childs) will now not be deleted but skipped. Or are Child products also deleted when the Parent get's deleted while running the Parent import? In that case all is fine.

I duplicated the website to test, reinstalled 7.20 and ran the Child template. It didn't halt on the products without ProductVariations like it does in 8.0 rc1.

The custom fields are stored in `#___virtuemart_customs` table and the relations of custom fields for all are stored in #__virtuemart_product_custom_plg_customsforall and #___virtuemart_custom_plg_customsforall_values tables. That is the reason when you set Delete custom relations to Yes, the values in these two tables are deleted and inserted as new rows which then gets new IDs.


Also in 7.20 it doesn't delete values from #__virtuemart_custom_plg_customsforall_values like it does in CSVI 8 with Delete custom fields relations set to Yes.
It also shouldn't delete those values because then when you reorder those values in a logical order in the VM Admin, you'll have to do it again after every import. So it should only remove the relations from #_virtuemart_product_custom_plg_customsforall and not the actual values from #__virtuemart_custom_plg_customsforall_values Can you fix this?


There is no ordering field in VirtueMart for prices so whatever order you are seeing with prices is with VirtueMart code. From what i noticed, product prices are sorted based on the highest price. Since other prices are same but discount price is higher for one row, that is shown as first price.


When I manually add a price, it will come below the original price and then the discount works (see screenshot). When I run the import with similar values, the discounted price comes on top and doesn't show up on the front-end.

manually-added-price.jpg

Also, there is this button next to the trash icon that enables you to sort the pricing. It allows you to drag one price-box over another. Then when you save it however, the price-boxes are back to the original position but the value for overriding has swapped. So basically there is a button but what is missing, is a column "ordering" in the VM table #_virtuemart_product_prices.

Cheers!
Attachments (2)
  1. one week ago
  2. RO CSVI
  3. # 2
Accepted Answer Pending Moderation
Hello Roderic,
I tested it again just now and it worked this time. I changed the path to Products/Product/IsDeleted instead of Products/Product/ProductVariations/ProductVariation/IsDeleted

But this is not a great solution since the deleted ProductVariations (Childs) will now not be deleted but skipped. Or are Child products also deleted when the Parent get's deleted while running the Parent import? In that case all is fine.

No, child products will not be deleted but the parent child reference will be reset when a parent is deleted. The product_parent_id field value will be set to 0 for all the child products when a parent is deleted.

You can have another import to delete the child products. Will that not work?

I duplicated the website to test, reinstalled 7.20 and ran the Child template. It didn't halt on the products without ProductVariations like it does in 8.0 rc1.

There were improvements made in code to read values from ProductVariations/ProductVariation node and a patch file was sent to you recently. With 7.20.0 version, were you able to read node ProductVariations/ProductVariation values? For example ProductVariations/ProductVariation/ProductId field.

Also in 7.20 it doesn't delete values from #__virtuemart_custom_plg_customsforall_values like it does in CSVI 8 with Delete custom fields relations set to Yes.
It also shouldn't delete those values because then when you reorder those values in a logical order in the VM Admin, you'll have to do it again after every import. So it should only remove the relations from #_virtuemart_product_custom_plg_customsforall and not the actual values from #__virtuemart_custom_plg_customsforall_values Can you fix this?

The issue of old values staying in database and creating conflicts was reported by you and was fixed for you. See attached screenshots and reply for it. Reseting the code will get the old issue back.

When I manually add a price, it will come below the original price and then the discount works (see screenshot). When I run the import with similar values, the discounted price comes on top and doesn't show up on the front-end.

I am not sure on what basis VirtueMart displays prices on frontend but with RO CSVI import, we look if there is a matching price row in database and if row exists it is updated or a new price row is inserted. The new inserted row takes the new increment ID and is added as a last row in the database table. We cannot have a row insert in-between existing rows or have any control over it.

Also, there is this button next to the trash icon that enables you to sort the pricing. It allows you to drag one price-box over another. Then when you save it however, the price-boxes are back to the original position but the value for overriding has swapped. So basically there is a button but what is missing, is a column "ordering" in the VM table #_virtuemart_product_prices.

Yes, thats correct. There is no ordering field in #_virtuemart_product_prices table. I noticed about the ordering icon in VirtueMart and also noticed it does not work.
Attachments (2)
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. one week ago
  2. RO CSVI
  3. # 3
Accepted Answer Pending Moderation
Hello Tharuna,

No, child products will not be deleted but the parent child reference will be reset when a parent is deleted. The product_parent_id field value will be set to 0 for all the child products when a parent is deleted.


Wouldn't this be a good option to have in the template settings? "On Parent product deletion, include deletion of childs: Yes/No"


You can have another import to delete the child products. Will that not work?


Sometimes the ProductVariations are missing and only the Parent product with IsDeleted=True is left, in this case this will not work. Otherwise yes, a separate import to delete child could work.

There were improvements made in code to read values from ProductVariations/ProductVariation node and a patch file was sent to you recently. With 7.20.0 version, were you able to read node ProductVariations/ProductVariation values? For example ProductVariations/ProductVariation/ProductId field.


Yes, in 7.20.0 this worked/works well with patchespatches applied from this topic. Version 8 broke this functionality but then the patch you mentioned brought back the functionality I had in 7.20.0+patches for the most part. But it's not 100% the same. I just read through a big part of this topic and I feel that most of the changes that were made in patches attached in this topic made it into 8.0.0 but not all of it.

The issue of old values staying in database and creating conflicts was reported by you and was fixed for you. See attached screenshots and reply for it. Resetting the code will get the old issue back.


Yes you are right. I am not sure if I used that patch on the live site in the last 4-5 months. I think I did not. That's why the customfield values stayed the same.
With delete customfields relations set to yes, What it should do, is remove values from:
#__virtuemart_product_customfields
#__virtuemart_product_custom_plg_customfieldsforall

What it should not do, is remove values from:
#__virtuemart_custom_plg_customfsforall_values

Why not?
- This messes up the order of the custom values and you have to reorder
- When you delete a value, it will also be removed from another product using that value. Meaning, you always have to do a full import otherwise there might be a few products with custom values missing.

So I think resetting the code would be the best option. I will then have to test if old values remain with a product or setting delete customfields relations to yes will be enough.

I am not sure on what basis VirtueMart displays prices on frontend but with RO CSVI import, we look if there is a matching price row in database and if row exists it is updated or a new price row is inserted. The new inserted row takes the new increment ID and is added as a last row in the database table. We cannot have a row insert in-between existing rows or have any control over it.


When I was testing this, I checked the database after every import and indeed saw that the newly imported prices indeed have a incremented ID.

UPDATE


I've been testing something and came to a few conclusions that I wanted to share.

The prices are being sorted by either cost, base or finalprice. Discounted price is being ignored. Thus, when those other 3 prices are the same, there is no proper ordering.

As soon as you change the costprice, the order changes. Even when it's a 0.0001 difference. The order is then based on the setting in configuration -> prices -> Order of multiple prices

It would be good when Virtuemart actually takes the discount field into account, but it doesn't. I will report this bug (?) on the VM forum.

As a workaround I've made a Margin rule that subtracts 0.0001 from the product_price field in the Action prices template. This then helps with the sorting.

prices-substracted.jpg


Cheers.
Attachments (1)
  1. one week ago
  2. RO CSVI
  3. # 4
Accepted Answer Pending Moderation
Hello Roderic,

Wouldn't this be a good option to have in the template settings? "On Parent product deletion, include deletion of childs: Yes/No"
That is a fair request and added it to the feature list. I cannot give you a timeframe as to when this will be available but it is planned for version 8.1.0.

Yes, in 7.20.0 this worked/works well with patchespatches applied from this topic. Version 8 broke this functionality but then the patch you mentioned brought back the functionality I had in 7.20.0+patches for the most part. But it's not 100% the same.
I really wonder what is different because I am not aware of giving you one patch and include another one in the release.

I just read through a big part of this topic and I feel that most of the changes that were made in patches attached in this topic made it into 8.0.0 but not all of it.
Yes, it has become a huge thread, perhaps we should start a new one with what you believe is missing in the current version and we can focus on that.

What it should not do, is remove values from:
#__virtuemart_custom_plg_customfsforall_values
I see what you are saying here and I believe we should extend it a bit more. Instead of deleting or not deleting, we should delete the values if there are no other products attached to it. This way you can at least do some housekeeping to keep things sort of clean.

It would be good when Virtuemart actually takes the discount field into account, but it doesn't. I will report this bug (?) on the VM forum.
Some may call it a feature :D
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. one week ago
  2. RO CSVI
  3. # 5
Accepted Answer Pending Moderation
Good morning,

I really wonder what is different because I am not aware of giving you one patch and include another one in the release.


So, in 7.20 it was able to read all the values from ProductVariations/ProductVariation/
After the update that stopped working. Then you gave me a new patch and it worked again with a few differences. Maybe CSVI has become more strict in general? I ran into a few issues that I fixed that should've come up before as well.

Yes, it has become a huge thread, perhaps we should start a new one with what you believe is missing in the current version and we can focus on that.


I will post further questions in a new topic, except for the next one as it's related

I see what you are saying here and I believe we should extend it a bit more. Instead of deleting or not deleting, we should delete the values if there are no other products attached to it. This way you can at least do some housekeeping to keep things sort of clean.


Did you get any further with this? For the last week (or 2) I've been running the import with Delete customfield relations set to No. This at least makes sure that the custom ordering of values for every customfield stays intact. However, I'm running into the following problem because of this:

After I've imported a fresh parent product, then child, it looks like this:

first-import-good.jpg

That's the desired result.

Then after the child template another time and every additional time it looks like:

2nd-import-notsogood.jpg

The functionallity in the shop still seems to be intact. However, it adds empty "Maat:" values.

empty-value.jpg

Which then can be seen in the Cart and order e-mails: (looks worse after several imports)

mates.jpg


When Delete customfield relations set to Yes, running the import deletes all the empty Maat values. But of course it also destroys the ordering like said before.

Now I'm wondering why it doesn't use the value that's already there but instead inserts a new one. This only happens with stockablecustomfields and not with customfieldsforall.

Cheers.
Attachments (4)
  1. one week ago
  2. RO CSVI
  3. # 6
Accepted Answer Pending Moderation
Hello Roderic,

So, in 7.20 it was able to read all the values from ProductVariations/ProductVariation/
Looking at the initial XML file you sent us, I can also see all values. The last XML posted in this thread has only 1 product but also invalid XML markup, so that won't read all together. As said, we can pick this up in the new topic.

Did you get any further with this?
Not yet due to some unforeseen circumstances. Will look into it over the next few days.

Now I'm wondering why it doesn't use the value that's already there but instead inserts a new one.
We do not posses all the knowledge when we build something, so at the time it made sense to delete the old values and just add the new ones so you do not get duplicates. The ordering is something that simply got missed.

This only happens with stockablecustomfields and not with customfieldsforall.
That can happen because it is completely different code.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. one week ago
  2. RO CSVI
  3. # 7
Accepted Answer Pending Moderation
Hello Roderic,

So I have created and tested a change in the logic where the entries in the #__virtuemart_custom_plg_customsforall_values table are only deleted if no other product is referencing them.

Could you give this patch a run and see if it works as you would expect?
Attachments (1)
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. 4 days ago
  2. RO CSVI
  3. # 8
Accepted Answer Pending Moderation
Hi Roland,

So I've tested this on a test site with 3 products containing the same value (One Size) for a customfield (Maat: ). I run the import with 2 of these products. After running the import, the value for the 3rd product is removed. Which it obviously shouldn't. Log is attached.

Cheers
Attachments (1)
  1. 3 days ago
  2. RO CSVI
  3. # 9
Accepted Answer Pending Moderation
Hey Roderic,

You are absolutely right, I missed to include the multi-language portion of this routine. Attached is a new patch file that includes the multi-language part in the check as well. Can you see if it works now for you?
Attachments (1)
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. 1 day ago
  2. RO CSVI
  3. # 10
Accepted Answer Pending Moderation
Hoi Roland,

I've tested a few times and it still deletes the values. I notice that when I attach a value to a product that's not in the import, the ID of the value will stay the same after the first import. Then running the import again will remove the value.

Before first import - one product with Maat value S:

before-first-import.jpg before-first-import-db.jpg

After first import:

after-first-import.jpg after-first-import-db.jpg

ID of S has stayed the same but value has been removed from product. M - L - XL ID's have changed as there was no product with that value.

After 2nd import:

after-2nd-import.jpg

ID has changed because there was no more product with that value as it was disconnected after first import.


Also, besides S, M-L-XL are being imported in the first import for those products. When running the same import with the same file/products again, why delete the values?


Cheers
Attachments (5)
  1. 5 hours ago
  2. RO CSVI
  3. # 11
Accepted Answer Pending Moderation
Hello Roderic,
Can you please post the import debug log of all three imports? I ran import here with patch file and don't see the values removed from product. Need to check what is different in your import to reproduce the issue.
Kind regards,

Tharuna

=========================
If you use CSVI, please post a rating and a review at the Joomla! Extension Directory
  1. 4 hours ago
  2. RO CSVI
  3. # 12


There are no replies made for this post yet.
Be one of the first to reply to this post!