Resolved XML error: Mismatched tag at line 81

Dear Tharuna

I have been engaged in a new project and last time I saw that the 4 sample products contained in Fiaber2 are complete with description. But now rechecking a sample product contained in Fiaber3 does not have the description while in the log file the data of icecat are present. this is the product code: HP3DXPLABK
I have a doubt
Could it be that it doesn't overwrite the old data in virtuemart?
Since you have access you can check if I have configured the templates correctly.
Thank you

Hello Lorenzo,

But now rechecking a sample product contained in Fiaber3 does not have the description while in the log file the data of icecat are present. this is the product code: HP3DXPLABK

I had a look at debug log of FIABER3 which ran today and it shows product description has been updated. Yesterday's debug log for FIABER3 template shows empty debug log as 0 records were processed. Import on which date did not have the description?

Could it be that it doesn't overwrite the old data in virtuemart?
Since you have access you can check if I have configured the templates correctly.

Yes I had a look at it. Your template settings are correct. The setting for overwriting data is on File tab and is set to Yes so that is not the problem.

Dear tharuna
Yes it was empty because the TMP folder was empty.
However I had also checked on Sunday and the icecat description was not there. See log Fiaber3 dated 02/04/2023.
Greetings

Hello Lorenzo,
Yes I had a look at Fiaber3 debug log for 02/04/2023 and see product description been added. Please see attached screenshot from debug log.

Dear Tharuna

Yes, I have seen but these data are not in the product sheet. No they are copied to the DB.
Why?

See product:
https://www.fatelettronica.it/pla-pha/bobina-pla-1kg-x-comp-hp3dxplabk.html

Tanks

Lorenzo

Hello Lorenzo,
The product link you posted shows product has a description as it is updated in import debug log. See attached screenshot. Not sure if it is with latest import but there is description or I am missing something?

Dear Tharuna
I told you that this morning and in recent days there was no icecat description.
Today the cronjob crashed and I had to restart it at 12:01 local time and it finished at 13:24. Soon after you saw that now there is the description.
You see it's a strange thing, almost random.
It seems that CSVI doesn't see well if there is information from icecat and it doesn't skip the description of my csv file. But it does it randomly and not always
Thanks

Lorenzo

Hello Lorenzo,

So I am going to look into this as well. First thing I would like to ask is, are there any products without a description since the last import? What I want to do is check the logfile and see if I can see and understand why this might have failed for a particular product.

Thank you.

Dear Tharuna

Yesterday I checked the latest added products and everything was fine. Yesterday evening I reloaded the icecat data. This morning I find the latest Fiaber2 products with photos but no description. If you check the log you see that icecat description data is there but it didn't transfer it to the DB. See product with MPN : 90IG06K0-MO3R10
A copy of the info email at the end of the cronjob may be useful. attached.

Hello Lorenzo,

Thank you for the info.

So the product 90IG06K0-MO3R10 has the URL https://www.fatelettronica.it/router-wifi/asus-rog-strix-gs-ax3000-router-dual-band-wi-fi-ax-4-porte-gigabit-usb-3-2-aimesh-2402-574-mbps.html

This product does have a description, albeit short but there is one. This is the description that comes from your import file I believe. So when you say that there is no description I guess you mean not the description from ICEcat.

The product XML for 90IG06K0-MO3R10 has no PROD_LONGDESC field in it's XML, that is why the description from your import file is used and not the one from ICEcat.

So to clarify what is happening here is that some XML sheets do have the PROD_LONGDESC so the description is updated with the ICEcat data, the XML sheets that do not have it, are not updated with ICEcat data but with what is in the CSV file.

Let me know what you think.

dear Roland

"So when you say there is no description, I guess you don't mean the description of ICEcat."

Yes, that's right, the description of ICECAT is missing.

I checked log file #1. 281 ( Fiaber2 of 04/13/2023 finished at 06:25) and I point out that the PROD_LONGDESC field is present only in the first lines (products) up to line 363. From 364 to the end it is no longer present.
all fields after the "Connection closed" field are missing. Immediately after, from line 364 the "Empty Product desc" field is missing and all the others up to the "Use field value for field product_sku" field.
What happen ?
Is something preventing you from running the "Empty Product desc" command?

As I wrote in the previous post, this thing is also random. Yesterday it broke down at line 363 but it's not repetitive.
I checked log file no. 270 (Fiaber2 of 04/12/2023 finished at 06:20) and there the PROD_LONGDESC field is present in all log lines, 4985 out of 6048, because the log was interrupted.
I also see a strange thing on line 4983 right after the PROD_LONGDESC field.

You have to understand why this thing happens if we will never get to the solution.

Gratings

Lorenzo

Yesterday I upgraded version 8.6.1 to 8.7.0.
It seems to me that we are back to the old way of handling ICECAT data. Indeed in the cronjob Fiaber3 stopped with the error:
Template FIABER3
XML error: > required at line 78.
Fiaber1 also aborted with error:
Template FIABER1
error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Only Fiaber2 concluded the job.
The PROD_LONGDESC field does not exist in the Fiaber2 log file.
Now I have manually started the Fiaber1 and 3 templetes. I hope it ends well.
Thank you

Lorenzo

Hello Lorenzo,

You have to understand why this thing happens if we will never get to the solution.

That is what I am doing and I think there are several different issues mixed up here, so I am going to try to keep them separate.

First is the PROD_LONGDESC field, In the one product I checked in my previous post, that field is not in the XML, so no ICEcat description will be imported.

all fields after the "Connection closed" field are missing. Immediately after, from line 364 the "Empty Product desc" field is missing and all the others up to the "Use field value for field product_sku" field.
What happen ?

What happened here is that only a partial XML response has been received. If you look at line 59498 in the log file, this ends in the middle of a sentence.

langid=\"5\" Value=\"Composizione del pro"

This is why this particular product is not imported. The message Closed connection is just a simple log message, not an error. The product on line 363 is the last one to receive an XML from ICEcat that has the PROD_LONGDESC.

Is something preventing you from running the "Empty Product desc" command?

Yes, because the description does not come back from the ICEcat data.

I checked log file no. 270 (Fiaber2 of 04/12/2023 finished at 06:20) and there the PROD_LONGDESC field is present in all log lines, 4985 out of 6048, because the log was interrupted.

That is actually not correct. In logfile 270 I checked the 80171412.xml on line 60572 (product 351) and the XML retrieved has an empty ProductDescription, so there is no description from ICEcat. The log file is interrupted I think because it is getting too big.

I also ran the import with one XML that did not read the description but of course in my test it did read the description.

Since you have updated and I have made some more changes to the logging and processing, I am attaching a new patch file. Please load this patch file and we can see what the new log will tell us.

Dear
Now I see two problems:
1) The FIABER3 cronjob fails after three attempts. An email arrives from the server, different from the others, where there are lines of php instead of the error report or something else. I put it in the attachment together with the txt file of the data to be loaded. I don't understand why only FIABER three fails like the others.
2) with the new patch the log files contain an excessive number of lines which also exceed the number of buffering bytes and therefore cannot be complete.

Best regards

Lorenzo

Hello Lorenzo,

This is not actually the RO CSVI logging but rather a long list of notices because the manufacturer product number cannot be found. I have optimized the import to not process ICEcat if there is no MPN.

As for the excessive logging, this was a bit added to see why the description is not being loaded. Now I have removed that and gone back to the old logging.

Please load the attached patch file for that. Let's see if that tells us more.

Dear
After adding your patch I was busy.
For commercial reasons I have removed the publication of some products (FIABER2 is empty) and added a new commercial category. For this reason I added a FIABER4 model where I set not to upload photos (file_URL) since they are provided to me. Compared to the other models it has only this difference.
If you go to see these products, there is no description either in my csv file or in ICECAT. Of course I checked in the log file where the icecat data is.
I'm writing you two MPNs to use for verification:
HDD-3000GB
HDD-6000GB
If instead the ICECAT data are not present, the description of my CSV file is inserted in the product. See these two MPNs:
VIKING-R2-8TB-V2
VIKING-VMS-CLIENT-I5-V3

In summary, new products have no description if ICECAT has one.
Maybe this helps more to understand the problem.

Greetings

Lorenzo

Hello Lorenzo,

After adding your patch I was busy.

No problem, take your time.

So I checked the FIABER4 logfile.

HDD-3000GB
This ICEcat data has no description and if your CSV file has no description, in that case it will be empty.

HDD-6000GB
Same as the 3000GB one, this ICEcat data has no description either.

In summary, new products have no description if ICECAT has one.

That is not correct because these 2 reference products have no description in their ICEcat XML data. Are you seeing the description somewhere? RO CSVI uses the field LongDesc and that is not in the XML data.

Yes, in fact they don't have an ICECAT description. In my import file instead there is a description but it is not written in the database. see product. https://www.fatelettronica.it/hdd-server-video/ernitec-hdd-3000gb-3tb-hdd-sata-24-7.html

Greetings

Now, after the last cronjob, I see that the description of ICECAT is in the reported product HDD-3000GB. It took at least 3 cron jobs to finish getting ICECAT data into the product.

See you in the log files

Greetings

Hello,

In my import file instead there is a description but it is not written in the database.

I do not have the import file so I cannot see. Can you post the file you have used? In this case it would be the import file used on April 21st.

It took at least 3 cron jobs to finish getting ICECAT data into the product.

That is not an issue of the cronjob but rather what data comes back from ICEcat.

So I have been running some more tests but cannot reproduce what I see in your logfile. What I need is the actual CSV file used for a given import. Is the import file always the same or is this automatically generated?