Dear Tharuna
I haven't been able to finish uploading data for a week. I don't understand what the error means and I don't see anything strange from the log file. I also checked the data contained in the upload file but nothing.
Can you tell me what I can do to fix it?
Greetings
Hello,
Do you see this error on import? Can you give us more details on the error? Post the import file, template XML and import debug log to check. To get debug log see How to collect debug information? document. To get template XML file, in RO CSVI maintenance menu select RO CSVI on first option, backup templates on second option, select the import template and download the XML file. Post the XML file.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
The Line number changes when I edit the file to import.
I am attaching what you requested
Greetings
Hello,
Thank you for posting requested files. There is nothing wrong I see from import file or the template XML or the debug log. I did a sample import with 5k records from your import file and with your template and import goes without any error.
Please try the below
- I see you are using RO CSVI 8.3.2 version, Please update to latest RO CSVI 8.6.1.
- Run import as cron job and see if import finishes without error. See Setting up a cron job in RO CSVI document to set import as cron job.
Let me know on how it goes.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Normally I've been using cron to upgrade data for a long time now and I've never had this kind of problem. Yesterday I updated the csv file and this morning after the end of the cron I get an email from the server with this error:
Use --help to see the help information
Processing started...
Template FIABER
XML error: > required at line 85
As you can see the line number changes
But which line and which xml file does it refer to?
I tried to remove some products from two suppliers and the job is successful. Could it be that there are incompatible characters in this data?
As for the update, I had serious problems with both virtuemart and csvi. If I do this, the site no longer works.
I'll try again tomorrow and hope it doesn't give me any problems
Greetings
Hello,
The issue you run into and with the changing line numbers is because of the ICEcat data. They are XML files and something is wrong with one of the XMLs coming from ICEcat.
What I want to ask is for you to go to RO CSVI -> Templates and click on the Options button in the top right corner. You are now at the global settings of RO CSVI. You will see the number of log lines set to 20. Please set this to 12000 and save the changes.
After that run the import and post the debug log here. This will be a big file so you may need to zip this. We should be able to see then which product is failing and we can see what we can do about that.
Let us know how it goes.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Good morning
This morning I see the cron job email informing me of tonight's error.
Template FIABER
XML error: > required at line 84
I am attaching the log file with all the lines, set to 16,000
but registration stopped at 9.126
Greetings
Dear
Today I upgraded to version 8.6.1.
Unfortunately the work ends with the usual error.
Is there any news for the solution?
Greetings
Dear Tharuna and Roland
Dear
Today I saw that the problem is the ICECAT xml file call in the latest log files.
But I can't see the difference between a valid xml file and the one that generates the error. They are the same!!!
Today the error is this:
Processing started...
Template FIABER
XML error: Not well-formed (invalid token) at line 1
I am attaching the last log file
Greetings
RO CSVI 8.6.1 |
Hello,
Thank you for posting the debug log. From previous debug log I see the import breaks at line 76 with ICECat URL https://data.icecat.biz/export/level4/IT/37649374.xml and with latest one at line 640 with ICEcat URL** https://data.icecat.biz/export/level4/IT/33047845.xml**. I am not able to access both these URL with my free ICECat account. See attached screenshot. It is possible that these XML files are the problem. When you open these URLs in browser, what do you see?
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
I am attaching the two xml files you requested.
Yesterday I excluded one vendor's data that seemed to be the most annoying and redid the cron job.
This morning the error reported is this.
Processing started...
Template FIABER
XML error: Not well-formed (invalid token) at line 1
I am attaching the log of this cron, it stops at the 74338243.xml file of which I am attaching the file. Line 1371
I am also sending you the 26525658.xml file which has been processed without problems (it is the one from line 1370)
I hope it gets resolved soon. I have the site with products not updated for a week.
Greetings
Hello,
I hope it gets resolved soon. I have the site with products not updated for a week.
I hope so too but it is a difficult issue to debug because it works fine for the first 1300 something products. So this makes me think it is something with the server running into a limit somewhere but what that we need to find out.
I have prepared a patch file for you. Please load the patch file and we can check the debug log after that. Once the patch is loaded, run the import and post the debug log please. Hopefully it will tell us some more.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Hello,
Did you get the same error again running it the last time?
The log shows nothing strange, just that it stopped after it closed the connection with ICEcat. Can you please upload the new patch file. In this one I also log the data that comes from ICEcat, hopefully this is the last and we can see what the response is from ICEcat.
I am thinking that you may be hitting a limit at ICEcat but let's see what the log tells us.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Roland
Yes, the error is always the same:
Processing started...
Template FIABER
XML error: Not well-formed (invalid token) at line 1
I am attaching the latest log with the latest patch
With icecat I have a full subscription and I don't think there are any limits.
Is the interruption completely random, perhaps due to network interruptions or protection systems?
Greetings
Hello,
Thank you for that log and I have found the problem. When it stops we do not get a valid XML from ICEcat but some text, which cannot be parsed as XML because it is JSON, so it fails with the error you see. This is what we receive then:
(
[Code] => 404
[Error] => Not Found
[Message] => Page not found.
)
Attached is the third and I hope final patch file. Could you please load the patch file and then run the import?
Let us know how it goes.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Roland
After numerous attempts, I won't write you the various reasons, in the end I only had to select the data of one supplier, about 6000 lines. This data has no photo and description, and needs to take photo and description from ICECAT data.
Now there is no longer the error and the upgrade is complete but looking at the products the description is missing, elaborated in the table and there is no photo. So the patch doesn't work properly.
I enclose the last log unfortunately too big even zipped so I reduced the size by cutting a few thousand lines. I left last hundred lines and the first two.
Greetings
Hello,
Thank you for the debug log. When we import from ICECat we check and read data from nodes for images and product description. Product description is read from LONGDESC node and image from PIC node. From your debug log I see these nodes have case sensitive names like LongDesc and Pic. This could be the reason description and images did not import. I have modified code to include these nodes too. Please load the attached patch file and run the import.
Let me know how that goes.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Unfortunately there is still a problem. Meanwhile, now the description and photos are loaded well.
But the cron job stopped due to this error sent via email.
Processing started...
Template FIABER
PHP Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398
Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398 XML error: Not well-formed (invalid token ) at line 1
I am attaching the log file
Greetings
Hello,
Can you please try with the attached modified file and check the import again? Let me know if cron finishes with this patch file.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Taruna
Nothing to do, the cron stops working.
This is the reported error
Processing started...
Template FIABER
PHP Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398
Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398 PHP Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398
Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398 PHP Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398
Deprecated: strstr(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 398 Failed to connect to inishop.com port 443: Connection timed out
I am attaching the log file
Greetings
Hello,
There is an error message at the end of debug log posted which is causing time out error. See the error below from debug log. This is something related to your server configuration so you need to check with hosting provider. From what I know Max Dynamic Response Body Size limit should be changed on your server. Please check this with your provider.
"The size of dynamic response body is over the limit, response is truncated by web server. The limit is set by the 'Max Dynamic Response Body Size' in tuning section of server configuration."
Also for the warnings, please load the attached modified patch file and check if those warnings are gone.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
After numerous attempts due to interruptions from remote sites, today the cron job ended with about 15,000 lines.
Now there is no longer the error and the upgrade is complete but looking at the products the description is missing and there is no photo. I have verified that the description and photos are missing in the latest products loaded. I checked the log file and the call to icecat finished without problems. I think the current setting has no effect with virtuemart and therefore the pach is not working properly.
Greetings
Hello,
Description and image was not loaded for all products? Can you post the latest debug log to check? Also provide one example product_sku for which description and picture were not loaded. Will check what has gone wrong.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
I am attaching a single line log file excerpt of a product which has no ICECAT photo and description. You can see the product published here: https://www.fatelettronica.it/access-point-wifi/zyxel-nwa110ax-nebulaflex-access-point-wi-fi-6-dual-band-poe.html
I have verified that the template settings have not changed.
Hello,
Thank you for the debug log. For product description we look for nodes LONGDESC or LongDesc in the XML response from ICEcat. From the debug log I see there is no such nodes in the XML received and so product description is not updated.
As of image please load the patch file and run the import. I have added log entries to see why image has not been imported. Send me the latest debug log to check.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Done a weekend of rehearsals and there's something strange.
From before I had a hunch that there was a difference between uploading from cron and starting from local. So I prepared an upload file of only 30 products. the test was positive, description and photos uploaded in both tests. To make sure of the result, I have always eliminated the first 5 products. But that's not the case with tonight's cron job. Always the same 5 products that have been uploaded as new, they don't have a description (the table doesn't appear) but the photo is present. It seems very strange to me.
I'll copy the response from the email at the end of the cron job.
"Processing started...
Template FIABER
PHP Deprecated: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 399
Deprecated: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 399 libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: cHRM chunk does not match sRGB Processed 16059 lines Total Result Status Image Incorrect: 2493 Image Information: 85 Manufacturer translations Updated: 15964 Medias Added: 9 Medias Updated: 13624 Product category relations Added: 82 Product category relations Updated: 15882 Product manufacturer relations Added: 15964 Product media relations Added: 13633 Product prices Added: 54 Product prices Updated: 15910 Product translations Updated: 15964 Products Information: 15964 Products Updated: 15910 Products Skipped: 59 Products Added: 54 Products Error: 72 Import has finished"
I would have liked to send you an attachment of a part of the log containing the 5 reference products but not all the lines are recorded in the log file. Of about 16,000 products, only 8,822 are registered in the file and there are no 5 reference products. Strange I have set to log a very high number of lines, see screenshot.
Hello,
If the products are imported then there should be an entry in debug log. Please load the attached patch file and send us the latest debug log along with details of 5 reference products. We can check what is going on with the reference products from debug log.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
dear Tharuna
This morning after the cron of 16,000 products I checked a reference product and as I told you yesterday it has no description but with photos. See screenshots X722_D & X722_F.
Then I replaced the file of 16,000 products with the 5 references. started the cron and the result is the one of the screenshots X722_D_5 AND X722_F_5.
The nightly cron log file stops at 8000 lines.
the log file of the 5 lines in the download has become a zipped file! not text.
I think all these patches have contributed to making it worse. Maybe it's better to restart from version 8.6.1.
Thank you
Hello,
The nightly cron log file stops at 8000 lines.
Please post the import debug log with this 8000 lines and also the import file you used. We will see if we can find a cause why debug log stopped at this line.
the log file of the 5 lines in the download has become a zipped file! not text.
Thank you for import debug log. Log file shows 4 out of 5 products have got description and image. The 5th product did not have a matching ICEcat ID so description and image were not updated.
I think all these patches have contributed to making it worse. Maybe it's better to restart from version 8.6.1.
Except for the patch file Roland sent which has fix for XML error all other patch files were for debug entries to see how image and description were processed. I don't think that is going to harm the import.
Post the requested files and we can see if we can find the cause why import stopped at 8000 products.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
The log file with 8500 lines is very large and even at maximum compression it is 16256 KB. I entered the info to access my site.
I attach the file from which I import the data to upload to virtuemart.
Greetings
Hello,
Thank you for the details. I had a look at the log files, the last import in log page shows import did not finish and log file has 1158 entries. Was this import cancelled or cron did not finish?
I then looked at the previous log files for FIABER import template and there are two from yesterday, one with 5 records and another with 16088 records. Checking the debug log of 16088 records shows 8558 records with an error in the end. See the error below. Import has finished with 16088 records but debug log is incomplete. I looked for the product 7ZT7A00545 from your screenshots sent yesterday which did not get description with cron import but that entry is missing in debug log. Is there a product which did not get description or image so I can check in the debug log of 8558 entries ?
The size of dynamic response body is over the limit, response is truncated by web server. The limit is set by the 'Max Dynamic Response Body Size' in tuning section of server configuration.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
dear tharuna
I have completed an upgrade and have selected an item for testing which is included in the log file. The item has this MPN: HP3DXPLABK
I am attaching an extract of the log file with only the line of the selected product and the icecat xml file.
Strangely when I selected some log files to delete, except the last one, it deleted everything even the unselected one.
Thank you
dear tharuna
Now I'm tired, I don't understand, after this morning's upgrade, I find the HP3DXPLABK reference product description. It wasn't there yesterday. Since one week the nightly cron has stopped, I have to restart the cron in the morning.
I go to see the four reference products not included in the log and these have no description. If I upgrade 4 products it works. Why ?
Maybe it's better to go back to the old version?
Maybe I need to limit the number of lines of the file?
Do I have to make three or more rows?
But I say, because one fine day (icecat and others have not made any changes) it no longer works and the error appears:
XML error: Mismatched tag at line XX ???
After a patch, the job is done but without the important parts of the icecat info (it's useless). Why should I pay for a FULL subscription with icecat if I can't use it!!
Maybe it's a server resource issue?
Now I open a ticket with the service provider.
Thank you
Dear Tharuna
Yesterday I divided the FIABER file into three parts. Now I have activated three cronjobs so we can have the complete log file where you will find info on the 4 products used in the test. Remember, if you work 4 lines the description is there.
Now the 4 products not an ICECAT description.
These 4 products are in the FIABER2 file
I hope it can be fixed now.
I enclose the 4 test products
Thank you
Hello,
I have completed an upgrade and have selected an item for testing which is included in the log file. The item has this MPN: HP3DXPLABK
I am attaching an extract of the log file with only the line of the selected product and the icecat xml file.
From the debug log I see that description is not updated to product. But when I tested import with only this product then I see description updated.
Now I'm tired, I don't understand, after this morning's upgrade, I find the HP3DXPLABK reference product description. It wasn't there yesterday. Since one week the nightly cron has stopped, I have to restart the cron in the morning.
That is really weird. Descriptions are updated with random products. Does the cron stop with any error? I am thinking if your server is reaching its limits and so import stops.
Maybe it's better to go back to the old version?
We did not do any changes in ICEcat code to say new RO CSVI version breaks the import. I don't think it is RO CSVI version is the problem here.
Maybe I need to limit the number of lines of the file?
Do I have to make three or more rows?
Yes this is a good solution to try. You can break import file into smaller files and save them all in one folder and run the import. RO CSVI can run import of multiple files from a folder. Please check Import multiple files from folder document for details.
But I say, because one fine day (icecat and others have not made any changes) it no longer works and the error appears:
XML error: Mismatched tag at line XX ???
We neither did any changes to ICEcat code in RO CSVI recently. The XML error is because of the way ICEcat is sending the response XML so I doubt if there are no changes made in ICEcat.
After a patch, the job is done but without the important parts of the icecat info (it's useless). Why should I pay for a FULL subscription with icecat if I can't use it!!
I totally understand this.
Maybe it's a server resource issue?
Now I open a ticket with the service provider.
This is quite possible. Your server limits are stopping the import. We have seen error in debug log about "Max Dynamic Response Body Size" limit. Did you also send the details of this error to service provider? Let us know on what you hear from them.
Now I have activated three cronjobs
You don't need three cron jobs. You can have one cron job with multiple files in folder. Please check the document linked above.
Now the 4 products not an ICECAT description.
These 4 products are in the FIABER2 file
I hope it can be fixed now.
I had a look at the debug log for FIABER2.txt file on your site and for the 4 products you sent in file. I see there is no product description (LONGDESC or LongDesc value) in the XML response received from ICEcat for first 3 products and the 4th product 471454-EDS there are no details about this product in the debug log. If you run import only with these 4 products then you see product description updated?
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Today we take stock of the situation.
Three crons are better so we have three complete logs and if one is interrupted it is faster to redo another.
Saturday (18/03) morning I check if the 4 products have the description and the result is that all of them are missing. Saturday night new cronjob and Sunday morning (19/03) I check the 4 products again and I see that now there is the description. But what is it? is it random?
So you have to check the log (FIABER2) of Saturday and Sunday.
While the product HP3DXPLABK (see FIABER3) now has no description when it used to have one and in the log file (FIABER3) there is the presence of ICECAT data.
Greetings
Lorenzo
Hello Lorenzo,
Saturday (18/03) morning I check if the 4 products have the description and the result is that all of them are missing. Saturday night new cronjob and Sunday morning (19/03) I check the 4 products again and I see that now there is the description. But what is it? is it random?
If there is problem then product description should not be updated at all but that is not the case. Sometimes it works and sometimes it does not. We need to find out why that happens.
So you have to check the log (FIABER2) of Saturday and Sunday.
While the product HP3DXPLABK (see FIABER3) now has no description when it used to have one and in the log file (FIABER3) there is the presence of ICECAT data.
Yes I see there is product description in ICEcat XML but it is not updated to database. I have added more log entries to RO CSVI files and loaded patch file on your server. Can you please send us the latest debug log once import is done? We can see if we can find any clue from the log entries.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
dear Tharuna
Yesterday (23/03) I checked the same products and they didn't have the ICECAT description. Today I see the description of ICECAT present.
One thing I did yesterday afternoon is upgrade the ICECAT indices.
I see that after inserting the patch the table is not like I made it. I need to put my lines back in the product.php file in the import template. But that's another issue and I think I'll open another discussion.
Can I now edit the product.php file or wait?
Thank you
Hello,
I see that after inserting the patch the table is not like I made it. I need to put my lines back in the product.php file in the import template. But that's another issue and I think I'll open another discussion.
That must be because the patch file I uploaded which had product.php file. I had extra log entries in the file to catch the issue with description.
Can I now edit the product.php file or wait?
Yes you can edit the file now.
If there is still problem with ICEcat descriptions then we need the latest debug log file to check.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Hello Lorenzo,
May I know if your issues with ICECat import are solved? We did not hear from you so wanted to check the status.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
I have been engaged in a new project and last time I saw that the 4 sample products contained in Fiaber2 are complete with description. But now rechecking a sample product contained in Fiaber3 does not have the description while in the log file the data of icecat are present. this is the product code: HP3DXPLABK
I have a doubt
Could it be that it doesn't overwrite the old data in virtuemart?
Since you have access you can check if I have configured the templates correctly.
Thank you
Hello Lorenzo,
But now rechecking a sample product contained in Fiaber3 does not have the description while in the log file the data of icecat are present. this is the product code: HP3DXPLABK
I had a look at debug log of FIABER3 which ran today and it shows product description has been updated. Yesterday's debug log for FIABER3 template shows empty debug log as 0 records were processed. Import on which date did not have the description?
Could it be that it doesn't overwrite the old data in virtuemart?
Since you have access you can check if I have configured the templates correctly.
Yes I had a look at it. Your template settings are correct. The setting for overwriting data is on File tab and is set to Yes so that is not the problem.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear tharuna
Yes it was empty because the TMP folder was empty.
However I had also checked on Sunday and the icecat description was not there. See log Fiaber3 dated 02/04/2023.
Greetings
Hello Lorenzo,
Yes I had a look at Fiaber3 debug log for 02/04/2023 and see product description been added. Please see attached screenshot from debug log.
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Yes, I have seen but these data are not in the product sheet. No they are copied to the DB.
Why?
See product:
https://www.fatelettronica.it/pla-pha/bobina-pla-1kg-x-comp-hp3dxplabk.html
Tanks
Lorenzo
Hello Lorenzo,
The product link you posted shows product has a description as it is updated in import debug log. See attached screenshot. Not sure if it is with latest import but there is description or I am missing something?
Kind regards,
Tharuna
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
I told you that this morning and in recent days there was no icecat description.
Today the cronjob crashed and I had to restart it at 12:01 local time and it finished at 13:24. Soon after you saw that now there is the description.
You see it's a strange thing, almost random.
It seems that CSVI doesn't see well if there is information from icecat and it doesn't skip the description of my csv file. But it does it randomly and not always
Thanks
Lorenzo
Hello Lorenzo,
So I am going to look into this as well. First thing I would like to ask is, are there any products without a description since the last import? What I want to do is check the logfile and see if I can see and understand why this might have failed for a particular product.
Thank you.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear Tharuna
Yesterday I checked the latest added products and everything was fine. Yesterday evening I reloaded the icecat data. This morning I find the latest Fiaber2 products with photos but no description. If you check the log you see that icecat description data is there but it didn't transfer it to the DB. See product with MPN : 90IG06K0-MO3R10
A copy of the info email at the end of the cronjob may be useful. attached.
Hello Lorenzo,
Thank you for the info.
So the product 90IG06K0-MO3R10 has the URL https://www.fatelettronica.it/router-wifi/asus-rog-strix-gs-ax3000-router-dual-band-wi-fi-ax-4-porte-gigabit-usb-3-2-aimesh-2402-574-mbps.html
This product does have a description, albeit short but there is one. This is the description that comes from your import file I believe. So when you say that there is no description I guess you mean not the description from ICEcat.
The product XML for 90IG06K0-MO3R10 has no PROD_LONGDESC field in it's XML, that is why the description from your import file is used and not the one from ICEcat.
So to clarify what is happening here is that some XML sheets do have the PROD_LONGDESC so the description is updated with the ICEcat data, the XML sheets that do not have it, are not updated with ICEcat data but with what is in the CSV file.
Let me know what you think.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
dear Roland
"So when you say there is no description, I guess you don't mean the description of ICEcat."
Yes, that's right, the description of ICECAT is missing.
I checked log file #1. 281 ( Fiaber2 of 04/13/2023 finished at 06:25) and I point out that the PROD_LONGDESC field is present only in the first lines (products) up to line 363. From 364 to the end it is no longer present.
all fields after the "Connection closed" field are missing. Immediately after, from line 364 the "Empty Product desc" field is missing and all the others up to the "Use field value for field product_sku" field.
What happen ?
Is something preventing you from running the "Empty Product desc" command?
As I wrote in the previous post, this thing is also random. Yesterday it broke down at line 363 but it's not repetitive.
I checked log file no. 270 (Fiaber2 of 04/12/2023 finished at 06:20) and there the PROD_LONGDESC field is present in all log lines, 4985 out of 6048, because the log was interrupted.
I also see a strange thing on line 4983 right after the PROD_LONGDESC field.
You have to understand why this thing happens if we will never get to the solution.
Gratings
Lorenzo
Yesterday I upgraded version 8.6.1 to 8.7.0.
It seems to me that we are back to the old way of handling ICECAT data. Indeed in the cronjob Fiaber3 stopped with the error:
Template FIABER3
XML error: > required at line 78.
Fiaber1 also aborted with error:
Template FIABER1
error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Only Fiaber2 concluded the job.
The PROD_LONGDESC field does not exist in the Fiaber2 log file.
Now I have manually started the Fiaber1 and 3 templetes. I hope it ends well.
Thank you
Lorenzo
Hello Lorenzo,
You have to understand why this thing happens if we will never get to the solution.
That is what I am doing and I think there are several different issues mixed up here, so I am going to try to keep them separate.
First is the PROD_LONGDESC field, In the one product I checked in my previous post, that field is not in the XML, so no ICEcat description will be imported.
all fields after the "Connection closed" field are missing. Immediately after, from line 364 the "Empty Product desc" field is missing and all the others up to the "Use field value for field product_sku" field.
What happen ?
What happened here is that only a partial XML response has been received. If you look at line 59498 in the log file, this ends in the middle of a sentence.
langid=\"5\" Value=\"Composizione del pro"
This is why this particular product is not imported. The message Closed connection is just a simple log message, not an error. The product on line 363 is the last one to receive an XML from ICEcat that has the PROD_LONGDESC.
Is something preventing you from running the "Empty Product desc" command?
Yes, because the description does not come back from the ICEcat data.
I checked log file no. 270 (Fiaber2 of 04/12/2023 finished at 06:20) and there the PROD_LONGDESC field is present in all log lines, 4985 out of 6048, because the log was interrupted.
That is actually not correct. In logfile 270 I checked the 80171412.xml on line 60572 (product 351) and the XML retrieved has an empty ProductDescription, so there is no description from ICEcat. The log file is interrupted I think because it is getting too big.
I also ran the import with one XML that did not read the description but of course in my test it did read the description.
Since you have updated and I have made some more changes to the logging and processing, I am attaching a new patch file. Please load this patch file and we can see what the new log will tell us.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear
Now I see two problems:
- The FIABER3 cronjob fails after three attempts. An email arrives from the server, different from the others, where there are lines of php instead of the error report or something else. I put it in the attachment together with the txt file of the data to be loaded. I don't understand why only FIABER three fails like the others.
- with the new patch the log files contain an excessive number of lines which also exceed the number of buffering bytes and therefore cannot be complete.
Best regards
Lorenzo
Hello Lorenzo,
This is not actually the RO CSVI logging but rather a long list of notices because the manufacturer product number cannot be found. I have optimized the import to not process ICEcat if there is no MPN.
As for the excessive logging, this was a bit added to see why the description is not being loaded. Now I have removed that and gone back to the old logging.
Please load the attached patch file for that. Let's see if that tells us more.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Dear
After adding your patch I was busy.
For commercial reasons I have removed the publication of some products (FIABER2 is empty) and added a new commercial category. For this reason I added a FIABER4 model where I set not to upload photos (file_URL) since they are provided to me. Compared to the other models it has only this difference.
If you go to see these products, there is no description either in my csv file or in ICECAT. Of course I checked in the log file where the icecat data is.
I'm writing you two MPNs to use for verification:
HDD-3000GB
HDD-6000GB
If instead the ICECAT data are not present, the description of my CSV file is inserted in the product. See these two MPNs:
VIKING-R2-8TB-V2
VIKING-VMS-CLIENT-I5-V3
In summary, new products have no description if ICECAT has one.
Maybe this helps more to understand the problem.
Greetings
Lorenzo
Hello Lorenzo,
After adding your patch I was busy.
No problem, take your time.
So I checked the FIABER4 logfile.
HDD-3000GB
This ICEcat data has no description and if your CSV file has no description, in that case it will be empty.
HDD-6000GB
Same as the 3000GB one, this ICEcat data has no description either.
In summary, new products have no description if ICECAT has one.
That is not correct because these 2 reference products have no description in their ICEcat XML data. Are you seeing the description somewhere? RO CSVI uses the field LongDesc and that is not in the XML data.
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Yes, in fact they don't have an ICECAT description. In my import file instead there is a description but it is not written in the database. see product. https://www.fatelettronica.it/hdd-server-video/ernitec-hdd-3000gb-3tb-hdd-sata-24-7.html
Greetings
Now, after the last cronjob, I see that the description of ICECAT is in the reported product HDD-3000GB. It took at least 3 cron jobs to finish getting ICECAT data into the product.
See you in the log files
Greetings
Hello,
In my import file instead there is a description but it is not written in the database.
I do not have the import file so I cannot see. Can you post the file you have used? In this case it would be the import file used on April 21st.
It took at least 3 cron jobs to finish getting ICECAT data into the product.
That is not an issue of the cronjob but rather what data comes back from ICEcat.
So I have been running some more tests but cannot reproduce what I see in your logfile. What I need is the actual CSV file used for a given import. Is the import file always the same or is this automatically generated?
Kind regards,
RolandD
=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
Log in to reply to this topic.