1. fatelettronica
  2. RO CSVI
  3. Monday, 20 February 2023
  4.  Subscribe via email
Dear Tharuna

I haven't been able to finish uploading data for a week. I don't understand what the error means and I don't see anything strange from the log file. I also checked the data contained in the upload file but nothing.
Can you tell me what I can do to fix it?

Greetings
Accepted Answer Pending Moderation
Hello,
There is an error message at the end of debug log posted which is causing time out error. See the error below from debug log. This is something related to your server configuration so you need to check with hosting provider. From what I know Max Dynamic Response Body Size limit should be changed on your server. Please check this with your provider.


"The size of dynamic response body is over the limit, response is truncated by web server. The limit is set by the 'Max Dynamic Response Body Size' in tuning section of server configuration."


Also for the warnings, please load the attached modified patch file and check if those warnings are gone.
Attachments (1)
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 21
Accepted Answer Pending Moderation
Dear Tharuna
After numerous attempts due to interruptions from remote sites, today the cron job ended with about 15,000 lines.
Now there is no longer the error and the upgrade is complete but looking at the products the description is missing and there is no photo. I have verified that the description and photos are missing in the latest products loaded. I checked the log file and the call to icecat finished without problems. I think the current setting has no effect with virtuemart and therefore the pach is not working properly.

Greetings
  1. more than a month ago
  2. RO CSVI
  3. # 22
Accepted Answer Pending Moderation
Hello,
Description and image was not loaded for all products? Can you post the latest debug log to check? Also provide one example product_sku for which description and picture were not loaded. Will check what has gone wrong.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 23
Accepted Answer Pending Moderation
Dear Tharuna

I am attaching a single line log file excerpt of a product which has no ICECAT photo and description. You can see the product published here: https://www.fatelettronica.it/access-point-wifi/zyxel-nwa110ax-nebulaflex-access-point-wi-fi-6-dual-band-poe.html
I have verified that the template settings have not changed.
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 24
Accepted Answer Pending Moderation
Hello,
Thank you for the debug log. For product description we look for nodes LONGDESC or LongDesc in the XML response from ICEcat. From the debug log I see there is no such nodes in the XML received and so product description is not updated.

As of image please load the patch file and run the import. I have added log entries to see why image has not been imported. Send me the latest debug log to check.
Attachments (1)
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 25
Accepted Answer Pending Moderation
Dear Tharuna

Done a weekend of rehearsals and there's something strange.
From before I had a hunch that there was a difference between uploading from cron and starting from local. So I prepared an upload file of only 30 products. the test was positive, description and photos uploaded in both tests. To make sure of the result, I have always eliminated the first 5 products. But that's not the case with tonight's cron job. Always the same 5 products that have been uploaded as new, they don't have a description (the table doesn't appear) but the photo is present. It seems very strange to me.
I'll copy the response from the email at the end of the cron job.

"Processing started...
Template FIABER
PHP Deprecated: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 399

Deprecated: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior in /home/uwjcyoig/domains/fatelettronica.it/public_html/administrator/components/com_csvi/helper/icecat.php on line 399 libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: known incorrect sRGB profile libpng warning: iCCP: cHRM chunk does not match sRGB Processed 16059 lines Total Result Status Image Incorrect: 2493 Image Information: 85 Manufacturer translations Updated: 15964 Medias Added: 9 Medias Updated: 13624 Product category relations Added: 82 Product category relations Updated: 15882 Product manufacturer relations Added: 15964 Product media relations Added: 13633 Product prices Added: 54 Product prices Updated: 15910 Product translations Updated: 15964 Products Information: 15964 Products Updated: 15910 Products Skipped: 59 Products Added: 54 Products Error: 72 Import has finished"

I would have liked to send you an attachment of a part of the log containing the 5 reference products but not all the lines are recorded in the log file. Of about 16,000 products, only 8,822 are registered in the file and there are no 5 reference products. Strange I have set to log a very high number of lines, see screenshot.
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 26
Accepted Answer Pending Moderation
Hello,
If the products are imported then there should be an entry in debug log. Please load the attached patch file and send us the latest debug log along with details of 5 reference products. We can check what is going on with the reference products from debug log.
Attachments (1)
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 27
Accepted Answer Pending Moderation
dear Tharuna
This morning after the cron of 16,000 products I checked a reference product and as I told you yesterday it has no description but with photos. See screenshots X722_D & X722_F.
Then I replaced the file of 16,000 products with the 5 references. started the cron and the result is the one of the screenshots X722_D_5 AND X722_F_5.
The nightly cron log file stops at 8000 lines.
the log file of the 5 lines in the download has become a zipped file! not text.
I think all these patches have contributed to making it worse. Maybe it's better to restart from version 8.6.1.
Thank you
Attachments (5)
  1. more than a month ago
  2. RO CSVI
  3. # 28
Accepted Answer Pending Moderation
Hello,
The nightly cron log file stops at 8000 lines.

Please post the import debug log with this 8000 lines and also the import file you used. We will see if we can find a cause why debug log stopped at this line.

the log file of the 5 lines in the download has become a zipped file! not text.

Thank you for import debug log. Log file shows 4 out of 5 products have got description and image. The 5th product did not have a matching ICEcat ID so description and image were not updated.

I think all these patches have contributed to making it worse. Maybe it's better to restart from version 8.6.1.

Except for the patch file Roland sent which has fix for XML error all other patch files were for debug entries to see how image and description were processed. I don't think that is going to harm the import.

Post the requested files and we can see if we can find the cause why import stopped at 8000 products.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 29
Accepted Answer Pending Moderation
Dear Tharuna
The log file with 8500 lines is very large and even at maximum compression it is 16256 KB. I entered the info to access my site.
I attach the file from which I import the data to upload to virtuemart.

Greetings
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 30
Accepted Answer Pending Moderation
Hello,
Thank you for the details. I had a look at the log files, the last import in log page shows import did not finish and log file has 1158 entries. Was this import cancelled or cron did not finish?

I then looked at the previous log files for FIABER import template and there are two from yesterday, one with 5 records and another with 16088 records. Checking the debug log of 16088 records shows 8558 records with an error in the end. See the error below. Import has finished with 16088 records but debug log is incomplete. I looked for the product 7ZT7A00545 from your screenshots sent yesterday which did not get description with cron import but that entry is missing in debug log. Is there a product which did not get description or image so I can check in the debug log of 8558 entries ?


The size of dynamic response body is over the limit, response is truncated by web server. The limit is set by the 'Max Dynamic Response Body Size' in tuning section of server configuration.
Attachments (2)
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 31
Accepted Answer Pending Moderation
dear tharuna

I have completed an upgrade and have selected an item for testing which is included in the log file. The item has this MPN: HP3DXPLABK
I am attaching an extract of the log file with only the line of the selected product and the icecat xml file.
Strangely when I selected some log files to delete, except the last one, it deleted everything even the unselected one.

Thank you
Attachments (2)
  1. more than a month ago
  2. RO CSVI
  3. # 32
Accepted Answer Pending Moderation
dear tharuna

Now I'm tired, I don't understand, after this morning's upgrade, I find the HP3DXPLABK reference product description. It wasn't there yesterday. Since one week the nightly cron has stopped, I have to restart the cron in the morning.
I go to see the four reference products not included in the log and these have no description. If I upgrade 4 products it works. Why ?
Maybe it's better to go back to the old version?
Maybe I need to limit the number of lines of the file?
Do I have to make three or more rows?
But I say, because one fine day (icecat and others have not made any changes) it no longer works and the error appears:
XML error: Mismatched tag at line XX ???

After a patch, the job is done but without the important parts of the icecat info (it's useless). Why should I pay for a FULL subscription with icecat if I can't use it!!
Maybe it's a server resource issue?
Now I open a ticket with the service provider.



Thank you
  1. more than a month ago
  2. RO CSVI
  3. # 33
Accepted Answer Pending Moderation
Dear Tharuna

Yesterday I divided the FIABER file into three parts. Now I have activated three cronjobs so we can have the complete log file where you will find info on the 4 products used in the test. Remember, if you work 4 lines the description is there.
Now the 4 products not an ICECAT description.
These 4 products are in the FIABER2 file
I hope it can be fixed now.

I enclose the 4 test products

Thank you
Attachments (1)
  1. more than a month ago
  2. RO CSVI
  3. # 34
Accepted Answer Pending Moderation
Hello,
I have completed an upgrade and have selected an item for testing which is included in the log file. The item has this MPN: HP3DXPLABK
I am attaching an extract of the log file with only the line of the selected product and the icecat xml file.

From the debug log I see that description is not updated to product. But when I tested import with only this product then I see description updated.

Now I'm tired, I don't understand, after this morning's upgrade, I find the HP3DXPLABK reference product description. It wasn't there yesterday. Since one week the nightly cron has stopped, I have to restart the cron in the morning.

That is really weird. Descriptions are updated with random products. Does the cron stop with any error? I am thinking if your server is reaching its limits and so import stops.

Maybe it's better to go back to the old version?

We did not do any changes in ICEcat code to say new RO CSVI version breaks the import. I don't think it is RO CSVI version is the problem here.

Maybe I need to limit the number of lines of the file?
Do I have to make three or more rows?

Yes this is a good solution to try. You can break import file into smaller files and save them all in one folder and run the import. RO CSVI can run import of multiple files from a folder. Please check Import multiple files from folder document for details.

But I say, because one fine day (icecat and others have not made any changes) it no longer works and the error appears:
XML error: Mismatched tag at line XX ???

We neither did any changes to ICEcat code in RO CSVI recently. The XML error is because of the way ICEcat is sending the response XML so I doubt if there are no changes made in ICEcat.

After a patch, the job is done but without the important parts of the icecat info (it's useless). Why should I pay for a FULL subscription with icecat if I can't use it!!

I totally understand this.

Maybe it's a server resource issue?
Now I open a ticket with the service provider.

This is quite possible. Your server limits are stopping the import. We have seen error in debug log about "Max Dynamic Response Body Size" limit. Did you also send the details of this error to service provider? Let us know on what you hear from them.

Now I have activated three cronjobs

You don't need three cron jobs. You can have one cron job with multiple files in folder. Please check the document linked above.

Now the 4 products not an ICECAT description.
These 4 products are in the FIABER2 file
I hope it can be fixed now.

I had a look at the debug log for FIABER2.txt file on your site and for the 4 products you sent in file. I see there is no product description (LONGDESC or LongDesc value) in the XML response received from ICEcat for first 3 products and the 4th product 471454-EDS there are no details about this product in the debug log. If you run import only with these 4 products then you see product description updated?
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 35
Accepted Answer Pending Moderation
Dear Tharuna
Today we take stock of the situation.
Three crons are better so we have three complete logs and if one is interrupted it is faster to redo another.
Saturday (18/03) morning I check if the 4 products have the description and the result is that all of them are missing. Saturday night new cronjob and Sunday morning (19/03) I check the 4 products again and I see that now there is the description. But what is it? is it random?
So you have to check the log (FIABER2) of Saturday and Sunday.
While the product HP3DXPLABK (see FIABER3) now has no description when it used to have one and in the log file (FIABER3) there is the presence of ICECAT data.
Greetings

Lorenzo
  1. more than a month ago
  2. RO CSVI
  3. # 36
Accepted Answer Pending Moderation
Hello Lorenzo,
Saturday (18/03) morning I check if the 4 products have the description and the result is that all of them are missing. Saturday night new cronjob and Sunday morning (19/03) I check the 4 products again and I see that now there is the description. But what is it? is it random?

If there is problem then product description should not be updated at all but that is not the case. Sometimes it works and sometimes it does not. We need to find out why that happens.

So you have to check the log (FIABER2) of Saturday and Sunday.
While the product HP3DXPLABK (see FIABER3) now has no description when it used to have one and in the log file (FIABER3) there is the presence of ICECAT data.

Yes I see there is product description in ICEcat XML but it is not updated to database. I have added more log entries to RO CSVI files and loaded patch file on your server. Can you please send us the latest debug log once import is done? We can see if we can find any clue from the log entries.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 37
Accepted Answer Pending Moderation
dear Tharuna
Yesterday (23/03) I checked the same products and they didn't have the ICECAT description. Today I see the description of ICECAT present.
One thing I did yesterday afternoon is upgrade the ICECAT indices.
I see that after inserting the patch the table is not like I made it. I need to put my lines back in the product.php file in the import template. But that's another issue and I think I'll open another discussion.

Can I now edit the product.php file or wait?

Thank you
  1. more than a month ago
  2. RO CSVI
  3. # 38
Accepted Answer Pending Moderation
Hello,
I see that after inserting the patch the table is not like I made it. I need to put my lines back in the product.php file in the import template. But that's another issue and I think I'll open another discussion.

That must be because the patch file I uploaded which had product.php file. I had extra log entries in the file to catch the issue with description.

Can I now edit the product.php file or wait?

Yes you can edit the file now.

If there is still problem with ICEcat descriptions then we need the latest debug log file to check.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 39
Accepted Answer Pending Moderation
Hello Lorenzo,
May I know if your issues with ICECat import are solved? We did not hear from you so wanted to check the status.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 40
  • Page :
  • 1
  • 2
  • 3


There are no replies made for this post yet.
Be one of the first to reply to this post!