Resolved Problem importing data from ICECAT

Hello,

you can change Import based on field to product_sku on Options tab in import template.

I realised the import based on field is what used for getting the product from ICECat so if product_sku field is set in import based on field then that value is searched against ean_upc field to get ICECat product. So please ignore my previous reply.

So if I load a product with another SKU but with the same gtin it does not let me download the ICECAT data in the new one but writes them in the old unpublished product.

That is correct. It finds the first matching record in database.

I deleted the old product from the DB and reloaded the new one now it has image and description. I tried this with some products.

May I ask why not delete unpublished products? We have a document explaining on Auto deleting discontinued products. Read through the document and see if that helps.

On some products it is not possible to download data from ICECAT and I cannot, at the moment, find a cause. Obviously the product is present with product_ID in the DB.

Can you post the import debug log with example product to check ICECat data? To get the debug log see How to collect debug information? document.

Hello,

Some new products did not have photos and description, which I had already reported some time ago. This is because in the DB there is already an unpublished product with a different SKU but with the same GTIN.

Now that ean_upc field is added in code with patch file you can change Import based on field to product_sku on Options tab in import template. That will use product_sku value to fetch existing record from database. That should solve your issue. Please try and let me know.

Dear
After installing the patch, the work is fine, now it is possible to download the ICECAT data with reference to EAN.
In the tests I noticed a strange thing and it is this.
Some new products did not have photos and description, which I had already reported some time ago. This is because in the DB there is already an unpublished product with a different SKU but with the same GTIN.
I deleted the old product from the DB and reloaded the new one now it has image and description. I tried this with some products.
So if I load a product with another SKU but with the same gtin it does not let me download the ICECAT data in the new one but writes them in the old unpublished product.
Unfortunately I use a system that, among various suppliers, chooses the product with the lowest price, but it is a long thing to explain.
Then there is also this problem: On some products it is not possible to download data from ICECAT and I cannot, at the moment, find a cause. Obviously the product is present with product_ID in the DB.

Hello Lorenzo,
Thank you for the debug log. We have made changes to ICECat code to use ean_upc value in search query. Please load the attached patch file and run the import. Let us know on how it goes.

I attach the entire log file

Hello Lorenzo,
There is no log file attached in your reply. Please post the import debug log so we can check.

Dear Tharuna
As I wrote you before, using product-gtin does not work at all.
Maybe the query used to find product_it is not correct.
I tested with this product from EZVIZ ean: 6941545607450.
I used the query from the ICEcat text export template provided by Roland.
I changed the line where 'prod_id' was and changed the 'ean-upc' field

SELECT product_id FROM `#__csvi_icecat_index` AS `i`
LEFT JOIN `#__csvi_icecat_suppliers` AS `s` ON ​​`s`.`supplier_id` = `i`.`supplier_id`
WHERE `i`.`ean_upc` = '6941545607450' AND `s`.`supplier_name` = 'Ezviz'

My model is set to search for product_gtin
the search response in my DB is positive product_id: 108686652

My model is set to search for product_gtin
the lines of the log file query shows me that it does not search for ean_upc
As you can see below the search is with EAN but the query is made to find prod_id.

Found ICEcat mpn reference: 6941545607450
2024-09-26 16:01:58 284 [DEBUG] Found ICEcat manufacturer name: EZVIZ
2024-09-26 16:01:58 284 [DEBUG] Find the ICEcat ID for manufacturer EZVIZ and part number 6941545607450
2024-09-26 16:01:58 284 [QUERY] SELECT product_id FROM `jos_csvi_icecat_index` AS `i` LEFT JOIN `jos_csvi_icecat_suppliers` AS `s` ON `s`.`supplier_id` = `i`.`supplier_id` WHERE `i`.`prod_id` = '6941545607450' AND `s`.`supplier_name` = 'EZVIZ'
2024-09-26 16:01:58 284 [DEBUG] Find the ICEcat ID by similar SKU for manufacturer EZVIZ and part number 6941545607450
2024-09-26 16:01:58 284 [DEBUG] Find the ICEcat ID as alternative ID for manufacturer EZVIZ and part number 6941545607450
2024-09-26 16:01:58 284 [QUERY] SELECT product_id FROM `jos_csvi_icecat_index` AS `i` LEFT JOIN `jos_csvi_icecat_suppliers` AS `s` ON `s`.`supplier_id` = `i`.`supplier_id` WHERE `i`.`m_prod_id` = '6941545607450' AND `s`.`supplier_name` = 'EZVIZ'
2024-09-26 16:01:58 284 [DEBUG] Received ICEcat data: []

It seems that the search fields prod_id and m-Prod_id are not correct

I attach the entire log file

Best regards

Hello Lorenzo,

Yes I have already tried this solution but it doesn't work.

Would like to see what does not work. Please post the latest import debug log file and import file to check.

Roland wrote that the search for the product_id in the database only happens with product_mpn, if I understood correctly.

I don't think he said it only works with product_mpn field. ICECat query uses two fields to find product_id, the field set in Import based on field on Options tab in import template and other is supplier name. Are you sure supplier name is right for the product_gtin you use for finding product_id?

If you post the import debug log I can find what happens in using product_gtin field.

Dear tharuna
Yes I have already tried this solution but it doesn't work.
Roland wrote that the search for the product_id in the database only happens with product_mpn, if I understood correctly.
From the test model provided by roland and from my other tests it doesn't work.

Thanks

Lorenzo

Hello,

So I write the index file from Icecat to my PC, I rename the file and insert it into the /tmp/com_csvi/ folder and start the upload of the icecat data. Once the work is finished I do the test with the model, it is fine. I start my test model with two products and check the log file and it is fine. I start my model and check the products of the "Itek" brand and they have photos and descriptions, it is fine

Yes, saving the file to local PC and then uploading to /tmp/com_csvi/ folder is the way to solve if you have problem in downloading file index file from ICECat.

Glad to know your issue is solved.

Can I ask you to add the product_gtin field in the product_Id search as an alternative to the product_mpn

In VirtueMart Product Import template on Options tab set Import based on field to product_gtin and run the import. That should solve the issue.

Dear Roland

I had some problems that I will now describe to you.
I cleaned the temporary files folder from RO CSVI by deleting the old.index file.
I proceed with the upload of the icecat index files. Once the work is finished I see that the index file has a smaller size than usual. I repeat the operation and the work ends with a different size but not the expected one. This is a problem

So I write the index file from Icecat to my PC, I rename the file and insert it into the /tmp/com_csvi/ folder and start the upload of the icecat data. Once the work is finished I do the test with the model, it is fine. I start my test model with two products and check the log file and it is fine. I start my model and check the products of the "Itek" brand and they have photos and descriptions, it is fine

So it could have been a problem of corrupted data from the passage from the icecat_index file to the database server?
Now it seems to be going well and in the next few days I will keep an eye on the work.

I add another problem in this post
I see that I have many products without photos and descriptions because they have the MPN not written correctly (the data is provided by a supplier) or it does not exist. Can I ask you to add the product_gtin field in the product_Id search as an alternative to the product_mpn

can we continue the discussion in this place or should I open another one?

Thanks

Hello,

You are correct, this means that entry is not in your database. Yes, you are correct that it is in the index.csv file. So I think there must have been a hiccup importing that file. What you can do is reload the ICEcat index file again and then see if the entry is there.

Let me know how it goes.

I did the test and the result is this:
"No data was found for the selections made"

So it does not find the product_Id, right?
But if it is present in the ICECAT.index.csv file!!!
Can it be solved?

Thanks

Hello,

After you ran the update list of available fields, please add the product_id field to your template and then run the export. This is a bit of a catch-22. RO CSVI does not restore the field because it does not exist but to be able to exist you need that template

Once that field is added the export should run.

Let me know how it goes.

Dear Roland
I did as you instructed and the result of your model is an error.
"Unfortunately, there are no fields set for this model. Now I don't know what to export. Go to models->model fields and add the fields you want to export."
I attach the log file and screenshot

I did the test with the ITGCACOLVB code and this gives the correct result.

Thanks

Hello,

Instead of running the query I have something better I think. I created an export template that outputs the product ID of the product you are trying to import if it exists. Can you restore the attached template via the Maintenance menu -> RO CSVI -> Restore template. After that run the Update available fields from the available fields page. Once that is done, you should be able to run the template and we can see what the result is.

Let me know how it goes.

"What I wonder is, if you run the SQL query, in PhpMyAdmin for example, for the second product as stated above, do you get any result?"

I don't know how to run a query from PhpMyAdmin.
I've never done it. If you write me some instructions I can try.

Thanks

Hello,

I definitely saw the righe_index.txt file, as I still have it open on my desktop As I mentioned before, the log shows that RO CSVI cannot find a reference to your second product, that is why no data is available. RO CSVI does not even try to contact ICEcat because it has no URL.

For the first product it immediately finds a reference so it can call ICEcat. That is why your log only shows this:

Find the ICEcat ID for manufacturer ITEK and part number ITGCACOLVB

while for the second product it tries 3 times to find a corresponding URL but finds none.

The first product is found using this query:

SELECT product_id FROM `jos_csvi_icecat_index` AS `i` LEFT JOIN `jos_csvi_icecat_suppliers` AS `s` ON `s`.`supplier_id` = `i`.`supplier_id` WHERE `i`.`prod_id` = 'ITGCACOLVB' AND `s`.`supplier_name` = 'ITEK'

RO CSVI is trying to find the second product using this query

SELECT product_id FROM `jos_csvi_icecat_index` AS `i` LEFT JOIN `jos_csvi_icecat_suppliers` AS `s` ON `s`.`supplier_id` = `i`.`supplier_id` WHERE `i`.`prod_id` = 'ITGCADCDSB' AND `s`.`supplier_name` = 'ITEK'

You said:

No I think this field is important

RO CSVI does not use this field.

What I wonder is, if you run the SQL query, in PhpMyAdmin for example, for the second product as stated above, do you get any result?

Dear Roland

I think you didn't see the file I sent you "righe_index.txt"
Inside the file there are two lines copied from the file "file.index.csv"
The two lines correspond to the products used for the test.
In the file there is the product ITGCACOLVB manufacturer ITEK (Line 1) and this has no problems downloading the data from icecat.
Then there is the product ITGCADCDSB manufacturer ITEK (Line 2) this is the one that has problems.
As you will see both have the supplier_id = 5237 = ITEK.

There is a field, on_market (field to the right of the EAN/UTC code field), which I don't know what it's for, I see that there is a 1 in the ok product while it is 0 in the product with the problem. No I think this field is important

Thanks

Hello,

Thank you for the provided information. So I had a look at the debug log you provided and decided to create my own import file.

So I created a super simple import file with just these lines and set the template to use file for configuration:

product_sku,manufacturer_name,product_name

ITGCADCDSB,ITEK,Test

That loads the ICEcat data as expected.
Now I had a further look at the log and we can see these lines:

• Find the ICEcat ID for manufacturer ITEK and part number ITGCADCDSB
  
• Find the ICEcat ID by similar SKU for manufacturer ITEK and part number ITGCADCDSB
  
• Find the ICEcat ID as alternative ID for manufacturer ITEK and part number ITGCADCDSB
  

This means that RO CSVI cannot find the ICEcat URL. This is why you get no data back. One thing we need to check is if your supplier file has a supplier name ITEK and what is the supplier_id? In my case and in the file you provided this ID is 5237. Now I think that your supplier list has a different ID for ITEK, can you check if that is the case?

That is the only reason I can think of why it would not load the data or that the ITEK supplier does not exist.

Let me know what you find.