Resolved Load icecat indexes Column count doesn't match value count at row 1

Hello,

If we get the column count, this means the line is even longer than the new length I set. To prevent such issues in the future I will set it to 0. This makes the import a little slower but we won't have a problem with very long lines.

Hello,

The column count error means there is something going wrong with the insert of the data. I am going to check that and get back to you.

As for the Full loadtype, the MySQL function LOAD DATA LOCAL INFILE must be enabled. On most hosting platforms this is disabled by default.

Hello,

I had a look and noticed that ICEcat added 2 new columns and this is causing the error you see. Attached is a patch file that you can load. This patch contains the fixes regarding the ICEcat change.

After you loaded the patch, please take the following steps:

1. Go to CSVI via Extensions -> CSVI Pro
2. Click on About in the left menu
3. Click on Verify Database in the toolbar (this takes a moment to complete)
4. Go to Maintenance
5. Now you can load the ICEcat indexes

This should complete without a problem now.

Please let me know how it goes.

i installed the patch
verified database
but in maintenance i have now another error: Column count doesn't match value count at row 24

when i do single and 20 lines then 20 lines are imported but following error:
Column count doesn't match value count at row 4

Hello,

Ok, this is a surprise. I ran the import with 3 lines and single mode and it runs and runs and after 5 batches I cancelled it assuming all is fine. Running with your setting of 20 lines I can see the issue as well.

Now I also know why. There is 1 line in that CSV of 100,000 lines that is super duper long compared to the rest. Almost 3000 characters long, we only read the first 2048 characters. So that caused an invalid entry. Anyhow, all fixed.

Please follow the same steps as before and you should be OK.

Let me know how it goes.

Dear Roland,

First i was glad it was running, but after a while got again error:
Column count doesn't match value count at row 971

when i do single 500 lines i got: Column count doesn't match value count at row 471

So i get a patch later?
Thanks Roland

Hello,
Can you try to load the attached patch file and try again?

patch_icecat2_20180417.zip

The load is running longer but still: Column count doesn't match value count at row 971

Hello,

With the latest patch I do not have the issue and I also checked the line 471 which is not really long in my case. So now we need to figure out why it fails for you. How many lines per import have you set?

When I set 20, it works just fine and also if I set it to 970.

If you set the Lines per import to 20, does it stop earlier?

Can you send me the ICEcat file you are trying to import? Perhaps best is to use a service like WeTransfer due to the size of that file.

Roland, i think you send me wrong patch
When i unzip patch_icecat2_20180417
i get files: patch_icecat2_20180416/administrator and ../plugins

See the date of file

Hello,

Can you update to CSVI Pro 7.6.1? This contains the latest changes I made. It does not include the new length of 0 but I still wonder how we get different results. ICEcat should deliver us the same file, assuming you also use the free version of ICEcat.

The other thing you can check is if there are any lines in the ICEcat file that are longer than 4096 characters.

That date in the path, is only something we type. What is more important is that the value on line 2199 of the maintenance.php file. That is correct in the latest patch.

I tried 970 lines. It was running long, but after a while again an error:
Column count doesn't match value count at row 821

I sent com_csvi.log.56.zip com_csvi.log.56.zip you my logfile

Roland, i am now testng with 20 lines. Its already running over 2 hours and still running. I let you know the final result

Hello,

Ok, sounds good. With 20 lines it will take a while because it is a huge file. While that is running, you could inform with your hosting provider if you can use the LOAD DATA INFILE LOCAL option, even if it is temporarily because that will make things go so much faster.

Roland, it now worked
It took al long time 20 lines but after that a unknown error.
My hoster does not change LOAD DATA INFILE LOCAL, because its a shared server and they tell its insecure

Hello,

Using 20 lines is not what you want with a file this size so you could try to increase that some more. That is why the default is set to 1000. The reason for the unknown error could be that it is simply a long running process, having that on a shared server is often not so liked by hosters.

My hoster does not change LOAD DATA INFILE LOCAL, because its a shared server and they tell its insecure

That they don't enable it on a shared server I can understand. Saying that is insecure I disagree I guess they do allow FTP? That is insecure as well as it sends everything in plain text

To recap, importing the ICEcat index file on a shared server is a daunting task because it is a very large file and requires quite some resources. The alternative might be to load the file locally and then import it to the database via the command line but that requires expertise.

And my serverlog at moment icecat maintenance stopped.
Strange why images go to root? http://