Resolved 90857 Imported, 391 image errors

Hello Chris,

I have the log of the import, its 911MB

That is not surprising with an import of over 90k items

I feature request would be a breakout option on the summary is to "show errors only" option so I could easily trace what caused the 391 errors??

The reason I do not have this is because I need to have the steps leading up to an error, the error alone isn't very meaningful if you can't see what happens before it.

What is the search string I need to use to parse and find where the errors are?

First have a look at the regular log via the Joomla backend. There you can filter on statuses of the log. That should also give you an idea of what is wrong with the images without diving straight into the debug log.

so what am I searching for within this file so I can figure out what the issue may be?

As mentioned in the previous paragraph, do that step in the regular log first and filter. You could search for the SKU as well.

A wild guess is that you may have images that are not found.

Both show log and view log (not download) cause an out of memory condition on the server, there is no way to filter it as it won't even load.

I only have the download log at this point.

Not sure what to do next.

Fatal error: Allowed memory size of 125829120 bytes exhausted (tried to allocate 955322576 bytes) in /homepages/xxxxxxxxxxxxx/test.aaron-beachproductions/administrator/components/com_csvi/models/logs.php on line 677

Do you know how to use PhpMyAdmin?

All the time....what do you want me to look at?

I also am on skype if you think that would be faster and easier to solve this too.

You can check the log yourself in PhpMyAdmin by executing this query

SELECT `jos_csvi_logdetails`.*

FROM `jos_csvi_logdetails`

LEFT JOIN `jos_csvi_logs` AS `l` ON `l`.`csvi_log_id` = `jos_csvi_logdetails`.`csvi_log_id`

WHERE (`result` LIKE '%Success%') AND `l`.`csvi_log_id` = 260 AND `result` = 'Success'

ORDER BY `line` ASC

change jos with your own prefix and change Success with Error. Finally replace the 260 with the ID of your log, you can find that in the URL from the Logs page. That should give you the error lines.

SELECT `ABP_csvi_logdetails`.*

FROM `ABP_csvi_logdetails`

LEFT JOIN `ABP_csvi_logs` AS `l` ON `l`.`csvi_log_id` = `ABP_csvi_logdetails`.`csvi_log_id`

WHERE (`result` LIKE '%Error%') AND `l`.`csvi_log_id` = 4 AND `result` = 'Error'

ORDER BY `line` ASC

Didn't return anything (Log ID is 4), but when I sort by result, all the failures come to the top. What I'm seeing is that anything that is "special char" like Expos(e') > it flags an error.

Here is an example:

1MICHAELXBUBLÉXYOUXDONXTXKNOWXMEX93X2005X6201344X1433807629X1350987996X255-FRONT_COVER

1MICHAELXBUBL (error) XYOUXDONXTXKNOWXMEX93X2005X6201344X1433807629X1350987996X255-FRONT_COVER

Only the front of error makes it into database, loses the rest, flags an error.

This behavior was NOT a problem in 4.5.1 and I have not changed anything about the export process or file naming conventions.

There is no way I want to do this all again, in terms of batching and fixing 90K files.

All the images are up there, if there's a way to get csvi to accept the special char, that would solve the issue and only be another 2 hours to empty and redo the import process for these.

Thoughts?

SELECT `ABP_csvi_logdetails`.*

FROM `ABP_csvi_logdetails`

LEFT JOIN `ABP_csvi_logs` AS `l` ON `l`.`csvi_log_id` = `ABP_csvi_logdetails`.`csvi_log_id`

WHERE (`result` LIKE '%Failure%') AND `l`.`csvi_log_id` = 4 AND `result` = 'Failure'

ORDER BY `line` ASC

This code works as expected, was looking for Failure not Error. Hope it helps others out if looking for the same thing.

Still need thoughts on how to fix two posts above with SQL truncation after the accent char is read.

Thanks Roland.

Chris

Hello Chris,

What I'm seeing is that anything that is "special char" like Expos(e') > it flags an error.

This is related to what I would think is number 4 on the import troubleshooting list. Double check if the file is really UTF-8. I can't think of any other reason why it wouldn't work as it is always a UTF-8 issue if the word is cut off. The forum is full of these questions

Hi Roland,

I went up and down that road years ago and my export templates specifically force UTF-8 formatting. You can from below images, I've covered all these bases years ago and confirmed again to be sure. There has to be some other issue or setting that is not right. Thoughts?

Hello,

Why is your text file showing XC9 instead of É? I would expect this to be in the file

1MICHAELXBUBLÉXYOUXDONXTXKNOWXMEX93X2005X6201344X1433807629X1350987996X255-FRONT_COVER

another word but same character.

There has to be some other issue or setting that is not right. Thoughts?

Create a 1-product import file with a product that you know has issues. Post the file and debug log of that import and let me check it. I will get a better understanding of what is going on and I can test with that file.

Well your patience has been helpful, and I'm glad I did this in bits...I actually have 10 files that I did for each export and then hand combined by copy and paste to create the master_all.csv file. Well the individual files look fine, but the master version of some seems to corrupt during the paste process?? I'm using notepad++ so it is very strange. I "think" all will be fine, I'll have to do it piece meal and try it again.

There may be no issue at all, just a merge problem......damn.

There may be no issue at all, just a merge problem

Sounds like it. Let's see once the master file is all good.

I didn't bother going down that road, I re-exported from my MP3 software in 5 batches of around 20K items. They all went in 100% without error, so clearly something with the copy and paste, but I don't have the time or energy to deal with why it went south.

Time to finish my website upgrade, thanks for all the help.

Chris

Hello Chris,

Understand, at least it is all solved now