Resolved image import

image import works fine now.

skype support was awesome. The field mapping issue is now fixed. It needed another patch to work. But everything's fine now.

the missing patch was here

http://www.csvimproved.com/forum?view=topic&catid=24&id=31895&start=6#41122

just did it again. do you see me now ?

Not sure if you added me already but I don't see a request.

ok done, you should see my request on skype. thanks.

I checked the log and all looks normal there. Add me on Skype and we can check further, the name is csvimproved

The other thing I tried is to change the Operation type. Do you even get the operation type? There seems to be no mention of it.

Yes the settings are K2, import, item import with the same CSV file as usual. If I try other settings like operation = category import instead of item import or if I change component, I still get an empty list.

With the original version or the first K2 image patch everything works as expected.

Another thing you can do is enable the Joomla! debug system option, if there is any database error it should show up there. You can do that via Site -> Global configuration -> System -> Debug System change to Yes and save the configuration. If you now create the map, if there is any database error, it will show up.

I turned the debug on and here is the debug log I get.

csvidebuglog.txt

If you feel you could get to solution faster with a more interactive testing session we could use skype or teamviewer. I can PM you the details.

I can confirm, I did it again like that.

I followed your process but still see the CSVI fields being populated. The only time I managed to have them empty is when I selected CSVI as component.

The other thing I tried is to change the Operation type. Do you even get the operation type? There seems to be no mention of it.

Another thing you can do is enable the Joomla! debug system option, if there is any database error it should show up there. You can do that via Site -> Global configuration -> System -> Debug System change to Yes and save the configuration. If you now create the map, if there is any database error, it will show up.

I can confirm, I did it again like that.

Uninstall CSVI
Reinstall V 5.19
Update available fields
Create test field mapping
set name, CSV, K2, import, Save
=> I see the list of fields (wrong order)

Load K2 patch (patch_k2_20141028)
Reset
Create test field mapping
set name, CSV, K2, import, Save
=> I see the list of fields (wrong order)

Load map order patch (patch_map_20141030)
Reset
Create test field mapping
set name, CSV, K2, import, Save
=> the list of available fields is empty

Just to be sure, you did upload the CSV file?

of course, I create a new field mapping (I erased all previous ones)
I set the name, the sample CSV, adjust settings to K2, import.
When I get the red message I press save and previously at this stage the list of fields is displayed.

The only change is to list the fields in the order they are saved.

That's frustrating. When I open the analysis tab the fields are displayed as explected. But no more in the mapping tab.

Should I restart from 0. Uninstall everything and reinstall patch1 then patch 2 ?

When I press save I see no fields in the list.

Just to be sure, you did upload the CSV file?

Something must be broken in the Filed mapping process

The only change is to list the fields in the order they are saved.

I loaded the patch, did reset, rescanned the available fields.
I verified the list of available fields for K2. That looks OK.

I checked my CSV file with the analysis tab. Looks OK as well.

I deleted my field mapper and tried to recreate a brand new one.
Parameters K2, import, items.
When I press save I see no fields in the list.

I double checked in the 'Available field' tab. If I select K2/item I see a list of available field.
Something must be broken in the Filed mapping process

Hello,

I have tried a number of things but I can't reproduce it. You state that the order is inverse but that is not the case either if I look at the file and the screenshot, the fields are all mixed up. This did give me an idea that the database might be returning the records in random order in your case.

Load this patch file and check if the fields are now in order as it forces a sorting. Once the fields are in the correct order, you can create a new template.

Let me know how it goes.

I'm using the same file since I started these tests.
This is the one catalog_export_20141027_094933_2014-10-29.csv

I used the same file to generate the field mapping and the template.
I use also the same file to test the import.

Now I understand that something goes wrong with the mapping process.
the order is the inverse of the CSV file order.
I did it several times to check and I can see that in my file, the field 'status' is the last field and in the mapping it's the first.

Therefore all the data is mixed up.

Can you reproduce the issue ?

If I look at the file you posted here and the screenshot of the fieldmap above, they don't match. The order of the fields in the file is different than what you have in the fieldmap.

If you have changed the file, you must upload it again in the fieldmap to redo the mapping with the new order.

The Big image field is 2nd in the file but 2nd to last in the screenshot.

I just check in my sample file (only 25 rows to import).
There is an image file in each row.

So I guess the issue is elsewhere.

I'm having an issue with the field mapping.
I turned off the error reporting and now the 25 rows are imported but

The image name is used for category_name
the item name is assigned a skipped field (producer name)
...

see here




I double checked my mapping and I see no error. Here is the mapping I'm using :

Do you have any rows where the image field is empty?

In any case, this is an issue with image processing, any kind of notice/warning/error will stop the import. What you can try is to go to the Global configuration and on the Server tab set Error repoting to None. So that all these messages are hidden and thus won't stop the import.

Let's stop here, this is where it goes wrong. Replacement rules can be used by both the import and the export. In your case we are talking about the import. So now that you have the replacement rule, you go to the template and on the Fields tab, click on the image field to open the settings of this field. There you see the replacement rules also, select here the rule you created and click on Save. Now the replacement rule is used for this field. That should create the full image name in your case.

Got it, now I have an error during import saying :

Cancel the import and check if all settings are correct and try the import again. Status error: 200 Status message: OK Warning: fopen("http://www.myserver.com/images/): failed to open stream: HTTP request failed! HTTP/1.1 403 Forbidden in /.../librairies/joomla/filesystem/file.php on line 327

It looks like the import is trying to open the root url and not the full url with the image name.
If with a browser I go to http://myserver.com/images/ I get the 'Forbidden' response.
If I go to http://myserver.com/images/myimage.jpg the image is displayed.

My replacement rules for the image field are:

Find : /^/
Replace with : http://www.myserver.com/images/
Multiple values : No
Method : Regular Expression

In the field mapping, I can't add or remove fields.

Correct, the field mapper is only a tool to help you create a template from a file you want to import. In the template itself (Process page) you manage the fields after you created the template. There you can add/remove fields as you like on the Fields tab.

The 7 steps you outlined for running your import is correct.

I understand that this replacement template is used together with an export process to create a new CSV from the K2 content

Let's stop here, this is where it goes wrong. Replacement rules can be used by both the import and the export. In your case we are talking about the import. So now that you have the replacement rule, you go to the template and on the Fields tab, click on the image field to open the settings of this field. There you see the replacement rules also, select here the rule you created and click on Save. Now the replacement rule is used for this field. That should create the full image name in your case.

So you don't need to do any export. Hope this explains it better.

If you don't want a field exported, just don't add it to the list.

In the field mapping, I can't add or remove fields. They are automatically generated from the source file. When importing data I can turn off the import of the fields I can't map to K2 by setting the parameter to 'skip'. In export I don't have such an option.

You will have to explain this further, as I don't understand what you are trying to do.

OK I will try to explain in detail what I did and what I understood :

My goal is to import a CSV file which can be refreshed quite often to K2.
The file contain a list of column with the various fields I want to import to K2 items.

1. I need to map the column names to K2 fields. That works with the map tab function.
2. I need to automatically create the missing categories. That works with the new delimiter option you provided with the patch
3. I need to import an image into the main item image. That fails because my CSV file contains only the name of the image and not the full URL to use to download the image. I would have liked an option in the image tab to specify the root URL for the images but it's not there and you suggested to use the replacement feature to add the root url to the image name.

That's what I'm trying to do and here is how I understood I should proceed :

For importing a CSV file :

1. I have a CSV file with multiple fields that I want to import to K2.
2. The field mapping tab allows me to create a map template.
3. To do so I create a K2/import mapping template, choose the CSV file to list all the available column/fields and associate some to K2 fields, skip the others.
4. From that mapping template, I generate an import template and name it, say K2 import
5. In the template tab, I load my K2 import mapping template, adjust a few parameters and save it back.
6. I the choose my CSV file to import and press proceed.
7. I double check in K2. This process works. but the images are not loaded yet.

From there, I need to use the new image import feature for K2.
My CSV contains the image name but not the url of the original server, from which I need to get the image to store in K2. You suggested to use the replacement option to replace the name of the image, say myimage.jpg with the full url, say http://myserver.com/images/myimage.jpg
For that I made the following :

1. From the replacement tab I create a new replacement template and edit it.
2. I set the parameters to add the root url before the image name as described in the doc. save the replacement template.
3. I understand that this replacement template is used together with an export process to create a new CSV from the K2 content
4. the generated CSV shall then be used to be download and store the images in K2 but ..

The generated CSV does not contain the original name anymore but the name exported from K2 which is already scambled.

In this scenario, how can I achieve my import task ?

You can't either because the original name is not stored anywhere. K2 filenames are a hash of the word Image and the ID of the K2 item. K2 generates this on the fly to find the images on the server. There is no database storage of the main images.

OK but then how to use the replacement option to alter the CSV file before it's imported.
My process seems not accurate and I don't see what step to follow.