Categories
View Replies (35)

Resolved image import

  1. iozo
    iozo
  2. RO CSVI
  3. Friday, 24 October 2014
  4.  Subscribe via email
Hello again,

I need to import images to K2.
The stream is coming with an image field as follows :

mc2/StillLife_10_140.gif
mc2/Syberia_10_140.gif
microapp/7816_140.gif

images are stored on a remote server with a known root url, let's say

http://media.remotesite.com/images/games

I want to grab the images :

http://media.remotesite.com/images/games/mc2/StillLife_10_140.gif
http://media.remotesite.com/images/games/mc2/Syberia_10_140.gif
http://media.remotesite.com/images/games/microapp/7816_140.gif

and store it locally in my Joomla images folder, ideally in a dedicated subfolder.
I'm expecting to see in the K2 media manager something like :

/images/games/mc2/StillLife_10_140.gif
/images/games/mc2/Syberia_10_140.gif
/images/games/microapp/7816_140.gif

I'm also expecting the appropriate K2 image field to be filled with the right image.

I described the image information in the field mapping. Here is what the analyser displays :

CSV fields Top

Big image
Product Name

CSV data Top
If you see any unreadable characters in this table, your file is not UTF-8 encoded. Make sure your file is UTF-8 encoded so special characters like ë are imported correctly.
1 2
mc2/StillLife_10_140.gif Still Life
mc2/Syberia_10_140.gif Syberia
microapp/7816_140.gif Scratches

How to setup CSVI pro for that :

1. Where to indicate the root remote URL
2. I checked the import image to yes,What else should I do to ?

thanks,
0
Responses (35)
rolandd
rolandd
Accepted Answer Pending Moderation
I don't see how to ignore some field in the export (no skip in the field combo for K2 export).
You can't. If you don't want a field exported, just don't add it to the list.

I don't know how to apply the generated export template to another source file.
You will have to explain this further, as I don't understand what you are trying to do.

Regarding the image processing, what I want is to get the exact same file with the image field modified with replacement rule.
You can't either because the original name is not stored anywhere. K2 filenames are a hash of the word Image and the ID of the K2 item. K2 generates this on the fly to find the images on the server. There is no database storage of the main images.

but the image name is replaced with the K2 scrambled name instead of the original name.
That is because the original name is not stored anywhere.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 21
iozo
iozo
Accepted Answer Pending Moderation
ok, I'm doing some (little progress) now.


I create the replacement rule and another field mapping template for exporting my original CSV into a conformed version. The idea is to add the root URL to each image.
In my mapping file I added all fields I want to export in the new file.
In the field options of the template I added the replacement rule to the image field.

However, I'm not exactly getting the result I'm expecting.

1. I don't see how to ignore some field in the export (no skip in the field combo for K2 export).
2. I don't know how to apply the generated export template to another source file. Currently the export already knows the source file (from the mapping I guess) and don't allow me to select another file in the template tab like for import.

Regarding the image processing, what I want is to get the exact same file with the image field modified with replacement rule.As a result, the exported file has the root url happened before the image name as expected, but the image name is replaced with the K2 scrambled name instead of the original name.

ie :

http://www.mydefaultserver.com/images/products/2fa67f482133f1c934235b73c2a03954.jpg
instead of
http://www.mydefaultserver.com/images/products/myimage.jpg
  1. more than a month ago
  2. RO CSVI
  3. # 22
rolandd
rolandd
Accepted Answer Pending Moderation
I think I forgot one step when I generated the map file. I'm checking now.
Definitely looks like that because the first field in the attached file is short description and not category path :)

Is it mandatory to indicate 'Skip' on the field you don't need.
Yes it is mandatory, otherwise CSVI will process the field.

Can you have a fast way to set multiple field as skipped ones. It's painful to set them one by one.
None that I known of other than pressing the letter S which will go straight to the first entry starting with the letter S. Press it again it goes to the second, etc.

If I have multiple step to conform a csv file and proceed with the import, do you have some kind of chaining mechanism to automate the process.
Not sure what you mean here however CSVI only does a single import run and no other tasks around it.

I will use the cron task for that but I'm not there yet.
A bash script is the way to go to have some management on import routines and related tasks.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 23
iozo
iozo
Accepted Answer Pending Moderation
Post your CSV file as well so I can see where the error is. There must be a field order misconfiguration. The debug log shows the very first field in your import file is the category path, which it most likely isn't. ;)


I think I forgot one step when I generated the map file. I'm checking now.
Is it mandatory to indicate 'Skip' on the field you don't need.
Can you have a fast way to set multiple field as skipped ones. It's painful to set them one by one.


Regarding image import where should I indicated the root url for the image.You should have that in the import file ... Why is that?


because The file is coming from a third party server. I don't master the format of the data.

What you can do is use a replacement rule to prepend the server name


That is a nice trick.

If I have multiple step to conform a csv file and proceed with the import, do you have some kind of chaining mechanism to automate the process. I will have ultimately to run it several times a week to add/remove/update the content on my sever.

I will use the cron task for that but I'm not there yet.
Attachments (1)
catalog_export_20141027_094933.csv
  1. more than a month ago
  2. RO CSVI
  3. # 24
rolandd
rolandd
Accepted Answer Pending Moderation
I get the short description in the category although I verified my mapping and everything looks OK.
Post your CSV file as well so I can see where the error is. There must be a field order misconfiguration. The debug log shows the very first field in your import file is the category path, which it most likely isn't. ;)

Regarding image import where should I indicated the root url for the image.
You should have that in the import file.

I can't have the image data in the CSV with an absolute path.
Why is that?

I'm expecting the import processor to concatenate a root url with the image name and process the image.
The import processor reads the data from your file, that is why it expects the data in the file.

What you can do is use a replacement rule to prepend the server name. See the Replacements tutorial. It has an example of how to prepend a value.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 25
iozo
iozo
Accepted Answer Pending Moderation
I tried the patch with a K2 item import.
I have items imported but the fields are not repecting the mapping I made.

I get the short description in the category although I verified my mapping and everything looks OK.
k2importitem.jpg


here is the debug log :
com_csvi.log.1414573884.zip


Regarding image import where should I indicated the root url for the image. I'm expecting some configuration in the image tab to indicate something like http://mymasterserver.com/images

If in my CSV file I have an image named myimage.jpg, I'm expecting the import processor to concatenate a root url with the image name and process the image.
In my sample it would download the image http://mymasterserver.com/images/myimage.jpg, do the magic to drop it in K2 media directory and register myimage.jpg in the item image field.

I can't have the image data in the CSV with an absolute path.

thank you
Attachments (2)
k2importitem.jpg k2importitem.jpg
com_csvi.log.1414573884.zip
  1. more than a month ago
  2. RO CSVI
  3. # 26
rolandd
rolandd
Accepted Answer Pending Moderation
The patch has been posted in this thread.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 27
iozo
iozo
Accepted Answer Pending Moderation
I don't think you get to decide that if the K2 team has decided it for you :)

That's right, but K2 articles can contain data coming from other components or plugins, like galleries or video players and these ones can have other requirements for the storage, but you are right, it's not an issue at the moment.

When I'll have the image import for K2 work, this will be already a big step.
  1. more than a month ago
  2. RO CSVI
  3. # 28
rolandd
rolandd
Accepted Answer Pending Moderation
I assume that the image import processor for K2 that you'll provide will automate the first step, i.e download the image from the provided URL in /media/k2/items/src/ and the file name out of the url to save the item in K2.
Correct.

Can it be used for categories as well ?
Once I complete that work, yes.

What about other media file import (galerie, video, sound file, ..)
I have not looked into that yet as there has not been any requests so far.

As media mapping is very component specific, do you see a way to provide a more generic media import processor.
This is decided upon per component. Each component handles media in it's own way. The VirtueMart implementation has no fixed path so you can use what you like but only because VirtueMart allows for it. K2 is more rigid and expects the media in one folder and a hashed filename.

Ideally it would also allow us to decide if the data must be imported or stay remote if the component allows it. For instance a Youtube video vs a local K2 item image.
I don't think you get to decide that if the K2 team has decided it for you :)
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 29
iozo
iozo
Accepted Answer Pending Moderation
OK I got it, thx.
I assume that the image import processor for K2 that you'll provide will automate the first step, i.e download the image from the provided URL in /media/k2/items/src/ and the file name out of the url to save the item in K2.

Can it be used for categories as well ?

What about other media file import (galerie, video, sound file, ..)
As media mapping is very component specific, do you see a way to provide a more generic media import processor. I would see a form where we indicate a CSV file with remote url / component field mapping (ie http://www.myremoteurl/mydata/myfile.ext -> K2/item/images or
http://www.myremoteurl/mydata/mysoundfile.ext -> Joomla/media/music)

Ideally it would also allow us to decide if the data must be imported or stay remote if the component allows it. For instance a Youtube video vs a local K2 item image.
  1. more than a month ago
  2. RO CSVI
  3. # 30
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

But one step at a time, importing the main image is my first target.
I am all for such approach :)

What is the process there ? If we consider that I have my media stored on a temp folder somewhere on the server, where do I indicate the temp source folder for the images ?
What should contain the image field to refer to the right image ?
You upload the images with their real name to the /media/k2/items/src/ folder. In your import file you only put the name of the file, for example myimage.jpg. During the import CSVI will update the images accordingly for K2.

If you can have the remote import processor for K2, that's just great.
I will implement it and post the patch when it's done.

When could you have a version available ?
This should be ready in the next couple days.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 31
iozo
iozo
Accepted Answer Pending Moderation
Hi Roland,

Is it me or this media manager is really slow? Anyway, it won't work store the images in the game folder for example, when you select an image from the game folder and store the item, K2 hashes the filename and places it in the media/k2/items/src folder.


Now I get your point (finally). The thing is that I will need to import images for the product description but also additional media files for slideshows or video ... some will use the K2 storage, other will use component dependent storage. I will need to import remote data and have the import processor drop it in the right place and adjust the fields accordingly.

But one step at a time, importing the main image is my first target.

For processing local images on the server that were put there via FTP for example.


What is the process there ? If we consider that I have my media stored on a temp folder somewhere on the server, where do I indicate the temp source folder for the images ?
What should contain the image field to refer to the right image ?

Sounds good?


If you can have the remote import processor for K2, that's just great.

When could you have a version available ? I'm ready to help for testing or debugging.
I don't want to put any kind of pressure, but I'm on a project which needs this import stuff.

many thanks,
  1. more than a month ago
  2. RO CSVI
  3. # 32
rolandd
rolandd
Accepted Answer Pending Moderation
This is my K2 media window. Ideally I'd like to specify somewhere a root url to download the images from and store it in the game folder of the K2 media space.
Is it me or this media manager is really slow? Anyway, it won't work store the images in the game folder for example, when you select an image from the game folder and store the item, K2 hashes the filename and places it in the media/k2/items/src folder.

That is real bad news as I purchased CSVI for its capability to import images. Is that feature only available for virtuemart ?
CSVI does import images, for K2 it doesn't do remote images.

If so, what is the meaning of the image import tab ?
For processing local images on the server that were put there via FTP for example.

If it's only for VM do you considere adding the feature to K2, Joomla or as a separate import process.
Not only VM but also EZ Realty.

In my case I absolutely need to get access to the image. I see a fe options. Which one would you recommend ?
Option 1 is true, since there is an image import for K2 just not remote images as you would like. Option 3,4,5 are not an option in my opinion. We stick with option 2, I will build the support for remote images and post a patch file, so you can have remote images import.

Sounds good?
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 33
iozo
iozo
Accepted Answer Pending Moderation
Actually I don't mind where it stores it as far as it is accessible by K2.

k2mm.jpg

This is my K2 media window. Ideally I'd like to specify somewhere a root url to download the images from and store it in the game folder of the K2 media space.

CSVI cannot retrieve remote images for K2.

That is real bad news as I purchased CSVI for its capability to import images. Is that feature only available for virtuemart ?

If so, what is the meaning of the image import tab ?
It looks generic whatever the import template we choose.
If it's only for VM do you considere adding the feature to K2, Joomla or as a separate import process.

In my case I absolutely need to get access to the image. I see a fe options. Which one would you recommend ?

1. I'm wrong and you have an image import processor for Joomla/K2 ..
2. It's only VM but you could have a beta version of the import processor for K2 to share.
3. I should consider using VM. Seriously it's possible although not my preferred path.
4. I should consider using extra field in K2 and use the remote image in my views
5. You have a better idea ..

please advice
Attachments (1)
k2mm.jpg k2mm.jpg
  1. more than a month ago
  2. RO CSVI
  3. # 34
rolandd
rolandd
Accepted Answer Pending Moderation
Hello,

and store it locally in my Joomla images folder, ideally in a dedicated subfolder.
That won't work because K2 uses a fixed folder where the images are installed. This folder is /media/k2/items/src/.

This is also the folder where CSVI expects the image to be as it gets renamed to the md5 hashed filename that K2 uses.

The fieldname for the image field is image. So you can add that in addition to the other fields you use for the import. At a minimum it is the alias and category_path fields.

CSVI cannot retrieve remote images for K2.

Are you using a dedicated subfolder now with K2? If so, how do you make that work? As far as I know, we only have the fixed folder.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 35
  • Page :
  • 1
  • 2


There are no replies made for this post yet.
Be one of the first to reply to this post!