Resolved Import new custom field skipped

Hello,

so I applied the patch to see if it resolved my issue, but unfortunately it has not.

It wouldn't because this thread is over 2 months old and any release after this will have the updated code included, so you should actually revert the change to make sure you don't have any outdated code. You can install CSVI over your existing installation to do that.

Please can you help as there is little help that I can find available for this version at the moment.

Of course I will help. The information from CSVI 5 applies for the majority to this as well. It's just the looks that has changed but not the way things work.

Looking at your log file, the import doesn't even get to your custom fields. Revert the patch first and then run your import again please and let's see what is in that log file. Also check if you get any errors or notices.

I appemnded this to the wrong forum reply, it should have referred to #43901 from sandstorm871

Fixed.

Hello Roland,

Have you had any joy with this yet, as I am being pressed by my client for a resolution so that the products can be uploaded.

if I import only the product data it works fine, but the problem is that I need the custom field data to be imported into the #__virtuemart_product_customfields table.

If I just use the custom field entries for the products, the custom field data is imported as a string into the #__virtuemart_customs.custom_value field as a string, whereas I need the individual custom field elements to be inserted in to the product custom fields table for use in the drop down custom value lists.

It would be much appreciated if you can help with this, as I am running out of time to finish the project, and this is the final step (e.g. I have to create a csv spreadsheet that the client can use as a "one step" product import or update process.

Thank you.

Regards

John R

Ps. My apologies, I did not see your reply before I posted this, as I do not receive the email notifications for the updates (I am a contractor for DDA). I will carry out your suggestions and then get back to you, but please be aware that I had completely uninstalled the earlier versions of CSVi first, and tested with a completely clean copy when I got the issue, and I only applied the patch as I had thought that it might fix the problem, but sadly not.

Perhaps the additional description in this post will shed some further light on the issue that I am having, but in the meantime I will re-install the CSVi to restore it back to the original.

Hello John,

A reply can usually be expected within 24-48 hours unless it is the weekend, it may take a bit longer, so you can take that into account.

whereas I need the individual custom field elements to be inserted in to the product custom fields table for use in the drop down custom value lists.

What you can do is create 1 product where this is setup manually in VirtueMart, then create a product export that exports the custom_title, custom_value, custom_param fields including the SKU of course. So you can see how it looks like. That should give you a good idea of how to structure your import file.

Hi Roland,

That is exactly what I did. The CSV file I attached previously was the csv export file. I set up the import based on this file, so I had expected it to work, but sadly not. I do not need to import the "custom-price" or "custom_param" fields for my purposes, but I have included them because they were recommended (though I did also try to export/import without them and got the same error messages).

I have re-installed the CSVi Pro again as suggested to reset the component back to its default install, but I still get the same error messages.

I have attached 4 screenshots for your review, the first 2 are the error messages that I am getting, and screenshots 3 & 4 are the field lists for the import and the export field listings. I have also attached again the single product export file that I am using as my template for the importing of the products.

If I were to import the product data without the custom field data, all works fine, but then I would have to manually import the custom field data, but as we will have around 400+ products to import this is not practical.

For the custom fields, when I manually added the product listed in the spreadsheet, there was a single entry created in the "#__virtuemart_products" and an associated entry in the "#__virtuemart_products_en_gb" tables. I also added a single custom field for the product "size" which generated a single entry in the "#__virtuemart_customs" to add the custom field title and field type (e.g. "S", and the product has 6 custom field paramenters recorded in the "#__virtuemart_product_customfields" table (1 row for each of the sizes listed in the "custom_value" column in the spreadsheet.

What I am looking to achieve is for the product data to be imported into the "#__virtuemart_products" and the associated "#__virtuemart_products_en_gb" tables, and the applicable entries in the "#__virtuemart_product_customfields" table 9no need for an entry in the "#__virtuemart_customs" table, as there are no new custom fields to be added).

In addition to the other files mentioned, I have also included the log file that I generated this morning after my latest attempt to resolve this issue, but I am really at a loss as I have followed the instructions in the documentation that you referred to, but either I am not understanding something in the documentation, or the documentation needs updating to reflect the changes to the latest version (you can see from the screenshots that I am using CSVi 6.1.2).

Please can you help or point me in the right direction as I am under heavy duty preassure to get this project completed before the end of this week, as the client has been waiting for weeks now for this to be resolved.

Thank you.

Regards

John

Hello John,

So I tried to import your file here and it works as expected. The issue is not with your file but with your site setup somewhere because you see the No supported fields message, going beyond that has no use since CSVI has no fields.

Here are two suggestions, try the first one first and if it doesn't work, try the second one:

1. Go to the Templates -> Template fields page and select all the fields and click Publish
2. Go to Templates -> Templates and edit your template. On the file tab set the option "Use column headers as configuration" to Yes.

If neither of these give any results, send me a private message with Joomla/FTP login details and I will check.

Hi Roland,

I tried both of your suggestions, but I still get the same error messages.

There does not seem to be any way to check if the fields are published in thetemplate fields for the import template fields. If I look at the Export template fields, I can see a "published" column to confirm that they are published, but this column is not available for the Import template fields (see the 2 new screenshots attached)

Just one thing to note, in your 2nd suggestion you mentioned setting "On the file tab set the option "Use column headers as configuration" to Yes.", but in my version the nearest I have to this is "Use file for configuration", so I set this to Yes instead.

As I am only a contractor for DDA I will need to check with DDA before I can give you access to the site, particularly fro FTP access as the FTP access has a security lock on it, so it can only be unlocked for a small time window unless you have a fixed IP address.

If DDA give me permission, I will send you a PM as soon as I can.

Regards

John

Ps. Just a thought. On this site they also have K2 installed for Article management, but for the shop we are using VirtueMart. Would this make a difference?

Hello John,

There does not seem to be any way to check if the fields are published in thetemplate fields for the import template fields.

It is not needed because the fields should always be processed. There is a small issue in the current release if any field happens to be unpublished it won't be processed. This is fixed for the next release.

Just one thing to note, in your 2nd suggestion you mentioned setting "On the file tab set the option "Use column headers as configuration" to Yes.", but in my version the nearest I have to this is "Use file for configuration", so I set this to Yes instead.

That is the correct option.

As I am only a contractor for DDA I will need to check with DDA before I can give you access to the site

Understand, I won't have a fixed IP until I get home on Thursday evening CET.

If FTP takes longer, let's start with Joomla access in that case to see if that helps. One thing you can also check is to save the file as UTF-8. It appears it is plain ASCII and that can be trouble as well.

Hi Roland,

Thank you. Ok, I saved the file as UTF8 from Notepad, but this will be a real pain for the client to use, as he will only want to use Excel, but for now I have done this for testing purposes.

Unfortunately this has still not fixed the issue, but the file did at least attempt to process. I am still getting the "no supported fields ... etc" error message at the preview step, but the "Start import" does now at least work, but the import process is still failing with the error message "These fields have been found but are not supported by CSVI"!! Anty ideas?

I have attached a screenshot of the process completion page, and a copy of the associated log file. In the meantime I will chase up DDA for the access approval for you to log in yourself.

Regards

John

Hello John,

The file has been saved as UTF-8Y, the Y is a BOM (Byte Order Mark). These are 3 invisible characters at the start of the file that will blow everything up. So you should be able to use this file but then you need to set the option Auto-detect delimiters to No and specify the delimiters yourself. Other than that, things are starting to look fine.

but this will be a real pain for the client to use, as he will only want to use Excel, but for now I have done this for testing purposes.

That is definitely the pain of using Excel as Excel has really bad CSV support. It will not let you choose which encoding to save in. There are several ways around this, of course not using Excel but for most people that is not a choice. Second is if you don't have too much data, save the file as XLS (not XLSX) and import the XLS file. The reason I mention not too much data is because an XLS file needs to be read into memory completely so it uses a lot more memory.

In general but not a given is if you start a new Excel sheet it will be stored as UTF-8. You could try with a CSV file exported by CSVI and using the option Excel CSV. This option will add the BOM so Excel should treat it as UTF-8.

Hope that helps.

Hi Roland,

Thank you, I will give it a try.

Just for your info, the csv file that I have been using was originally generated from CSVi, and was then opened in Excel 2010, and when saved, was saved as a CSV file, so apart from the exported file, it has never been saved as an excel file of any version.

Ref the BOM, I was unaware of this as I only have excel and notepad to use for converting, and it is not visible to me as a UTF-8Y, only as a UTF-8.

Regards

John

Hello John,

The original export, was the Excel CSV option used is probably the question. As it had no BOM before I am guessing probably not.

OpenOffice/LibreOffice are much better in giving you control over the file and it's properties. How to save the file as UTF-8 with them is explained in the Save a CSV file as UTF-8 tutorial.

Hi Roland,

you are the best. Thank you, this has solved the problem, and I have successfully imported the sample product using the manual delimiiter settings along with the UTF-8 file.

I also noted that you suggested saving the file as an XLS file, and then importing the XLS file directly, but this failed, so from the XLS version of the file I saved as a CSV file and then imported it, and success, it worked.

In summary, it looks like the file needs to be initially saved into an XLS format before doing the "save as" CSV file, and then import the resulting CSV file. this method at least cuts out the need to open with notepad and save as a UTF-8 file, so at least this will please the client.

Before I go, and this is totally unrelated to the above questions, and no worries if the answer is no, but is it possible for the CSVi Pro to import images as well as the product data in the CSV files, and if so, is there any documentation on this?

thank you again for all of your help, it is very much appreciated.

Regards

John

OpenOffice/LibreOffice are much better in giving you control over the file and it's properties. How to save the file as UTF-8 with them is explained in the Save a CSV file as UTF-8 tutorial.

Thank you for this, I will have a look through it.

John

Hey John,

Yes you can. Have a look at the Import VirtueMart images tutorial. In case you have any further questions about the images, please open a new topic.

Hi Roland,

Thank you, much appreciated.

Thank you also for all your help, the support you give is much appreciated.

Best regards

John