Categories
View Replies (5)

Resolved CSV File manipulation before transfert

  1. ekcsvi
    ekcsvi
  2. RO CSVI
  3. Friday, 03 January 2020
  4.  Subscribe via email
Hi Roland

I have an important question about CSV file manipulation.
The original product file from our supplier is very big and I need to be able to import/update the products from it into VirtueMart very often.

The idea is to generate 3 little files in order to have specific import CSVI scripts for each one and make it far more quicker.

Lets say we have 2 big CSV files : Now.csv and Before.csv

I would like to be able to generate automatically :

File 1- DELETED PRODUCTS = products which where in Before.csv but no more in Now.csv
File 2- NEW PRODUCTS = products which are in Now.csv but not in Before.csv
File 3- UPDATED PRODUCTS = the rest of the products with any change between Before.csv and Now.csv

Please let me know how I could achieve this project, if you have any idea, if you know anything existing...

Thank you in advance !
Eric
0
Accepted Answer
tharuna
tharuna
Accepted Answer Pending Moderation
Hello Eric,
Click the Advanced button on top right corner as mentioned in my previous reply to see the option.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # Permalink
Responses (5)
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Eric,

The idea is to generate 3 little files in order to have specific import CSVI scripts for each one and make it far more quicker.
This is a good approach I have used before to import a couple hundred thousand of records. One file will have the basic product information, another file will have the image information. The images are imported separately because they can take a lot more time/resources. So you can check what is needed in your case. Turning off or on database table indexes can speed things up as well.

As for your file 1, this is possible with CSVI by unpublishing all records before import and then publish the ones you import. The unpublishing before import is an option in the template. You can also setup the Auto-delete of discontinued products.

As for new products you can create an import template to skip existing products, so only new ones are created. The opposite can be done for existing records. Yes, the other records are still processed but since they are skipped this is fast.

If you need a script to generate those files, that will be custom work because no code is going to know which type of product you have in your file.

One thing you may need to look at is your database settings, as this can be tuned as well to handle large imports. At some point CSVI will be pushing so hard, the data is queued at the database.

When you say very big, how many records are we talking about?
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 1
ekcsvi
ekcsvi
Accepted Answer Pending Moderation
Hi Roland

Thank you very much for your answer : this will help me !

this is possible with CSVI by unpublishing all records before import

Please let me know where/how I can do this : I don't find this option :(

I will start with this today and let you know my progresses ;)

Eric
  1. more than a month ago
  2. RO CSVI
  3. # 2
tharuna
tharuna
Accepted Answer Pending Moderation
Hello Eric,
In your VirtueMart Product import template on Options tab set Unpublish before import field to Yes and save the template. If you do not see this field click on Advanced button on top right corner.
Kind regards,

Tharuna

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 3
ekcsvi
ekcsvi
Accepted Answer Pending Moderation
Hi

Can't find the option
Unpublish before


Please see screen capture herewith attached...

Please advise
Eric
Attachments (1)
Unpublish-Before.png Unpublish-Before.png
  1. more than a month ago
  2. RO CSVI
  3. # 4
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!