Categories
View Replies (17)

Resolved Export Performance Issue

  1. atrus
    atrus
  2. RO CSVI
  3. Tuesday, 21 July 2015
  4.  Subscribe via email
Hi Roland,

Have been doing some export tests and noted the following performance issue, this is the scenario:

We have the following category tree:

Parent Category (Total 1)
1st-Level Subcategories (abt 50)
2nd-Level Subcategories (abt 550)
3rd-Level Subcategories (abt 1000)

All the products are assigned to 3rd-Level Subcategories, so Parent,1st-Level,2nd-Level are empty.

If we try to export Parent Category only, using the "Include Subcategories" -> Yes option, the export takes long time, and at some point it throws an internal server error at 120 secs (server limits).

While if we export all 2nd-Level Subcategories (now, that's a pain!!!), the export completes ok in abt 63 secs, almost half time.

Don't know if this difference is justifiable, what i do know though is that we cannot maintain such exports where we have to input 500+ categories one by one :S

Would appreciate your feedback on whether there is some room for optimization in such cases, ideally we need to input only Parent Category for export.

Rgrds,
Chris
0
Responses (17)
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

The category filter is a painful and complicated filter for sure.

what i do know though is that we cannot maintain such exports where we have to input 500+ categories one by one :S
Are you sure? :silly:

Would appreciate your feedback on whether there is some room for optimization in such cases, ideally we need to input only Parent Category for export.
There is always room for optimization, perhaps we can take a closer look at this after the final release. If you can send me a backup of your site, I can test with the same dataset as you.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 1
atrus
atrus
Accepted Answer Pending Moderation
Thanks Roland,

There is always room for optimization, perhaps we can take a closer look at this after the final release. If you can send me a backup of your site, I can test with the same dataset as you.


I will send you soon the details of our backup site. For now i'll go back entering my 500 categories :p

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 2
atrus
atrus
Accepted Answer Pending Moderation
Hello Roland,

Noted also that when you change the language, the categories are deleted, is this normal?

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 3
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

When you change the language, the category list is reloaded in the selected language. This is expected behaviour indeed.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 4
atrus
atrus
Accepted Answer Pending Moderation
Hello Roland,

After some further testing i think i have i better picture of what is going on:

- If i export 2nd or 3rd level subcategories, the export works ok. If i export Parent Category or 1st-Level subcategory, CSVI exports ALL products from all categories and ignores any other settings (e.g. Only Parent/Only Child prods etc..)

This is why it was taking so long for the export of the parent category, it was also exporting all other parent categories etc.

See for example attached log from an export of a 1st-level subcategory, which has 8 nos 2nd-level subcategories, 16 nos 3rd-level subcategories, totalling 23 products. Instead CSVI exports 7400+ products :blink:

So it seems that currently "Include Subcategories" works for the bottom 2 subcategories only and not for bigger nesting, at least this is what i understand :p

Would appreciate your feedback on this.

Rgrds,
Chris
Attachments (2)
cat-exp-2.jpg cat-exp-2.jpg
com_csvi.log.638-2.zip
  1. more than a month ago
  2. RO CSVI
  3. # 5
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

Did I mention that the category filter is a painful filter? :P

I will have to go back into the code and see what is happening here exactly. With your feedback, I will have something specific to look at, thank you.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 6
atrus
atrus
Accepted Answer Pending Moderation
Hello Roland,

Well done on releasing 6.0.1! Will keep testing, let me know if you manage to troubleshoot the categories issue.

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 7
atrus
atrus
Accepted Answer Pending Moderation
Hello Roland,

Well done on releasing 6.0.1! Will keep testing, let me know if you manage to troubleshoot the categories issue.

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 8
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

Thanks :)

I am going to look at the category issue today.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 9
atrus
atrus
Accepted Answer Pending Moderation
Thanks Roland, hope you can manage smth before the weekend, we are currently stuck with our exports due to this :S

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 10
rolandd
rolandd
Accepted Answer Pending Moderation
Hey Chris,

I have it on my todo list for today, so I will get back to you later today.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 11
rolandd
rolandd
Accepted Answer Pending Moderation
Chris,

I can't seem to locate the email with the backup details, can you resend it please?
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 12
rolandd
rolandd
Accepted Answer Pending Moderation
Hello Chris,

No need, I have been able to reproduce it with a very simple setup. I have found that the Include subcategory option only takes the next level of subcategories and not the deeper ones. This I believe is the root cause of your issue.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 13
atrus
atrus
Accepted Answer Pending Moderation
Hi Roland,

Thanks for the feedback, yeap, this is indeed the root cause of the issue, so addressing that should be enough i guess :whistle:

Rgrds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 14
rolandd
rolandd
Accepted Answer Pending Moderation
Chris,

Ok, here is an attempt :unsure:

Please load this patch and it should export all products from all subcategories of the selected category.
Attachments (1)
patch_category_20150729.zip
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 15
atrus
atrus
Accepted Answer Pending Moderation
Thanks Roland, that's a good attempt B)

First tests work nicely! Will revert if i notice smth

Rgds,
Chris
  1. more than a month ago
  2. RO CSVI
  3. # 16
rolandd
rolandd
Accepted Answer Pending Moderation
Great, let me know if you run into anything else.
Kind regards,

RolandD

=========================
If you use our extensions, please post a rating and a review at the Joomla! Extension Directory
  1. more than a month ago
  2. RO CSVI
  3. # 17
  • Page :
  • 1


There are no replies made for this post yet.
Be one of the first to reply to this post!