Resolved Server Error 503

Hello,

Result: Before the process completes, at some point in time (usually after the 1000-1500 processed items) the server gives out a 503 error.

503 error can happen when the server is overloaded. Are you trying to import images from external links?

Two things you can try

1. Reduce Lines per import to 100 or 50 and time between imports to 5 seconds. Save the template and run the import.
2. Set up import as cron job.

Hello and thanks.

The images are placed on server.

I just tried with disabled image processing, and it seems to complete the import successfully.

Could I split the process into two templates, one for all product data (without images), and the second just for the images? Is this possible? Do you think it would make a difference?

Thanks

Hello,

The images are placed on server.

Do you have any image processing set? something resizing of large images? Wonder why image import takes so long when images are on server. Please post debug log to check. To get the debug log check How to collect debug information? document.

Could I split the process into two templates, one for all product data (without images), and the second just for the images? Is this possible? Do you think it would make a difference?

Yes you can do that. You can create two import templates. In the first import template set file_url field to skip field so images are not processed and import rest of the fields. In the second import template use only product_sku and file_url field and set other fields to skip. This might make a difference considering the setup you have on image processing.

We have contacted our VPS manager, and they did some tests with CSVi.

They replied that they adjusted the nginx server, to be compatible with the Long Polling method that CSVi uses.
I don't know exactly what this means, but hopefully it has fixed the problems for us (we will be doing more extensive tests in the following days).

I am letting you know in the case you want to include info about this in your documentation, or make any code adjustments in the future (even if just warning the user that the server is not configured properly).

Hello,

Thank you for the info. I can explain what the Long Polling is. So most servers have a maximum timeout of 30 seconds, within these 30 seconds all work needs to be done. In most cases those 30 seconds is not enough to process large import files. So there are several options, one is to increase the 30 seconds to for example 300 seconds. In this case RO CSVI can send a request that can last 300 seconds instead of 30 seconds. If you do not have the option of changing the timeout, they only other option is to send multiple requests. So we send a request and before the 30 seconds run out we stop and tell the browser how much has been imported, we then send a new request for the next 30 seconds and continue this until the file is imported. Sending these multiple requests is called long polling.

Do you know which changes they made? I am interested to know.

Thanks!

We had the option to change the timeout, and we had already set it to a high value (4000 seconds).

But the server errors were coming way sooner than that. (which is what troubled us)

I have no info on what changes they made. They reported us they configured the Long Polling on the server to be compatible with CSVi.
My guess is that the server was not respecting the CSVi requests, and after some point it just quit. (that's what it looked like to us)

I could ask for more detailed information, but I don't know what to ask for. If you have specific questions I'll be happy to pass them over.

Hello,

But the server errors were coming way sooner than that. (which is what troubled us)

Aha, that is possible too. There are a lot of variables to take into account. In the past I have also seen that RO CSVI is too fast for the database

All is fine and I am happy your issue has been resolved.