Resolved import shows blank screen and fails to run

Hello John,
The debug log did not have entries because of the error. So testing import with your template and import file, import says file cannot be read. I found few issues with template

1. On file tab set Use file extension to CSV instead of XLS. The source file link is getting CSV file and not XLS
2. On Source tab, click on Advanced button to see Encode URL setting, change it to No, save the template.

Run the import and check.

Hello John,

'0 Call to a member function getName() on null'.

This error can occur when the uploaded import file exceeded the max post size defined in PHP settings of your server. In your import template settings on Limit tab reduce the number of lines of import to 10, save the template and run the import. Check if the import works.

Hi Tharuna,

Thank you for your quick response. Based on your answer I have a couple of questions.

First, when I click the Advanced button from the Source tab it only toggles visibility of the Delete After Import field. I do not see an Encode URL option. I am using the latest version of CSVI. Where is the Encode URL option?

Second, my source file has an XLS extension and when I download it from the site I linked to above and then open it with LibreOffice Calc (Spreadsheet) it opens perfectly without the need for any of the import setup (delimiter definition, etc) as it would with a CSV file. Even stranger is the fact that on my original site with the same XLS and the exact same template, the file is processed without a problem using the XLS option in CSVI. Most importantly, why would I choose CSV as the file type when it I am trying to import a file that is clearly an XLS file? This seems confusing and counterintuitive. I did notice that by choosing the 'Take from file' option in the Use file extension field I no longer see the blank screen but receive a message saying that my source file is '...not a readable file'.

I included the template again in this email because I am concerned that perhaps I sent the incorrect template the first time. (I am very confused by the CSV/XLS issue.)

Thanks for your help.

Kind regards,

John

Hello John,
Ok i see there are two URLs in the template, i used the api one with Load from URL option and that worked with the solutions i gave in my previous reply.

I included the template again in this email because I am concerned that perhaps I sent the incorrect template the first time. (I am very confused by the CSV/XLS issue

The confusion was because of the file included. The file you linked in Source file text in your previous reply is different from the one set in template. The file downloaded from Source file is with name regulation8_consolidated_2.xls and the file given in the template is /home/rjeuvxmy/public_html/codebase/AUScsl_daily.xls. May i know which is the right file to check?

Hi Tharuna,

My system is setup with a large number of cron jobs that download different files from various sources to my server. The cron jobs rename the downloaded files when they are saved to my server. So for example in this case my cron job downloads a file called regulation8_consolidated_2.xls and saves it on my server as AUScsl_daily.xls.

You do not have access to my server where I have renamed the file but I sent you the link to access the file from the source. You can download the test the file from the source (regulation8_consolidated_2.xls) and use it for the import by changing the name of the source file in my template or you can download the file and rename it to match the name that my template uses.

My apologies for the confusion.

Thanks,

John

Hello John,

My system is setup with a large number of cron jobs that download different files from various sources to my server. The cron jobs rename the downloaded files when they are saved to my server. So for example in this case my cron job downloads a file called regulation8_consolidated_2.xls and saves it on my server as AUScsl_daily.xls.

Ok it is clear now.

You do not have access to my server where I have renamed the file but I sent you the link to access the file from the source. You can download the test the file from the source (regulation8_consolidated_2.xls) and use it for the import by changing the name of the source file in my template or you can download the file and rename it to match the name that my template uses.

I uploaded the file using Load from computer option and it did work. The difference i see in the template you attached in your previous post is that you do not have Use file for extension to XLS and Skip empty custom tables to no. Skip empty custom fields is not related to this import but for imports which has custom fields so you can leave it to default value No. I have attached the settings i have on File tab of the template and the values i see on Import preview page. Please note that i only have few fields created for custom table to test the import. See if import works for you with these settings else send me a screenshot on what you see in import preview.

Hi Tharuna,

I have done further testing of this issue but I am still unable to import the file.

Here is what I have just tried:

On my original website where I don't have any trouble with CSVI I changed the source of this import to my computer. When I run the import, I browse to the file on my computer and select Preview. The file opens and the preview works as expected.

On my new website I did the same thing and changed the source of this import to my computer. When I run the import, I browse to the file on my computer and select Preview. I get a blank screen and nothing happens.

In my test, both sites used the same source file from the same location. One works perfectly, the other fails. So far, all other imports that I have on my new website work fine so I don't think it is a corrupt or broken installation of CSVI. We also now know it is not a bad or corrupt copy of the source file as I ran both tests using the same file.

Next, on my new server, I created a new Import routine in CSVI using the source set to my computer. I left everything set to the default and did not add any fields and I left it set to autodetect delimiters and use file for configuration. I ran the Import and selected the same file from my local computer that was used in the tests above. The result was a blank screen. Now we know that the specific CSVI import routine is not the problem as the same error occurs with different import routines using the same file - but only on this server.

Finally, you should know that this source file caused trouble for CSVI before. Here is a link to the thread that discussed and resolved the previous issue. I wonder if this has something to do with the problem I am having now.

Kind regards,

John

Hello John,

When I run the import, I browse to the file on my computer and select Preview. I get a blank screen and nothing happens.

This generally means there is an error but error reporting is turned off. Can you go to System -> Global configuration -> server and set Error reporting to Developer. Save the changes and after that run the import again. Let's see if you see an error.

Hi Roland,

I tested the Import various times with error reporting switched to Development earlier today. The response was the same. No error, just a blank screen. I also tested a few other things that did not make a difference including reloading the XLS patch file and re-saving the XLS source file with LibreOffice and trying the import with the new file.

The only breakthrough I had was when I tried using Chrome instead of Firefox a few minutes ago and it actually produced this error - '0 Call to a member function getName() on null'. Clicking on this error message (still in Joomla environment) caused a server error 500 to appear in the browser.

Thanks for your help.

John

I am amending my last post here - I reported that I got the '0 Call to a member function getName() on null' error first and then a 500 error. Actually, the 500 error occurs first in Chrome (as opposed to a blank screen in Firefox) and attempting to reload the page produces the '0 Call to a member function getName() on null'.

John

Hi Tharuna,

I ran the import with the lines limited to 10 and got a blank screen just like every other attempt. I have other imports that work fine that are using much bigger source CSV files (more lines and more columns) than this XLS file so I don't think it is an file upload size limitation that is causing the issue. In fact, I have 50+ other imports that ran as cron jobs this morning on this server and they all worked perfectly (the difference is that all are CSV or XML files). At this point I am going to remove the the CSVI installation from my website and then reinstall it.

I will let you know what happens.

Thanks,

John

Update: Uninstalling and reinstalling the entire CSVI package made no difference. I still just get a blank screen when importing the XLS file.

John

Hi Tharuna,

Here is some more information for you. I just opened the original XLS file and copied all the columns with data and the first 1000 rows to the clipboard. I pasted them into an empty spreadsheet and saved the new file as XLS. The import worked fine. Next, I copied all the columns with data and all rows with data (a little over 6000 rows) to the clipboard and pasted them in a new, empty spreadsheet and saved the new file as an XLS. The import worked fine with this too.

So now we know there is something about the original XLS file that makes the CSVI import on this server fail. I have simply tried re-saving the original file with my copy of LibreOffice as an XLS and the error persists. Copying only columns and rows with data to the clipboard and pasting them into a new spreadsheet solves the issue. The only difference I can see is that empty rows and columns are not being introduced into the new file. I wonder if somehow the original file has some hidden bytes in extra columns and rows that is making the XLS file seem too big - but this does not explain why the original file gets imported without errors on my old server (and has been imported every day for about a year) and fails on my new server.

Thanks for your patience.

John

Hello John,

but this does not explain why the original file gets imported without errors on my old server (and has been imported every day for about a year) and fails on my new server.

Thank you for the detailed findings. What I am thinking now, is that it has something to do with the text itself. Some kind of encoding that screws things over.

Could you save the XLS file as CSV from LibreOffice and see if that imports? That is also easier to debug generally as it is plain text.

Hi Roland,

Saving the XLS file as CSV and redefining the Import routine to import the CSV file works as expected. I did not actually run the import yet but I do not get a blank screen, and instead see the Preview page which seems to be working fine. The Preview screen parses the first few records correctly and reports that the file contains 7128 lines. This is interesting because according to LibreOffice the last row with real data is 6215. The last visible record in both files is the same.

Edit: New Detail - Google Sheets shows the last record of the XLS file with valid data is on row 6215. Sheets shows a total of 6244 rows for this XLS file - the last 29 records appear to be blank/empty.

John

Hello John,

Google Sheets shows the last record of the XLS file with valid data is on row 6215. Sheets shows a total of 6244 rows for this XLS file - the last 29 records appear to be blank/empty.

The CSV file seems to have invisible records so you see 6244 rows for 6215. If you open the file with plain text program like Notepad or TextEdit, what do you see at the end?

Hi Tharuna,

I saved the XLS file as CSV using a semicolon as the field delimiter because many of the records have a comment field that often contains commas. I used a straight text editing tool to open the CSV file and see that it has a number of blank/empty records at the end. There are no characters in any of the empty rows.

Yesterday I reported that CSVI was showing 7128 lines in the Preview of the CSV file. Today I used the CSV file I created using semicolons as delimiters (instead of commas like I did yesterday) and CSVI shows a more accurate line count that matches what my text editor shows.

I am confident that this CSV file, will import into my system without any problems.

So when I convert the XLS file to CSV there is no problem for CSVI. There does not appear to be anything strange or unexpected in the converted CSV file except for a few blank/empty rows at the very end which does not seem to be an issue.

The XLS file continues to import into my original website without any trouble and did again successfully last night. The same XLS file still causes CSVI to produce a blank screen when it tries to open on my new server. All other imports continue to work as expected on my new server.

Thanks,

John

Update: I loaded the original XLS file into Google Sheets and then exported it as an XLSX file. I changed the CSVI Import routine to XLSX and it runs fine. CSVI still shows a line count that reflects the empty records at the end of the file but it seems to work without any issues.

Of course, this still does not solve the problem with the original XLS file but may help track down the issue.

John

Another update: I just opened the XLSX file that I created with LibreOffice and then saved it as an XLS. CSVI was able to open this file without any trouble.

This new XLS file has the same number of lines as the original XLS file and does not appear to be different (but I can't really compare all 6000+ plus lines by hand).

John

Hello John,
We tested your file with my dev site, Roland's dev site and our development server, there is no problem with import of the file. There is something very specific to your new site which stops the import of the file. The file itself is not a problem as it works fine with the sites we tested and also works for you on original site. You mentioned that there is a 500 error with this import. We need to find what is causing this 500 error on your new site and that should give us a lead to the error.