[PAID] Google Merchant Static Feed

[jobloggs]

Hi Martin,

Sorry for not being in touch earlier. Unfortunately it is not possible for me to simply give ftp access to a live site. We have many customers and credit card information stored in our db and restrictions set by IP for access ;(

We will have to do something about this debugging process as requesting ftp details to debug is not ideal. Maybe create a seperate log file during the export holding errors?


Neverthless I have summarised the current problems I have encountered below:


1)
Undeclared variables
--------------------
Notice: Undefined index: children in /home/store/public_html/includes/classes/class.froogle.php on line 1260
Warning: in_array() [function.in-array]: Wrong datatype for second argument in /home/store/public_html/includes/classes/class.froogle.php on line 1260
Notice: Undefined index: 839 in /home/store/public_html/includes/classes/class.froogle.php on line 1324
Notice: Undefined variable: row in /home/store/public_html/includes/classes/class.froogle.php on line 577
Notice: Undefined index: gmtaxonomy in /home/store/public_html/includes/classes/class.froogle.php on line 982
Fatal error: Maximum execution time of 0 seconds exceeded in /home/store/public_html/lib/module.php on line 312

its Good to declare variables and types. You should switch notices on during development.


2)
Missing Images
--------------

/modules/shoppingcomparison/googlemerchantgben/images/*

Although I understand its a quick release, have you tried mapping categories and trying to figure out which categories are mapped and which aint without an image? Please make this available to we can add to the module


3)
Check XML Structure
--------------------
It would be helpful to check the XML structure for validity after the XML feed has been generated. On some occasions where froogle.php has exceeded maximum execution time, I have re-ran it (3 times to be precise) and monitored the file size as it was generated, I could see the tmp file growing and growing, and exceeded the normal size of 23MB. Finally when it overwrites the original xml, it became 28MB. When uploaded it seemed like it had some formatting errors:


"Error while parsing/validating XML feed
Our system encountered an error when processing your XML data feed. Please see the following Help Centre topics for more information:

XML tag mismatch"

I guess this is when froogle.php has not terminated properly and has opened a tag which it failed to close. How do you deal with what has already been written and what you are waiting to write? Seems like this needs to be looked at again?


4)
Missing recommended attribute: product type (3307 warnings out of 13831)
------------------------------------------------------------------------
Maybe to create a debug.log file where you can push any errors during the feed generation or log important details needed for debugging this issue. I cannot see the relation between the items which do not have a


5)
Invalid UPC value (11 warnings)
-------------------------------
Do you truncate the UPC or just explode it? Seems like UPC = 12 characters and EAN = 8, 13 or 14 characters

http://www.google.com/support/merchants ... wer=160161

Ofcourse if you dont truncate, this is not the problem of the module.

[jobloggs]

regarding issue 3)

I think this was a problem with manual upload. Screws with the formatting when you download and upload to google. Done a direct fetch by url and no errors in formatting.

Would be nice to know the process of how it "carries on from where it left off" though.

Thanks

[Martin]

Sorry for not being in touch earlier. Unfortunately it is not possible for me to simply give ftp access to a live site. We have many customers and credit card information stored in our db and restrictions set by IP for access ;(

Unfortunately that makes it rather difficult to debug specific scenarios in a focused way...

If you could at least provide restricted access to the products/categories in the ISC admin backend and details on your feed naming and location, that would help me do some reverse engineering to see which categories might be hitting problems...

Worth noting that I'm on one of two static IP address but understand the security/certification requirements.

Neverthless I have summarised the current problems I have encountered below:

Thanks for that... the details help..

Undeclared variables
--------------------
Notice: Undefined index: children in /home/store/public_html/includes/classes/class.froogle.php on line 1260
(snip)

its Good to declare variables and types. You should switch notices on during development.

Gotta point out that I'm writing this at speed while trying to keep 100 other plates spinning so it's very much hack and slash in places to resolve some of the issues and niceties like undefined indexes are something for tidying up later...

I'm also having to develop on a live site (a huge no but all I can do at present) so putting notices on is not a good plan in that environment.

Missing Images
/modules/shoppingcomparison/googlemerchantgben/images/*
Although I understand its a quick release, have you tried mapping categories and trying to figure out which categories are mapped and which aint without an image? Please make this available to we can add to the module

Hazards of using Firefox to test and work in... It doesn't show those broken images so I didn't even know the indication of mapped or not was even there. Quick test in IE8 and I see what you mean now...

See the attachment at the bottom of this thread for the missing images.

Check XML Structure

As you found... it's working... just some parser when downloaded and re-uploaded at fault it seems.

Missing recommended attribute: product type (3307 warnings out of 13831)
------------------------------------------------------------------------
Maybe to create a debug.log file where you can push any errors during the feed generation or log important details needed for debugging this issue. I cannot see the relation between the items which do not have a

Not sure what you were going to say at the end of that sentence but this is the one thing that is critical... Only thing I can think is that there's a timeout issue or something similar. It's a unique problem that really does need direct access to do some cross checking with...

Invalid UPC value (11 warnings)
-------------------------------
Do you truncate the UPC or just explode it? Seems like UPC = 12 characters and EAN = 8, 13 or 14 characters

The UPC is handled as follows:
1. Trim the UPC field value
2. Explode the remaining value using comma as delimiter
3. Test for an array of greater than one and use the first array entity if true
4. Reject any UPC codes over 13 characters (I missed that 14 was a valid value so that's a bug fix for next release - see below)

Bug Fix (allow 14 character UPC values)
Find:

Code: Select all

if(strlen($row['upc']) > 13) {

Replace with:

Code: Select all

if(strlen($row['upc']) > 14) {

[Martin]

Would be nice to know the process of how it "carries on from where it left off" though.

Pretty simple stuff really, just a very specific regex and read backwards function that looks for indications of a successful feed completion and if not there, the last successful product export.

The rest is down to the use of a temp file for feed generation and the same checks and balances above to determine if it's time to move the feeds about to bring the latest version live.

[Martin]

Undefined index: children in /home/store/public_html/includes/classes/class.froogle.php on line 1260 - key_exists() test applied
Undefined index: 839 in /home/store/public_html/includes/classes/class.froogle.php on line 1324 - (See below)
Undefined variable: row in /home/store/public_html/includes/classes/class.froogle.php on line 577 - incorrect index used - fixed
Undefined index: gmtaxonomy in /home/store/public_html/includes/classes/class.froogle.php on line 982 - key_exists() test applied


Thinking the issue on 1324 may be the source of our missing category /product-type problem although it could simply be a hidden category that is unset earlier in the code.

You may find that setting the value for $this->include_hidden_categories to true may resolve that error.

[Brandeline]

Hey Martin, when you mentioned a fix for the disappearing categories, was that supposed to be in 1.4.0 Beta 2? I downloaded it today and installed it to replace my 1.3.1. install, but I'm seeing very weird behavior in the interface to select the correct categories for the products from google taxonomies.

On the very first category "Live Animals" I get offered secondary categories starting with "Bird Supplies", fine so far.. but select Bird Supplies, I get everything that was supposed to be in the Cat Supplies section, like Cat Beds, Cat Collars, etc, and not the bird supplies child entries.

When I go down to Arts & Entertainment, my only choice is 'Wine Coolers'. Do you think this is likely to be an array index problem maybe? Seems to get worse the further down the list I go. (I'm your windows guinea pig if that may matter. Browser Firefox 6 on Windows 7, Hosted on Windows 2008 server with IIS 7.5 and Helicon Ape.)

Ideas?

Thanks!
Brandeline

[Martin]

I'll be honest, those new problems are pretty erm.. "unique"... It's almost certainly taxonomy related but not sure why it's loaded so weird.

You could try emptying the productcomparison tables in your DB and try disabling, then reenabling the GM module again..

What version ISC are you on btw?

[Brandeline]

I had a look at the categories stored in the database, and they look fine there at least at first glance. They have their proper parent categories assigned etc (Cat Collars is not stored under Bird Supplies in the DB).

I did empty the tables, turn it off, and re enable it as suggested. (I Emptied product_comparison, and Shoppingcomparison_categories both.) But it still behaves the same.

I'm using 6.1.1.

(Edit) ADDITIONAL: Just had a look at the taxonomy text files themselves on my filesystem and I see one difference. The ones that came with Interspire (Nextag, Pricegrabber etc) have UNIX line endings and ANSI encoding. The googlemerchant files are Dos/Windows line endings and UTF8 encoding. Could it be that simple?

[Brandeline]

That was EXACTLY what it was. Silly line endings and different encodings.

I resaved the taxonomy file with ANSI encoding and UNIX line endings. Now it works perfectly. Trust a windows system to notice where *NIX won't.

[Martin]

Good catch!! That's flipping silly but one to add to my list.. Will update with thenext release.