"Loading..."

Opencart XML Product import

E-Commerce sites could be very different in terms of platform, concept, industry etc. Some shops are single-product sites and WooCommerce (WordPress E-Commerce Plugin) can be an easy solution. However, we don’t like it personally. In my opinion it really lacks flexibility, friendly management and performance. Considering you are going to sell a single product or a dozen of products, I would really go for Shopify or Opencart.

But what if you need to setup a shop having thousands of products updated daily via multiple supplier feeds of different types (CSV, XML, JSON etc)?

We’ll share our experience creating a sustainable XML Import for Opencart that works with large files, however this solution is suitable for any other platform since it’s a standalone import class.

It’s okay to try SimpleXML PHP parser when you have a relatively small XML file containing few hundreds of products. But if you need to load a bigger XML import file, let’s take, 1GB you would most likely exceed your server’s RAM.

You could consider something event-driven, like SAX parser, but there are way simpler options.

We went for a tiny class created by Christian Weiske (Thanks Chris!) called ProgramIterator. The base of this class is XmlReader with added iteration functionality.

ProgramIterator

class ProgramIterator implements Iterator
{
    /**
     * XML file path
     *
     * @var string
     */
    protected $strFile = null;
 
    /**
     * XML reader object
     *
     * @var XMLReader
     */
    protected $reader = null;
 
    /**
     * Current program
     *
     * @var array
     */
    protected $program = null;
 
    /**
     * Dummy-key for iteration.
     * Has no real value except maybe act as a counter
     *
     * @var integer
     */
    protected $nKey = null;
 
    protected $strObjectTagname = 'program';
 
 
 
    function __construct($strFile)
    {
        $this->strFile = $strFile;
    }
 
    public function current() {
        return $this->program;
    }
 
    public function key() {
        return $this->nKey;
    }
 
    public function next() {
        $this->program = null;
    }
 
    public function rewind() {
        $this->reader = new XMLReader();
        $this->reader->open($this->strFile);
        $this->program = null;
        $this->nKey    = null;
    }
 
    public function valid() {
        if ($this->program === null) {
            $this->loadNext();
        }
 
        return $this->program !== null;
    }
 
    /**
     * Loads the next program
     *
     * @return void
     */
    protected function loadNext()
    {
        $strElementName = null;
        $bCaptureValues = false;
        $arValues       = array();
        $arNesting      = array();
 
        while ($this->reader->read()) {
            switch ($this->reader->nodeType) {
                case XMLReader::ELEMENT:
                    $strElementName = $this->reader->name;
                    if ($bCaptureValues) {
                        if ($this->reader->isEmptyElement) {
                            $arValues[$strElementName] = null;
                        } else {
                            $arNesting[] = $strElementName;
                            $arValues[implode('-', $arNesting)] = null;
                        }
                    }
                    if ($strElementName == $this->strObjectTagname) {
                        $bCaptureValues = true;
                    }
                    break;
 
                case XMLReader::TEXT:
                    if ($bCaptureValues) {
                        $arValues[implode('-', $arNesting)] = $this->reader->value;
                    }
                    break;
 
                case XMLReader::CDATA:
                    if ($bCaptureValues) {
                        $arValues[implode('-', $arNesting)] = trim($this->reader->value);
                    }
                    break;
 
                case XMLReader::END_ELEMENT:
                    if ($this->reader->name == $this->strObjectTagname) {
                        $this->program = $arValues;
                        ++$this->nKey;
                        break 2;
                    }
                    if ($bCaptureValues) {
                        array_pop($arNesting);
                    }
                    break;
            }
        }
    }
}

The whole thing is very simple. You don’t have to load the whole large XML import file, instead you iterate through its nodes.

We used this useful class to import firearm data from multiple suppliers. One of them was SportsSouth. Each supplier had its own class since the data sets, attributes, structures were quite different and we had to standardise data imported from all 4 suppliers.

And then we start the processing by creating an iterating object:

        $items = new ProgramIterator($feed_file, 'Table');

Once the object is created you can run a regular foreach loop to process the import data.

         foreach ($items as $item) {

              var_dump($item); 

          }

As I have mentioned before, we had a task to import and standardise data from multiple suppliers, providing the same products. So the XML import itself was a tiny part of the whole process. We had to programmatically detect the best product picture and description among all the same products supplied by different sources. We also had a SEO product title generator, specific for firearm industry. Another tricky thing was unifying firearm attributes (sizes, calibers, materials). Basically, we had an idea to standardise and unify all the same products, by processing all supplier feeds, then choosing the best data from each and updating the imported product with this data.

Considering the above mentioned post-import processes that were actually a part of the whole cron task, the import of 130K products from 4 suppliers took around 30 minutes. I’m pretty sure it’s going to be much faster if your needs are less complicated.

Still if you need something very specific, we are always happy to help 😉

CONTACT US

We're happy to take your call Mon-Fri 9am-3pm UTC

Russia

+7 (985) 334 51 24

Europe
+371 20009500
Skype
incapitalgrp
©2018 ICG Commerce