• Blog >
  • Fixing magento improving data flow profiles

magento-flow.jpg

We recently undertook a project to translate a Magento website from English in to several other languages.

As part of this, we needed to update a large number of products to translate the text for descriptions and attribute text.

This required extracting the relevant text from Magento to be provided to a third party translation company. We chose to use Magento’s built in Advanced Data Flow profiles, providing CSV files to the translating team, that we could then re-import into Magento.

On the whole, the system works well, however there are a couple of issues that we fixed along the way to help with the with the import process.

The first issue is that under each data flow profile, there is a history section, however a bug in the Magento code means that the relevant data is never recorded, so you will never see any history in here. 

Secondly when you run a data flow import through the browser, it will report any errors that occur. However, as this does not report line numbers, the output is not actually very useful when trying to track down what is causing the issue. We found that due to mistakes made by the third party editing the translation files, it was essential to have this information in order to quickly resolve issues with importing the data.

Fixing both issues requires overriding the Magento code, as always, this is done using a module placed in the ‘local’ code pool rather than editing the Magento core directly. 

The first issue is fixed with a change to the following file

app/code/core/Mage/Dataflow/Model/Profile.php

The problem is with the run() method, where the profile history is saved, the ‘userId’ field is never set.

The original code looks like this 

Mage::getModel('dataflow/profile_history')
   ->setProfileId($this->getId())
   ->setActionCode('run')
   ->save();

With fixed code looking like this 

$adminUserId = $this->getAdminUserId();

Mage::getModel('dataflow/profile_history')
   ->setProfileId($this->getId())
   ->setActionCode('run')
   ->setUserId($adminUserId)
   ->save();

With the getAdminUserId function added to the class as below:

protected function getAdminUserId()
{
   return Mage::getSingleton('admin/session')->getUser()->getId();
}

The second issue is resolved by editing the following file 

app/code/core/Mage/Adminhtml/Block/System/Convert/Profile/Run.php

This is the code that is used every time a profile is run, and handles the output to the browser.

The fix is to this part of the code:

$this->setBatchConfig(
   array(
       'styles' => array(
           'error' => array(
               'icon' => Mage::getDesign()->getSkinUrl('images/error_msg_icon.gif'),
               'bg'   => '#FDD'
           ),
           'message' => array(
               'icon' => Mage::getDesign()->getSkinUrl('images/fam_bullet_success.gif'),
               'bg'   => '#DDF'
           ),
           'loader'  => Mage::getDesign()->getSkinUrl('images/ajax-loader.gif')
       ),
       'template' => '<li style="#{style}" id="#{id}">'
                   . '<img id="#{id}_img" src="#{image}" class="v-middle" style="margin-right:5px"/>'
                   . '<span id="#{id}_status" class="text">#{text}</span>'
                   . '</li>',
       'text'     => $this->__('Processed <strong>%s%% %s/%d</strong> records', '#{percent}', '#{updated}', $this->getBatchItemsCount()),
       'successText'  => $this->__('Imported <strong>%s</strong> records', '#{updated}')
   )
);

Where the line starting ‘template’ determines the output that is used for reporting when importing a line of the file. 

The fix is to include the ID of the row in this template, meaning that for each error output, you will see the line that has caused the problem. The fixed line looks like this:

. '<span id="#{id}_status" class="text">Row #{id} - #{text}</span>'

with the whole call looking like this 

$this->setBatchConfig(
   array(
       'styles' => array(
           'error' => array(
               'icon' => Mage::getDesign()->getSkinUrl('images/error_msg_icon.gif'),
               'bg'   => '#FDD'
           ),
           'message' => array(
               'icon' => Mage::getDesign()->getSkinUrl('images/fam_bullet_success.gif'),
               'bg'   => '#DDF'
           ),
           'loader'  => Mage::getDesign()->getSkinUrl('images/ajax-loader.gif')
       ),
       'template' => '<li style="#{style}" id="#{id}">'
           . '<img id="#{id}_img" src="#{image}" class="v-middle" style="margin-right:5px"/>'
           . '<span id="#{id}_status" class="text">Row #{id} - #{text}</span>'
           . '</li>',
       'text'     => $this->__('Processed <strong>%s%% %s/%d</strong> records', '#{percent}', '#{updated}', $this->getBatchItemsCount()),
       'successText'  => $this->__('Imported <strong>%s</strong> records', '#{updated}')
   )
);

These fixes were tested in Magento Community Edition,  If you do choose to make these changes to your Magento installation, ensure you test them thoroughly on a development environment, as we can not guarantee the fix is applicable for all versions.

An example of how to build a Magento module to include this fix can be found on our Github page

● ● ●